Based on some recent events, I find it necessary to post this as a notification to all those who take certification exams. This applies to certifications across different vendors, not just Microsoft.
During the opening screens of the exam, before you reach the actual question portion, a non-disclosure agreement (NDA) is presented. Like most exam candidates, myself included, the thought process is to get to the exam content so you can complete the exam, get your score, and get out the door and back to your busy work life. This is a normal and accepted behavior and I’m not advocating a change on that aspect.
What I am posting about is reading, understanding, and adherence to the NDA. If you are not sure what it is or what is contained in it, take the time on at least one exam to read it completely. Essentially, by clicking the Accept button on that screen, you are agreeing to a legally binding document. So, in a nutshell, what are you agreeing to?
In essence, you are agreeing that when you have completed taking the exam, you will NOT discuss with anyone, at anytime, in any forum, in any form whatsoever, what was covered on the exam. You are not permitted to discuss the content of the questions in any form. The only content you are permitted to discuss is what is publicly available in the form of exam preparation guides. The content on the exam is intellectual property and is owned by the exam provider such as Microsoft.
Exam developers like Cisco, CompTIA, Microsoft, Oracle, etc. spend considerable time, effort, and cost to create certification exams and the content that is required to adequately test candidates on the material necessary to achieve the certification. This effort helps to ensure that the certification you acquire when passing the exam(s) provides you with a valid measure of your skill and knowledge for that technology.
By talking openly on what the content of the exam is, you are in essence making it easier for those who come after you to pass the exam without really knowing the content. You also make it easier for the brain dump sites to get a head start on what the exam covers. By not discussing the content of the exam with anyone, you help to ensure the validity and value of the certification you just earned.
Certification providers include in the NDA the fact that if you are caught divulging this information, you can lose your credential and become banned from further certifications. They are within their rights to enforce this although I am not aware of Microsoft doing this in the time I have been here. If you value your certifications and want them to continue to hold value in the industry, you need to partner with your certification organization to help prevent piracy and ensure that those who achieve the certification, actually deserve it.
Comments and questions welcome.
Gerry
Perhaps you could post a copy of the NDA for people to peruse in advance instead of making thinly veiled threats?
While I personally don't have any problem with the language in the NDA I always thought it was EXTREMELY unfair that you never got to see the terms of the NDA until you sit for the exam. By that point you have already paid your $125 for the exam and passed the deadline for being able to cancel and get a refund. In that sense it is very much like the shrinkwrap/clickwrap software licenses that don't hold up in court because by breaking the seal/booting the PC you were agreeing to the terms of an agreement that you couldn't see until AFTER you had broken the seal or booted the PC.
What happens if you click "I do not agree" on the NDA screen? Do you get your money back? If not then it seems like there would be a strong case for the contract having been entered into under duress, considering that you would be forfeiting the test fee by not agreeing. And if your employer required you to earn and maintain certifications it seems like the case would be doubly so.
Here you go:
www.microsoft.com/.../exam-policies.aspx
www.microsoft.com/.../nda.mspx
This way you can see NDA before you take exam. In my memory I also have not heard of anyone being banned by Microsoft! But there are cases that people do discuss exam questions but after a non-formal warning they edit their posts and cease illegal activity.
Microsoft Certifications Program (MCP from here on out) biggest issue is not NDA breaches it is quality.
If MCP wants to mitigate brain dumps vary question shapes and change up wording and quantity of questions per objective. Group questions covering the same command/feature so you can avoid repeat questions. If I can get the jist of the test by reading a few brain dumps the problem is not breach in NDA it is in the Quality of your exam. If the breadth and depth of your questions increased Brain dumps would not be an effective means of studying. The assumption of your post is they are valid way of studying even though they may be cheating.
Why isn't the MCP program more successful/useful?
Well you are legally not allowed to talk about your experiences come exam day. So community around exams is discouraged as your post would suggest. So in short what you see as cheating I see as the community showing MCPs world view as wrong and not in keeping with reality. The only reason anyone would pass your exams is not to show skill level, as that is shown by reputation and portfolio, is to qualify their employer for certain benifits or status. In exchange they get a better job with better compensation. I would say if you want MCP to be successful step one is to realize your quality issue. Work on treating your customers as customers not criminals/children. And most of all improve your product so it is not brain dumpable. (quality, depth, breadth, quanity of questions in the question bank)
If I am able to pass your exam with a careful study of brain dumps then I either know the topics or your exam doesn't show anything beyond being able to memorize buzzwords.
How about taking words serious and start "Responsible Behavior" from yourself MSL?
JR Kincaid makes a great point about quality of the tests. An example (something that still irks me), there was a trivia on borntolearn, you answer the question right get a free voucher. The person who answered it right was asked, why'd you choose that. The answer to the question by the way was outdated. He commented he wasn't sure why that was the answer, but just remembers there was a similar question or topic when he studied for his exams, and that was the answer, so he went with that answer. Come to find out if I recall, somebody mentioned if you had seeked the first edition of a book, you'd have gotten the right (outdated) answer. However, most people referenced Microsoft's website, and got it wrong.
Some exam questions are similar to that, there's no rhyme or reason to the answer. You can study your butt off understanding and becoming an expert on the knowledge (and that's a good thing), but at the end of the day when there isn't any rhyme or reason to an answer, you study for answers instead of knowledge (and that's a bad thing).
On a semi-related note, what are the thoughts on using virtual labs as part of the certification process (like the 70-113 exam)? It'd be difficult to memorize the steps for each possible process that could be included on the exam, without understanding the material.
Also, what is Microsoft doing to enforce the terms of the NDA? I agree that when people violate the NDA, it weakens the validity of the certification. Personally, I'd like to know what y'all are doing (if anything) to go after people who post information about exams.
There was a live meeting "Exam Forensics and What Microsoft is Doing to Catch the Cheaters". Where NDA breach issues were briefly discussed also.
www.livemeeting.com/.../view
I also strongly favor virtual labs over "theoretical questions about practical things" .
Outdated questions are a major issue. Windows Server 2008 and Windows Server 2008 R2 are a prime example. Service Packs also alter the answers to questions. You have to 'guess' which answer is accurate based on how up-to-date the answers are. It's a bit sloppy.
If you can't talk about the exams after you've sat them, every trainer I have ever used breaches the NDA with their "Exam alerts" or "Exam Tips" during the training. Doesn't even MS Press do this? What are those exam tips if not to warn of exam content?
Not to say I haven't learned a lot while studying for MS Certification, I have, I really have. Some of this just seems a little silly.
Outdated and irrelevant questions are a particular bugbear of a friend of mine. He failed the 70-270 three times, and each time citied a particular little-used techonology as a large part of his failure. Yes, he might have broken the NDA by venting his frustration to me (although I had already passed 270, so the knowledge didn't help or hinder me any).
His frustration was quite understandable, since the books he was revising from (official MS press, though slightly old) didn't have more than a paragraph on it. As for "questions that reflect real-world working practice" how many times have any of us had to [deleted to keep in line with NDA]? Precisely never in 11 years. And do you see how stupid it is to restrict the discussion like this? The exams should not have to be protected by "security through obscurity".
Instead of 40-60 multiple choice questions, just make every exam 5-10 lab based scenarios, that would be encountered in the real world. A user who can't access a service/folder/file because of permissions, a firewall with incorrect configurations, specific subsets of information needed for a report etc.
Or as an interim solution, structure the exams like 70-285. Testlet scenarios that once completed, cannot be returned to, with smaller time limits. So you can flick between the questions in the testlet, but once you say complete, you can't go back. 6 or 7 testlets per exam would, I think, result in braindumps becoming less useful. Not useless, but less useful. Which is a start.
Because in it's current format, the exam actually rewards cheaters more than it does those of us who study and work with the technologies covered.
A paper-MCP in the UK can memorise brain-dumps, get the certification and bluff their way into a job with little effort. Adverts on TV boast of "A career in IT with an average salary of £37k" and have testimonials from people who say they were in unrelated jobs and now work as IT Directors.
I have encountered some of these paper-MCPs, and they seem quite happy to confirm that they did little more than memorise just enough to get through the exams. But their performance does not match what I would expect from someone who had earned the knowledge through working with the technology. Their managers agree, and their faith in MCP is reduced.
Often, those of us who take the exams without resorting to braindumps will encounter a question that seems to make no sense, or perhaps the answers all seemed unsatisfactory. I had one recently where two answers seemed equally valid. I read the question over and over to see if one of the steps could be implicitly included in the "things already done" section. Nope. In fact, the more I read it the more I was convinced that neither step had been done, because both were mentioned as being completed _for section A_ and the question asked how to resolve the problem for section B.
I answered by guessing, and apparantly guessed wrongly, because the black line on the report for that section (sideline: could the reports be more obscure? A bar chart with no numerical markings is not helpful) was all but a tiny bit full.
Now, I can't ask anyone about this with any specific details, because that breaks the NDA. But a braindumper has no such issue. They will go and get a braindump, read the answer _without any explanation_ and know for next time if they need a retake. I can't do that. Even by going through all the relevant material, I see the two options as being equally valid, unless the question is missing some information. And I know I am not alone in querying a question on an exam, but being unable to find an explanation for it.
So legitimate examinees are being restricted, whilst the braindumpers have no such issue.
Wow, some great comments and dialog. Thanks to all who have responded thus far. While I don't feel it is necessary to defend my position or Microsoft's position on the NDA, I would like to address a few of the comments so that you can understand my thought process, that of MSL, and to help clarify some points.
The first reply stated;
"Perhaps you could post a copy of the NDA for people to peruse in advance instead of making thinly veiled threats?"
At first, I was somewhat offended by this comment but then I spent some more time thinking about it. Then I went back and read my post to see what could have possibly triggered this comment. All I can find is the last paragraph where I mention that violation of the NDA could result in decertification and being banned from the program. So, if that is perceived as a threat, then it has been seriously misinterpreted. I say nothing more in the paragraph than what the NDA itself states about violating the agreement. It's informational. It's no more a thinly veiled threat than your local law enforcement agency broadcasting a commercial on television or radio telling you that if are speeding and get caught, you could face fines, license revocation, etc. It's making you aware of the regulations and potential resulting repercussions. You may also have noted that I stated to the best of my knowledge, we have not decertified anybody for violation of the NDA. Those that have posted on blogs etc. and have been contacted by us have merely indicated they were not aware it was a violation, apologized and removed the offending material. Microsoft took no action although we are perfectly within our rights to do so. We're not here to make enemies with this NDA and we don't enforce it at every opportunity.
I must admit a fault in the post because I should have posted the link to the NDA that is available on our web site so those who don't wish to take the time to read it during an exam can read it ahead of time. However, in defense of the statement about paying for the exam first and then seeing the NDA for the first time, the Prometric site links to the Microsoft certification pages which list the policies and NDA before you even get to the point of registering and paying for an exam. Thanks to Aigars for posting the links in the response.
To answer the question on not accepting the NDA, yes you will be reimbursed. If you do not accept the contents of the NDA, the exam ends which means you did not complete the exam. Prometric will work with the candidate to either reimburse them or offer a voucher for another exam if the candidate wishes.
For those of you who took the converstation to a quality issue, I'm more than happy to have that discussion with you as part of another post. Exam quality and how we develop our exams or what is coming in the future are great topics for future posts but they really aren't necessary here. Remember, this post is about awareness of the NDA for exams. We have opened our beta exams to a wider audience recently and as a result, there are a lot of new entrants into the certification process and they may not be aware of the NDA and what the expected behavior is after taking an exam.
I invite anyone who has an issue with our exam quality or processes to become involved in exam design and development. We use subject matter experts to design, develop, review and test our content. If you think you have the skillset necessary, let me know. Our new fiscal year is coming up which means new exams. You have the power to help us change what you deem to be wrong. Too many candidates make generic comments about how Microsoft answers are the only correct answers on the exam and are not real-world. These are unfounded allegations and I stand behind our processes 100%. I'm not saying we are perfect and that there isn't room for change or improvement. We are working on that every day, but what I am saying is simply this, if you don't know what it takes to create a certification exam, sign up, see what happens. Drive the change you want.
Now, I digressed into the quality issue rather than sticking to the subject at hand. The NDA.
Oh and I'm also curious about this statement, "How about taking words serious and start "Responsible Behavior" from yourself MSL?", posted by lololo. I can only take something serious when it is serious in itself. There is nothing I can do with this statement. How are we not behaving responsibly? If you want action on an issue, you must be clear what the issue is.
So, if you want to discuss the NDA and what is in context about that, continue to do so. You want to talk about exam quality, sign up to address the issues you perceive and we are more than happy to work with you to create better certifications for you and everyone else.
1. True, the current format rewards cheaters - I would love to see a 100% lab based exam. If you have a server in front of you things definitely flow better than trying to figure in your mind which strange option or sub-menu you have to right-click to perform a specific task. I recently took and passed 83-640 and loved the labs (although they ran super slow). I know I've failed some questions in the past because the answers looked extremely similar and i didn't have a server to validate the command or option.
2. Braindumpers are devaluing MS (and other) certifications ridiculously fast - Last year I interviewed several "MCSEs" that could not RDP to a servers console, have never heard of APIPA, FSMO or Round Robin. Amazing! My employer does not trust MS Certified candidates so everyone been considered for an IT position gets a "hands on" test on our test network.
3. Keep that pool of questions fresh. I know there is a cost associated with developing exams but don't let this be an excuse.
4. The saying is that there are 3 ways to do things: the right way, the wrong way and the Microsoft way. Some questions are too obscure.
Many employers realize one or more of the above and couldnt care less about certifications. It's time to do something. Look at the NCLEX (nursing exam) as an example. Look real hard and you won't easily find actual exams questions anywhere.
Hi John, I'd like to address your points, all valid too BTW,
1. We are working toward all performance-based testing. Currently there are many considerations involved in this such as server hardware, testing center hardware, network bandwidth, and the fact that candidates want a pass/fail as soon as the exam is completed. That's not currently possible with existing technology. We will get there, but we need to work through the kinks first.
2. I agree that brain dump sites and those who use them have devalued certifications. We are not sitting idle on this and are working to actively identify these sites and have them shut down. That is a daunting task as you might imagine. There is no magic jelly bean for this one.
I also advocate for employers to use their own onsite testing for potential candidates. It's the only way to ensure that candidate is the person you want working at your company.
3. We are creating new content for exams on a regular basis and seeding that into the exams. It's not an easy task either as there are more issues involved outside of just adding new questions. They need to be psychometrically validated to ensure that candidates are being tested fairly.
As for cost being an excuse, I'm afraid cost is a very influencing factor. This doesn't make it an excuse, it makes it a valid business reason and justification for decisions that impact the exams.
4. I will completely disagree with this statement. I will agree that exams in the past may have had that issue but not in the past few versions. The Microsoft way is not present on current exams. Subject matter experts from the industry design the exams, write the questions for the exams, and validate and review the items. Microsoft does not dictate what goes on the exam nor what the correct answers must be.
I can't comment on the NCLEX exam because I know nothing about it. I can tell you that the Federal Aviation Administration in the US has a set of exams for pilot certifications. They have a knowledge-based exam that is multiple choice and must be passed before a student pilot can take the practical test. They make all the questions available on their web site for download. No brain dump required because they are available anyway. Anybody can download the entire question pool consisting of 400 items. They don't provide the answers with the question pool however.
Student pilots can then spend their time memorizing or doing whatever they want to prepare for the knowledge-based written exam. Does this mean they are less qualified from a knowledge perspective because they knew exactly what was going to be on the test? The federal government doesn't think so but still, they are required to pass an oral exam with an FAA examiner and then take a practical flying test. The written test costs ~$100 and the practical is over $400 if you pass it the first time around.
My point is that you are correct when it comes to tech industry certifications and cheating. It is time to do something. We are doing something, although it's not always visible. It's also not an easy problem to solve. Microsoft hasn't solved it yet but neither has Oracle, Cisco, or CompTIA.
I just want to bring this (in my original comment) to MSL's attention and invite you to reply:
"If you can't talk about the exams after you've sat them, every trainer I have ever used breaches the NDA with their "Exam alerts" or "Exam Tips" during the training. Doesn't even MS Press do this? What are those exam tips if not to warn of exam content?"
@Gerry O'Brien "Responsible Behavior" means that you take responsibility for your own words and agreements. Exams starts with NDA where users are warned what will happen if they do not fulfill some rules - that is fine. Later is exam that ends with score report. Have you seen score report? There is PROMISE that one will get Welcome Kit soon. For about whole year you haven't taken any responsibility for your own words and promises. Your exams looks outdated not only in exam questions but whole process looks obsolete and forgotten. There are constant issues with Prometric, your MCP site, MCP exams on your sites and especially local MS sites. I have reported such things many times here - looks you haven't found global solution to keep your own informations up to your own changes in your own offer. This is really pathetic and irresponsible behavior. How can you expect that people will treat your word serious if you don't treat your words serious?