Loading ...Have you ever wondered how we decide which tasks will be covered by a specific exam? Or who writes the exam questions? Or how to find out what the maximum seat time is before you get to the test center? Here’s your chance to find out.
The nice folks on the MSL Community and Evangelism team have invited the Certification Development team to do some guest appearances on this blog. We’ll answer your questions about how MCP exams are created, delivered, scored, and secured. We can’t answer everything, of course—some things are just, well, secret. But we’ll at least tell you what we can’t tell you.
So ask away…
Posted by Krista
Comments
Tuesday, February 17, 2009 11:56 PM by Steve Maier
# re: Know the score: Demystifying MCP exams
I’ll start with a few questions. When you are going over a beta exam to figure out what questions are included and what ones are not, what critia do you use? Do you look at the background of the people that take it or just use the answers to decide? How much do you look at the comments that people put into the exam?
Wednesday, February 18, 2009 12:55 AM by Krista
# re: Know the score: Demystifying MCP exams
Great questions, Steve! Since you’re the first to respond, you’re first in line for answers. Watch for posts about the item selection process and beta exam comments over the next couple of weeks.
Wednesday, February 18, 2009 4:24 AM by wicky
# re: Know the score: Demystifying MCP exams
I have only one question on MCP exams. That is how people get 1000 out of 1000 without knowing at least the course outline.
Wednesday, February 18, 2009 7:37 AM by Edward Laverick
# re: Know the score: Demystifying MCP exams
OK here’s a few from me…
What do the percentages refer to on exam outlines? Are they the number of questions to expect on each subject or the weighting for each question?
Possibly related to the previous question… How are the scores calculated to be out of 1000?
In simulations are you penalised for taking a long route to get to something or is it just the end result that counts?
Are any of the multiple choice questions awarded partial marks or is it just full marks for a correct answer?
Wednesday, February 18, 2009 8:28 AM by Jesse Schuchart
# re: Know the score: Demystifying MCP exams
Good day to all.
I have worked for a CPLS and Prometric Center. I have seen people score 1000 out of 1000 and complete the exam in 30 minutes. Test King and others are still in business, which destroy the credibility of the certification. Are you working with Prometric to review scores, time to take exams, number of exams taken in a day, week, month, in order to find, investigate, and eliminate brain dumpers? The problem does not seem better, but worse. In 2006 (http://arstechnica.com/microsoft/news/2006/08/5013.ars), this was supposed to be fixed, and yet it is still out there with the 2008 products.
Wednesday, February 18, 2009 8:33 AM by Jesse Schuchart
# re: Know the score: Demystifying MCP exams
Related to scoreing:
More points for sims vs. multiple choice?
Do all multiple choice questions have the same point value, or are some weighted with a high point potential?
Are sims graded as correct or incorrect, or is partial credit offered? Penalties for taking too long, number of clicks, counted clicks, etc? Lots of students have heard urban legends about sims.
Rephrasing Edward;s question – give choose 2 or more for multiple choice, if I miss one of the choices, is partial credit given for is it all or nothing?
Wednesday, February 18, 2009 9:30 AM by Aaron N.
# re: Know the score: Demystifying MCP exams
How do you sign up to write some of the questions for a new exam? I had the honor last summer of being one of the people chosen to do the review on the 70-451 exam questions and REALLY enjoyed the experience. So how do I get on the question authors list?
Wednesday, February 18, 2009 10:42 AM by F Russ Trated
# re: Know the score: Demystifying MCP exams
How do you live with the choice of Prometric as a provider? Pearson Vue were so much easier to book exams with, find test centers by ZIP code, and deal with any issues such as interrupted tests.
Wednesday, February 18, 2009 10:48 AM by Rachel Jones
# re: Know the score: Demystifying MCP exams
I would like to know:
1) the overall percentage of test takers who pass an exam the first time, second time, or > than two.
2) Is there a difference in this percentage between IT and developer exams?
Thank you for addressing our questions.
Wednesday, February 18, 2009 11:26 AM by Dana
# re: Know the score: Demystifying MCP exams
Hey Jesse – I’d love to hear some of the urban legends around simulation exams! Do share!
Wednesday, February 18, 2009 2:12 PM by Michael Dragone
# re: Know the score: Demystifying MCP exams
I have the same question as Aaron.
Wednesday, February 18, 2009 4:32 PM by Will Harper
# re: Know the score: Demystifying MCP exams
All questions – No answers!!
Don’t waste my time.
Wednesday, February 18, 2009 4:54 PM by Jesse Schuchart
# urban legends around simulation exams
Hey Dana. As well all know, the exams are like Fight Club. Rule #1 – Don’t talk about what is on the exams. Rule #2 – DON’T TALK ABOUT WHAT IS ON THE EXAMS.
With that said, my primary questions regarding scoring are the same:
More points for sims vs. multiple choice? What is the rough spread?
Do all multiple choice questions have the same point value, or are some weighted with a high point potential?
Are sims graded as correct or incorrect, or is partial credit offered?
Are there penalties for taking too long, number of clicks, counted clicks, etc? I hear this all of the time as questions, but I do not have a real answer for them.
Wednesday, February 18, 2009 5:20 PM by NiallMerrigan
# re: Know the score: Demystifying MCP exams
Why oh why on the Developer exams do we get order of params as a type of question because of intellisense!!
Anyways thats enough a rant!
Wednesday, February 18, 2009 7:26 PM by Krista
# re: Know the score: Demystifying MCP exams
Wow, that’s a lot of questions!
Just to clarify, we’re not going to answer all these questions in the comments for this post. An answer to each question or topic (such as how sims are scored) will appear as a new post sometime in the next few months.
Regarding Jesse’s comment about Fight Club, that’s hilarious. Truth is, we are going to try to be as transparent as possible in our answers without jeopardizing the MCP program. Some of these questions I don’t know the answer to, but I’ll try to find out. Some of these questions I do know the answer to, but I have to figure out how much I can tell you without getting, um, fired. Some of these questions we’re just going to tell you that we can’t tell you.
So thanks for your great questions, and thanks also for your patience as we get you the answers you’re looking for.
That said, here is one answer that I can provide to part of one of Jesse’s questions off the top of my head: There is no per-simulation-item time limit. There is an overall time limit for the exam, and for some exams the simulations are grouped in a separately timed section. However, you will not be penalized for spending a long time on one simulation. Either you complete the task successfully and you earn the point or points associated with that simulation, or you don’t.
The only disadvantage to spending a long time on one simulation is that you are eating up time you could be spending on another item.
Keep the great questions coming.
Wednesday, February 18, 2009 7:38 PM by Michael Dragone
# re: Know the score: Demystifying MCP exams
I thought of another one: Which exams these days have simulations? I haven’t taken a test in awhile; they were on 70-290 and 70-291 at first and then 70-293 and 70-294. I thought that they weren’t on any of the New Genny certs yet.
Then again I recall a beta exam awhile back to test a new simulation delivery method for the New Genny certs. I didn’t get a chance to take it, but I vaguely recall it being based on 70-620.
Thursday, February 19, 2009 3:41 AM by Edward Laverick
# re: Know the score: Demystifying MCP exams
Michael: the beta exam you’re refering to was based on 70-640 and was briefly released as 83-640 though I notice that’s now fallen off Prometrics lists.
Thursday, February 19, 2009 11:03 AM by libertymunson
# re: Know the score: Demystifying MCP exams
I’m Microsoft’s psychometrician (more on what that is in a future blog), but for simplicity, let’s just say I’m responsible for the ins and outs of our exams from development through maintenance once the exam is live. You’ll probably hear from me a lot because I love this stuff.
Let me quickly address the comments about the simulations because both Michael and Edward are correct in some ways, and I want to clarify this for other readers. Simulations appear in 290, 291, 293, 294, 431, and were recently added to 620 (more exams are likely to follow). We also have an exam that contains emulation labs–83-640. This version of 640 is only available in certain countries, but more countries are added monthly as test centers demonstrate that they meet the minimum requirements to launch the labs. We did a worldwide beta of this exam late last year to get feedback on the labs. This beta ended on December 31. Because the live version of this exam is in "limited release," it appears to have "fallen off" Prometric’s list of exams, but if you’re in one of the countries where it’s available, it’s still on the list.
The difference between simulations and labs is simple. Simulations imitate some, but not all, of the product’s functionality during the exam; in labs, on the other hand, candidates access one or more virtual machines that have the software product installed on it. As a result, nearly all the functionality that the software has in a normal environment is available during the emulation. Pretty cool, huh? If you had a chance to try out the beta, you know this; if not, wait until you see them! We will be adding these types of items to more exams over the next few years.
Liberty Munson
Microsoft’s Psychometrician
Thursday, February 19, 2009 12:43 PM by Michael Dragone
# re: Know the score: Demystifying MCP exams
Very cool, Liberty! Try and get your own TV show with a title like that. "When no one else can, they call in…"
Like I said, I didn’t get a chance to take the beta last year (and I’m not sure some of the test centers around here have test computers that meet the "minimum requirements" to launch Windows 98, let alone a VM running 2008) but I’ve been a big proponent of adding more and more sims/labs to the MCP program.
Thursday, February 19, 2009 4:06 PM by Edward Laverick
# re: Know the score: Demystifying MCP exams
Thanks for the info Liberty, the new emulation labs do sound very cool 
Thursday, February 19, 2009 4:41 PM by Elias Markelis
# re: Know the score: Demystifying MCP exams
Are the Developers going to see any simulation exams
It is not very nice to have questions with 10-20 lines of code and after that an answer wich has to do only with the syntax.
Don’t forget we have Visual Studio to check the syntax for us.
Thursday, February 19, 2009 5:45 PM by Jesse Schuchart
# And Liberty is Super Brilliant Smart!
So, I am going to say we all need to bow before the alter of Liberty.
Her Bio:
Liberty Munson is the Psychometrician Program Manager for Microsoft Learning. A psychometrician is a person who practices the science of measurement, or
psychometrics. The term psychometrics refers to the measurement of an individual’s
psychological attributes, including the knowledge, skills, and abilities a professional
might need to work in a particular job or profession.
She has been working for Microsoft since January 2007 and is responsible for ensuring that psychometric standards are rigorously applied in all phases of exam development and maintenance.
Prior to joining Microsoft, she worked at Boeing for seven years. At Boeing, she developed a wide variety of selection tests, including multiple choice and written exams, team based exercises, and problem solving activities as well as structured interviews and interviewer training, assisted with the development and analysis of Boeing’s internal certification exams, and acted as a co-project manager overseeing the development, administration, and analysis of their Employee Survey.
She received her BS in Psychology from Iowa State University and her MA and PhD in Industrial/Organizational Psychology with minors in Quantitative Psychology and Human Resource Management from the University of Illinois at Urbana-Champaign.
Thursday, February 19, 2009 7:31 PM by Andrew Storrs
# re: Know the score: Demystifying MCP exams
Hi Liberty, nice to see you’ll be contributing to the blog (I’ve always enjoyed the IS/SS sessions you’ve run).
Having you guys clarify some of the the processes here will make it so much easier to answer peoples questions "on the street", etc (since it will be clear what’s public and what’s still under NDA).
I’d love to see some posts from Jim C. too.
If you, along with him and others, can write about the exam development process as well (from deciding which exams to create, SME selection for question authoring, IS/SS, periodic reviews, etc.) that would be informative – a lot of people don’t understand the amount of work (and the cost to Microsoft) of exam development – and it might be a bit of an eye-opener.
Friday, February 20, 2009 10:46 AM by libertymunson
# re: Know the score: Demystifying MCP exams
Ahhh… It seems my reputation precedes me. I’m going to assume that’s a good thing. My intro blog about psychometrics will be posted soon. Stay tuned!
Our plan with this blog is to answer your questions about exams and to reveal “the man behind the curtain” when we can (although as Krista alluded to in the original post, there are some things that we simply can’t tell you). Anything we share with you on this blog is information that you can share with others. I think this will help clarify what is covered under NDA–if we can’t tell you about it, you shouldn’t talk about it if you happen to know the answer because of an exam that you took.
To the question from Elias about simulations on developer exams… This decision is made during the title planning process when we decide which exams to create. We will ask one of our planners to describe the title planning process in a future blog.
Keep asking questions! To be honest, though, I’m finding it very difficult not to answer them as soon as they are posted because I love talking about this stuff. Alas, we want to put the answers in blogs, so I have to control myself… I’m seriously getting sucked into the world of blogging!
