The Truth About Scoring: 700 Does NOT Equal 70%

The Truth About Scoring: 700 Does NOT Equal 70%

Liberty Munson (Microsoft)

I know…it’s been a while since you’ve heard from me. You’re probably wondering what happened to me. I don’t know exactly. I could venture a guess, but it would only sound like a rationalization. (For those of you who have been paying attention, that’s a line from one of my favorite movies. Anyone?)

Now, that I’m back, let’s talk about something that’s near and dear to your heart and mine—scoring. It is my mission to correct a common misperception about our scores…700 is not equal to 70%. Let me explain.

All MCP exam scores are scaled. This process converts the number of points that you earned on the exam—in most cases, this is equivalent to the number of questions that you answered correctly (this is called a “raw score”) to a common scale that in Microsoft’s case ranges from 0-1000. Scaling scores makes it easier to compare scores from one administration to the next. When we make changes to our exams, the raw cut score (and percent of items that you must answer correctly in order to pass) changes because the cut score is based on the difficulty of the content on the new version of the exam. With scaled scores, you can compare your performance across retakes; if we provided raw scores or percentages, it would be very difficult for you to know if your performance had improved from one attempt to the next if we had made a change to the exam between your attempts.

As an added benefit, scaling scores makes it much easier for us to communicate the passing score for our exams—a score of 700 is required to pass any MCP exam. I have to manage ~125 live exams, and if someone asks me what the passing score is for a particular exam, a common cut score across all our exams simplifies things. What’s the cut score for 298? 700. 350? 700. 640? 700. 680? 700. By the way, scaling scores is not unique to Microsoft; most certifying organizations provide scaled scores rather than raw scores for these reasons.

Here’s the key that bears repeating—700 is not equal to 70%. I hear this a lot… I need to answer 70% of the questions correctly to pass the exam. This is not true; in fact the actual percentage of items that you have to answer correctly varies for exam to exam and can range from roughly 50% to 85%. I can’t tell you what the range is exactly for our exams or what the cut score percentage is for a specific exam, but I can tell you that it varies—on some exams you need to answer a higher percentage of items correctly to pass than you need to on other exams. The cut score for a particular exam is based on input from SMEs, the minimum qualifications for competency, and the difficulty of the item pool. Carrying this one step further, this also means that the score that you see in the score report is not the percentage of items that you answered correctly (unless you answered all the questions correct (1000=100%) or incorrect (0=0%), but these are the only exceptions). It’s simply an indication of your performance in relation to the cut score.

Are you sensing that this is a pet peeve of mine? Why do I care so much? I care because Microsoft doesn’t arbitrarily decide what the cut score should be on an exam. We set the cut score based on input from SMEs who help us determine the point at which minimal competency is demonstrated given the target audience for the exam. Every step of the exam development process is driven by SME input; nothing is done arbitrarily, including setting the cut score.

Now, I’ll step off my soap box. What else do you want to know about how we set the cut score or report scores? Ask away. And, for those of you who recognize the movie quote from the first paragraph, you’ll earn my respect (which is actually a modified version of a line from a different movie). :)

Comments
  • Anonymous
    |
    Question from a MSL employee :) if I got 500 out of 700, does that mean I got 71% (500/700*100) of the passing score?
  • Anonymous
    |
    In your infinite polling capabilities.. have you ever seen a score come across with 0% or a score of 0 out of 1000? That.. frankly I d love to hear if it exists! I know we ve all seen and heard about scores of 1000 at times :)
  • Anonymous
    |
    Hi Erwin, There is not a consistent mathematical relationship between the scaled score and the percent of items that you answered correctly. The conversion process between the raw score and the scaled score depends on where the cut score is set and the number of items on the exam; both of which vary by exam.
  • Anonymous
    |
    Hi Christopher, I have seen 0 s before but only when we are testing to be sure that scoring is working correctly (this is something we test before we release any exam) or when there is a problem with exam delivery. I have never seen a 0 for someone who was taking the exam legitimately. It s very hard to get a 0 on a multiple choice exam given that possibility of guessing correctly; as we move to performance based testing, my guess is that 0 s will be more likely (although still uncommon). :)
  • Anonymous
    |
    Must be: Grosse Pointe Blank http://www.imdb.com/title/tt0119229/ Tagline: Even A Hit Man Deserves A Second Shot! Now that s funny! Cheers, --Jason van Haard
  • Anonymous
    |
    I d forgotten that was the tagline. That is perfect!! (I m still laughing.)
  • Anonymous
    |
    Hehehehehhehehehe ‘Even A Hit Man Deserves A Second Shot!’
  • Anonymous
    |
    Actually Liberty, Erwin s correct and that is a legitimate mathematical relationship regardless of the model. if someone got 500 but needed 700 to pass they did get that percentage of the passing score - there was no mention of number of questions answered in Erwin s post. Sorry I get a bit pedantic at this time of the morning :)
  • Anonymous
    |
    Hi Peter, You re correct. I misread Erwin s comment. To be clear, this calculation represents the PERCENTAGE OF THE PASSING SCORE, NOT THE PERCENTAGE OF ITEMS ANSWERED CORRECTLY. But, this calculation doesn t tell you how many items you answered correctly or the number of items you "missed" passing by. In Erwin s example, he missed the passing score by 200 scaled score points or 29%. This might be 1 point or it might be 10 points. The answer to this depends on the number of points on the exam and where the raw cut score is.
  • Anonymous
    |
    why is it not possible (or allowed) to see the incorrectly answered questions after the exam is scored? It would be somewhat helpful....
  • Anonymous
    |
    Hi Gregor, Although we have large item pools, they are not unlimited. As a result, it s possible that you will see some of the same questions again if you have to retake an exam. We want to be sure that you pass the exam because you know the content area rather than because you remembered the questions that you missed on a previous attempt. Additionally, it’s a security risk to expose content in this way. This practice is not unique to Microsoft; most certifying bodies follow similar procedures.
  • Anonymous
    |
    Finally, someone to tell everyone what I always said (and people didnt believe me :) 700 != 70% But 1000=100%? Not necessarily. I once made 1025 on 70-218.
  • Anonymous
    |
    Liberty Prime, you have no idea how much you have crushed my spirit with this brutal truth about passing scores. Oh, I ll still do my sad little division ritual before each exam, superstitious as I am. But I know that a little part of me will die inside each time when I do, vainly trying to fool myself into believing it matters, and that 70% does = 700.
  • Anonymous
    |
    Hi Olavo, The process we use today for scaling scores (with a range from 0-1000) wasn t the same process we used to scale scores when 218 was published. All of the Windows Server 2000 exams (and those that were created at about the same time) used a different process for scaling scores. But, I promise you, that if you take an exam today and get 1000, you answered all the questions correctly. :)
  • Anonymous
    |
    Hi Chris, A little piece of me died when I read that I ve crushed your spirit. [Insert evil laugh here.] Seriously, as a test taking strategy, it s probably still useful especially if you are running low on time or are really struggling with particular items; just recognize that it s not perfect.
View All