Dissecting Your Score Report: What Does It All Mean? Part 2

Dissecting Your Score Report: What Does It All Mean? Part 2

Liberty Munson (Microsoft)

In my first blog in this series of posts designed to help you understand the information that we provide in your score report and how to use it, I walked you through the first section of the score report, including the bar chart showing your performance on each major section (functional group) on the exam. This post will focus on two additions that will appear on score reports in the very near future: a list of top 3 skills to priority when retaking the exam (on fail reports only) and a chart comparing your performance to others (on both reports).

If you have been following my blogs over the past few years, you know that I have been looking for ways to improve the value and usefulness of our score reports. About two years ago, I started the process of asking for your feedback on what you'd like to see in the score reports--what types of information would be the most helpful to you. Not surprisingly, most wanted more detailed information on areas where they can improve their performance as well as some idea on how their performance compared to others. After many iterations on what this might look like given some of our tools limitations, I finally settled on including a list of the top 3 skill areas to prioritize when preparing to retake the exam (this only appears on fail score reports) and a chart that provides a comparison of your performance to that of others who have taken the exam in the last 6-12 months. Let's look at each of these separately.

If you fail an exam (I know this never happens, right?!), you will be provided with a list of the top 3 skills to prioritize when preparing to retake the exam. It will look something like this:

These objectives (skills) are those where you had the lowest performance in terms of the percentage of questions answered correctly. If more than 3 objectives could be listed (because you answered the same percentage correctly across more than 3 objectives), the "tie" goes to the objective with a higher number of points. This is brand spanking new, so I don't have any "frequently asked questions" to address... but I suspect that one I'm likely to get is "why does this only appear on fail score reports?" Honestly, the rules for which objectives to list for those who pass became too complicated to implement, especially for those with high scores. There are too many scenarios in which it doesn't make sense to display anything in this table (think perfect or near perfect scores) or when it would be difficult to prioritize the objectives because the percent answered correctly is extremely high across many or most objectives.By definition, this is really more informative and useful to people who fail because they do have (more) opportunities for improvement than those who pass.

The second new chart that will be added to the score reports is a comparison of your performance to that of people who have taken the exam over the last 6-12 months. It will look like this:

The most important note about this chart is that it is provided for informational purposes only! Because it's a comparison of your performance to that of others, it may be unrelated to your overall passing status and to the information provided in the bar chart that I described in my first post. For example, you may have done well on a section (as represented by a longer bar in the bar chart) that is considered an "opportunity for improvement" here when compared to others who have taken the exam; alternately, you may have done poorly (as represented by a shorter bar in the bar chart) on a section that is considered a "strength" when compared to others here. Interpret this with caution! It is not intended to be diagnostic but to provide some perspective on your skills compared to others who have taken this exam.

Several other notes about this comparison:

  1. To oversimplify a bit, if you score below average on a skill area, it will be considered an "opportunity for improvement" and above average will be considered a "strength."
  2. If we don't have sufficient data to make these comparisons, this chart will not be provided on the score report.
  3. The data used to make the comparisons is updated as we sustain our exams; for some exams this happens more frequently than others. Ultimately, this means that if you retake the exam, the data used to make these comparisons may not be the same as for a previous attempt. As a result, the same level of performance could result in different comparison outcomes.
  4. This is brand spanking new as well and subject to change as we get more data and information about how we're categorizing your scores in comparison to others. Do the rules I have in place for these categorizations really provide enough differentiation for candidates to understand their performance in comparison to others? Only time will tell as candidates take our exams and their results are plugged into this chart. I will update the logic of these categorizations if needed as I learn more about the actual application of the rules that I've created.

Again, this chart is new, so I don't have any frequently asked questions to answer here, but I'm curious what questions you have now that you have a better idea of what this looks like and how it works. As you start to see this chart on your score reports, please let me know what you think and if you have any questions!

Comments
  • Luke Edson
    |

    Can we get this in tests we pass as well? It's not often I don't pass a test, but after the last two tests I took, where I thought I was well versed in the product, studied the same method as I previously had, I barely passed the test, and was left with a: "what did I do so dismal at?" feeling.

    I obviously need to bolster my training methodologies, but with no real knowledge as to where I really was weak, it's difficult to tackle that training. - I not only use the certification process to stand above my peers, but mostly use it to learn a new product, so just because I passed the test, that doesn't help me know where to focus my deeper learning efforts.

    Thanks!

  • EspiOne
    |

    I tested yesterday on the 70-640, I did not pass, but my exam report had "NO" bars or percentages on the test objectives,  I have done this in the pass, by making a copy of my reports and folding them and then folding them again to get the 25%, 50% and 75%.    But my last report showed "NO" bars.   Failing score was noted, top 3  skill to prioritized areas noted, but "NO" percentage bars.  I did not even notice a second chart.

  • hs.vpeter.hotmail.com
    |

    I tested on 70-414 couple of day ago, I did not pass, and my exam report had "NO" bars or percentages either.

    I think this must be a feature, not a bug.

    Why is Microsoft is so reluctant to give some feedback on our (poor) performance?

  • eroosendaal.computrain.nl
    |

    I guess Microsoft is just very concerned about questions getting all over the internet, They are worried about people specializing in 'taking the exam' rather than real world experience. I can well understand their concerns.

    Over almost twenty years I've passed many exams and my experiences tell me: if you really know your stuff you should always pass. I passed a few times when I suspected I might not, but never the other way around.

    It's just when your knowledge is on the brink (or worse) that questions appear ambiguous or unclear - and you may fail. And only then do you start worrying about the score report. But even if I do fail (it happens) I hardly glance at the report - if you're close to passing you are usually well aware of your weaker points. And you can usually remember a few topics which bemused you totally.

    Having said that: exams *are* a lot harder these days than (say) ten years ago. Well, only experts should pass, so I guess it makes some sense. There are the MTA exams now for people with only a few months of experience.

    Bottom line: score reports are close to irrelevant in my opinion. If you don't know your own areas for improvement it probably means most areas should be improved.

  • Liberty Munson (Microsoft)
    |

    Hi all,

    I'm looking into why the bar charts are not generating properly. stay tuned. Thanks.

  • Liberty Munson (Microsoft)
    |

    Hi all,

    We have figured out why the bar charts aren't printing for some candidates. We are in the process of fixing this to ensure that you get this information on your score reports. Given the root cause, this shouldn't be happening very often (it happens if you took an exam that has the new score report and is running an old version of IE and a specific setting isn't set properly). Thanks for bringing this to my attention!