You’ve probably heard—Microsoft Certification is twenty years old this year! That’s a lot of exams developed, a lot of exams taken, a lot of certifications earned. And time for a lot of improvements made to the exam development process.
The fundamental process is based on industry-recognized best practices, so its basic flow has remained pretty much the same from the beginning: define the content domain, develop the blueprint, write the items, review the items, run the beta test, finalize the item pool, set the cut score, and then publish and sustain for the life of the exam.
But, over these past two decades, we’ve made some big improvements in our item types and who is involved in the process:
Today
Back Then
External subject matter experts write items
Product groups wrote items or hired contractors to write items
More interactive item types are being added to exams; in fact, most new exams will contain a minimum of 3 different item types
We used mostly multiple choice items
You will not see true/false items on our exams (good T/F items are very difficult to write well and easy to guess)
A few true/false questions were sprinkled through our first exams
We’re pilot testing a short answer “fill in the blank” code snippet item type on our code based exams
Our very first exams had a few short answer “fill in the blank” types of questions but these were removed because of scoring was extremely complex
External subject matter experts review items
Items were reviewed by internal Microsoft employees
Beta exams are free
Beta exams cost $50
External subject matter experts set the cut score
The cut score was set by internal Microsoft employees
You can see there’s been a big shift from relying on internal Microsoft employees to working with external subject matter experts to develop exam content. In fact, today, the product group is rarely involved in exam content development. This means we’re testing the real world use of Microsoft technologies and not the “Microsoft way.” Our process today is aimed at developing exam content that tests relevant, practical usage, not how the product group hopes the technology will be used.
And another change that you may have noticed is that we’ve become more transparent about these processes. We’re proud of our exam development process—and when you’re doing something so well, why not tell the world?
So, we’re always looking for ways to shed some light into what has been a black box in the past. What parts of the box remain dark for you? What do you want to know about our exam development and sustainment processes today? Tell us what you think.
I would like to see the "Number of MCPs wouldwide" side again - why it was removed?
I would like to know how many people pass by the first time ....
Richard
Liberty, thanks for continuing to keep us up to date with the process. For so many years, this WAS a mysterious "black box"! I think, It helps add industry value understanding the rigorous process that exam development goes through. Keep the articles coming!