As part of our annual commitments, everyone on my team is required to sit a Microsoft Certification exam...in other words, to "eat our own dog food." This year, I decided to take 70-646 because the new director for the Product Development Group in Microsoft Learning (my boss's boss) had taken this exam and had some very specific feedback that I wanted to better understand. Because I also commit to passing the exam that I take, I typically opt to take an Office exam to meet this commitment. So, this would be the first time that I would take one of our truly technical exams as part of our "eat your own dog food" requirement.
As some of you know, 646 is Windows Server 2008, Server Administrator Pro Level exam. It's one of our longer exams because it includes case studies and non-case study items as well as variety of item types, such as multiple choice, choose all that apply, drag and drop, build list, etc. Here's what I learned and what I'm changing as a result of this experience:
1. Forcing candidates to scroll to the end of an item before moving to another is irritating and annoying...really, really irritating, especially when you are essentially at the end of a question, but you haven't actually scrolled to the magical bottom that allows you to move to a different question. So... no more of that. You will NO longer be forced to scroll to the end of an item before moving to another. This functionality was designed to ensure that candidates read the entire item before proceeding, but I think I can trust you to be responsible to read the item...after all, your ability to answer correctly is clearly tied to whether or not you actually read the item in its entity. This was so annoying, in fact, that I'm expecting to hear a "woots," "hurrays." and other expressions of joy as people read this blog!
2. In the past, case studies were timed separately, meaning that you had to complete each case in the time allotted. However, if you were able to complete a case more quickly than in the allotted time, you "lost" that time (it didn't carry over to other cases or non-case study items). We timed cases separately to help candidates manage their time on the exam (i.e., to be sure that they didn't spend too much time on any one case at the detriment to the other cases/items on the exam). Although the time we provided was a data driven estimate designed to ensure that most could finish the case in the time allotted, some candidates had too much time for some cases and not enough time for other cases. No more. Cases will no longer be timed separately. You can take as much time or as little time as you want on a particular case and any remaining exam time can be used on other cases or non-case study items. Time will no longer be "lost." Again, I'm empowering you to be responsible in managing your time during the exam. Use it when and where you need to. Just remember...don't spend too much time on any one case (they will still be in separate sections so once you leave it, you can't go back) because there may be more items/cases on the exam. We do let you know how many items on remaining on the exam, but because you can't leave a case and come back to it, you'll have to manage your time carefully to be sure you have time to answer all the questions on the exam.
3. The instructions have been streamlined. We provide you the information that you need to know to take the exam...everything else can be found in our FAQs at http://www.microsoft.com/learning/en/us/help/self-help-faq.aspx (by the way, there's a lot of great information here; you got questions, we got answers. If you can't find it here, email me at email@example.com, and ask your question.)
4. The tutorial at the beginning of the exam, unfortunately, cannot be modified as it's something that Prometric owns and is included with the delivery of all their exams regardless of the client. As a result, it needs to include information about item types that Microsoft doesn't include on their exams or may not be the exam that you're actually taking. To prepare for the items that you may see on Microsoft exams, review our item type tutorial rather than the one at the beginning of the exam.
All of these changes are rolling out with the new user interface that Stephanie Jordan blogged about earlier this week. So, they don't apply to all exams, but if you're taking an exam with the new UI, these changes apply.
And, if you're curious, I didn't pass. I haven't used the technology, I did next to no preparation, so this wasn't a surprise...my exams work! Unqualified people shouldn't be able to pass. And, I am clearly unqualified.
Liberty, Thank you so much for "eating your own dog food." It is important for your team to see what the exam experience is like out in the field. On a related note, I recently took one of the new beta exams featuring color formatted code. Please extend my thanks to the team for that. It made reading the code segments so much easier.
the scrollingthing sometimes was a bit strange, even if there was only whitespace left you had to scroll down. *thimbsup*
but talking about scrolling. when i sit in the testcenter near work, they have 14" displays.
so one time i did the beta for sharepoint 2010 with that huge neverending texts, where you had to scroll and scroll...
so i asked that guys if they wouldn´t upgrade to a bigger screen like i saw in another tescenter.
the answer was, we do what prometric requires....
i also wrote that in the prometric surveys, nothing happens...
maybe you can
Agreed, 14'' screens are very annoying, but it can't be replaced with bigger displays because the Prometric Engine's requirements. Someone should push Prometric to remove this silly limitation.
Great post Liberty! After taking many exams, where both issues occured, it´s good to hear, that they will disappear. This will lead to a better exam experience. Thanks!
PS: I´m looking forward to your next dog food exam.
I love the post. I am glad that "scrolling to end of the item" is going away. One other annoyance that myself and other test takers have had is with the build, list and reorder questions. If you drag an answer in a box from the left and place it on the appropriate rectangle on the right, then the answer in the box automatically moves back to the left. Augh! That is so frustrating. The key is to barely place the answer in the box from the left barely on the rectangle on the right, not completely on the rectangle. After the second build, list and reorder question on the test, I usually figure it out. I have been in the Prometric center where test takers would get up and ask the test administrator for help in getting the build, list and reorder answers to stick. Please fix this.
Liberty, Great post. Great observations and glad you're taking action!
In regards to screen size, I recently took an exam that was on a huge, wide screen monitor with lots of room. But that was just a dangling carrot! The exam was designed for a 14" monitor and did not re-size itself to take advantage of the space. It took up less than 1/2 of the space available :(
I love, love the new types of questions that require 'hot spots' , "active screens" , and "drag-drops" but these require a lot of scrolling. This is challenging on a small screen area.
Just a bit more "dog food for thought" while you're still munching!