We're Listening! Results from the MCP Survey!

Liberty Munson (Microsoft)

Every year, we randomly select...well...a lot of MCPs to participate in a survey to gauge satisfaction with the MCP program overall, evaluate the Microsoft Certification Program relative to programs of other IT vendors, and determine areas for improvement. We recently received the results from that survey, and my history doing survey research at Boeing as well as at Microsoft has repeatedly demonstrated that people are more likely to respond to surveys if they think something will change based on the results. And, if you've been following my posts, you know that I take this stuff seriously and that I really do value your feedback and will leverage it to make improvements to our exams and program.

Here are some highlights that I found particularly interesting:

  • MCPs obtain certification to enhance their career opportunities (skill building and updating) and distinguish themselves from other.
    • What was particularly interesting about this finding was that MCPs from the Americas were more likely to earn certifications as a way to distinguish their skills while those in EMEA and APAC earned certification to build their skills.
  • The value of certification is driven by the job opportunities that the certification creates, which is not surprising...
    • More surprising and interesting to me is that you're more likely to value a certification if it gives you the skills to effectively identify and resolve technology-based issues. In other words, Microsoft certifications are valued by MCPs for effectively implementing, building, coding, testing, and/or deploying Microsoft technologies in organizations. We need to make sure that the skills that we assess are ones that will help you be more effective in helping organizations become more productive!

  • Hiring managers continue to use MCP certifications as part of their hiring process and use has increased over the last year. More important, the perceived value of our certifications among hiring managers has increased significantly over the last year as well.

Themes that we need to focus on to improve our program, including training and exams:

  • Respondents noted that our content (both training and exams) was not relevant because it was assessing skills that are not used in the real world. You want to be tested on features that you actually use on the job. Honestly, this will always be a challenge for a global certification program because our technologies are used very differently by different people and different organizations. I am curious about YOUR ideas on how we can make the content more relevant and real world for you given our global reach. Got ideas? Let me know!
  • Certifications need more 'real world' value in the sense that they help you obtain jobs and solve real world business problems. Any ideas on how Microsoft Learning might move the needle on this one?

We are going to do mini-variations of this survey each quarter, so we can be more proactive in identifying opportunities for improvements. YOU might be randomly selected to participate. So, make sure your email address is up to date, keep your eye open for the survey invite, and participate if you have the opportunity. I would love to hear from you!

  • Mark Grogan
    | |

    Hi Liberty,

    Responding to the question asked about Real World, what ever happened to the simulations? (I am thinking back to exams like 70-291 where you would configure a DHCP server)

    No doubt these could still be Brain Dumped, but putting that aside as another issue, it would allow for those of us that work day in and out with the product the chance to show that if we get asked to accomplish task a, b and c, that we know HOW to do it. Rather than 4 multiple choice answers where you have both chance on yourside and only need to know that yeah to configured a,b and c I use this tool.

    Being able to demonstrate this would (I believe) be more relevant, and help obtain jobs and solve real problems because you then have certified proof that you can actually do the task.

  • View
    | |

    I tend to agree more with MCPs in EMEA and APJ that certifications are mainly for building new skills.

    For associate/entry level exams,  the goal should be focusing on "book knowledge" which you can obtain from Microsoft training materials,  online help, and other documentation.  Those in my opinion should not focus too much on complex real world problems.  That's just not fair to junior engineers and consultants.  

    On the other hand,  professional or advanced level certifications should have more focus on real-world and hands-on experiences.  There should be ways to verify real hands-on problem solving skills.  Possible ways include a hands-on portion of the exam (like how's done in a competitor certification.),  or writing a custom solution that needs to be graded manually....etc,.)   Of course that could reduce the number of new professional certification headcounts.  But isn't the goal of professional certification to identify "professionals"  not junior engineers who have just been trained?

    Regardless,  I would not use certification as a primary factor for hiring someone.  They would always be a plus/bonus but should never be mandatory for people who already have good project experiences.

    For people new to the field having a certification would be a big plus, in addition to other credentials.

  • pdiamond
    | |

    My biggest gripe with the newer ITPro exams is the focus on the command line and PowerShell. I understand these technologies are being used more and more by larger organizations, but the problem is that it switches the focus of the learning from high level concepts and theory to low level memorization of the syntax of the commands.

    In the real world it is important to know and understand, for example, the reason to have multiple DNS servers, different types of zones and when to use them, and the different types of records a DNS zone can have. These are the types of concepts the exams should be focusing on.

    Questions that require you to memorize  whether the DNSCMD command uses /<command> or -<command> or memorize whether the command to delete a zone is DNSCMD /zonedelete or DNSCMD /deletezone are a pointless waste of time and place the focus on memorizing the syntax of hundreds of commands rather than putting the focus on the appropriate situation to use the command. Understanding and mastering the high level concepts allows an IT professional to use their knowledge and skills in any environment, regardless of whether they are using the GUI or the command line.

    If I work with the command line or PowerShell every day, I will learn the syntax of the commands I use on a regular basis. For the infrequent tasks, I can easily look up the proper syntax.

    It is far more important to know WHAT needs to be done and WHY, rather than knowing the proper syntax for a PowerShell command but not really having any idea what it does or when would be the proper time to use it.

    Please keep the exams more conceptual and theory focused as opposed to focusing on syntax memorization.

  • dmsaxon
    | |

    I agree, wholeheartedly, with pdiamond.  As I have fed back on every appropriate exam question, expecting students to remember in an exam situation which PowerShell cmdlet or switch option is necessary to achieve a given goal is completely ridiculous.

    PowerShell, as per it's designers own comments, is meant to be *discoverable*.  As such, you can use tab completion, get-help and other such features to work out exact syntaxes.  Except, in an exam, you can't.  As such, it's not even simulating real world - it's even harder, with a simple 'pass/fail' mentality on whether or not you can remember the exact syntax.  As I mentioned earlier, completely ridiculous.

    Similarly, a question on an (granted beta) exam I took a few years ago, asked me which switches msiexec used to start a quiet installation.  The four answers had four very similar command lines, with only /i vs /x, and /qb- vs /qb.  This doesn't test understanding, it tests recall.  I don't think I'm alone in thinking that this devalues the Microsoft exams.

    Other thoughts in line with the initial requests:

    * If MOC courseware does not cover specific elements of the exam, then it should mention this.  I'm not sure if this is commonplace, but (I think) it used to be the case that the courseware stated which exam objectives it dealt with, and provided guidance where a student could find the missing elements.  I know there is a view that states that the courseware / classroom learning isn't enough, and I broadly agree.  However, not being told what the courseware doesn't cover is disappointing, I think.  I'd also comment, as an employer, why would I send my employee on a course that doesn't teach them enough to actually do the job I want them to do.  I think this proves that I'm in the EMEA side of the survey - It's very clear the US treats these exams differently.

    * I don't have enough experience of the other cultures to talk about a global exam can address all needs.  I would say that, in line with the other posters here, the purpose of exam should be about understanding the concept, not the specifics, at least for the more 'senior' exams.  As such, asking us to recall anything that is broadly available in the marketing material (for example, how much space each O365 mailbox gets on a certain plan) is unnecessary.  I'd go so far as to say that many of the exams I've taken recently could be vastly streamlined with less questions and more testing of understanding.

    * One other thought - and forgive me if this isn't the right place to ask - does Microsoft have plans to re-release the Architect exams?  As a consultant, I often don't get involved in the technical detail, but need to understand the bigger picture.  So far as I know, there are no Microsoft qualifications that I can gain that show understanding at that level.  Might be worth considering something 'technical' similar to the partner 'sales and pre-sales technical' exams?

    Just some thoughts!


  • abettany@outlook.com
    | |

    Interesting points made.  Can you share the statistical/smaple size information - ie how many MCPs made this type of comment regarding real world content is not on the exam?  

    Is this implying that the actual software is no longer relevant in the real world, the scenario, or the specific question?

    Personally I believe that the exam should test the ability of the test taker to know and use the software product. If there is a specific switch or parameter that can be set in the software then it must be there for a reason, and therefore it is fair game to be tested!

  • John Dennis
    | |

    I would have to echo the sentiments of many others on this and previous discussions and tell you that currently the exams are WAY to specific.  Though knowing all of the powershell commands and syntax is handy if you are an IIS administrator for a 10,000 server cloud hosting company, it's not very useful if you are 1 of 4 system administrators at a rural hospital in Michigan.  (And it's really easy to look up if you need exact syntax.)

    Any Microsoft Certified Gold partner that I've ever hired have ALWAYS had to look up the syntax and error messages for Microsoft products.  They don't know them off the top of their heads.  To be honest, in most cases I could have figured it out just as quickly as they did.

    At this point, you've made the exams so difficult that the only way to pass them is to use exam crams.  (And that's just wrong to me.)  Your target audience shouldn't be folks who are memorizing exam crams (only to forget it in month), it should be IT professionals who have the broad range of knowledge to take on anything that you throw at them!  

    I got my original MCSE back in the Windows 2000 days and have to tell you that the tests were much better designed then.  Since then I've passed several exams including several from Microsoft, and have only seen the MS exams become more and more "stuck in the weeds" of detail.  

    You need to test on knowledge of:

    -What the feature / role is

    -What the feature / role does

    -How the feature / role integrates with others

    -What the overall big win of using the feature / role is.  (This is really important as your best sales force is your engineers.)

    -From an implementation standpoint:

     -Any of the high level stuff is fair game (Typically the bolded stuff in a technet article.)

     -Any of the low level granular stuff is not (Typically the stuff that is indented in a technet article.)

     -Knowledge of what can and cannot be scripted is essential, but testing on syntax is prohibited.

    I don't think that any of us want these exams to be easy, we just want them to be both fair and attainable without the use of exam crams.   I work with a lot of college students at this point am discouraging them from pursuing any Microsoft certifications because of the issues described above.  

    Getting certified should be an enjoyable learning experience, not a miserable death march!  The goal should be to celebrate how much they've learned in the process, not chastise them for not knowing some random powershell command (was that New-IPAddress, or Set-IPAddress).

    Please let me know when these changes have made their way into the MCSE 2012 track.  Although I've passed 70-410, I'm going to hold on 70-411 until they do.  Meanwhile I'm going to pursue some Cisco certs...

    Thank you for your help in improving this process,



  • Anthony Piti
    | |

    I must agree with the comments made by "pdiamond", "dmsaxon" and John Dennis.  Rote memorization of PowerShell syntax is a needless distraction from demonstrating that a test taker understands underlying principles of the test material.  Having to memorize exact command lines & switches is pointless.

    One other question to pose is if Microsoft plans on making the System Center 2012 Configuration Manager exam (70-243) part of any MCSA or MCSE program?  As it currently stands, passing that exam gets you a MCTS, and nothing more.  Does Microsoft still plan on making the 70-243 exam a dead-end certification strategy?

  • Demyan1
    | |

    Since Microsoft Learning are listening, may I ask: why, taking 070-467, I cannot see the current question number, and have to keep count on my notepad to know how many questions are left? Is there a Microsoft Learning usability expert willing to step forward as the author of this bright idea?

  • Liberty Munson (Microsoft)
    | |

    Hi Dimitri,

    Can you provide a bit more information about your exam experience? Where did you take it? Prometric or VUE? Registration ID? Feel free to post here or send me an email: mslcd@microsoft.com


  • Samantha Lindsey-Ahmed (Microsoft)
    | |

    All question numbers are listed on the exam in the upper-left corner in large font.  It also shows how many questions remain.  



  • Demyan1
    | |

    I don't believe that is the case, Sam. Number of remaining questions is not shown, and the question number displayed is the within-section number, which is essentially useless. As communicated offline, I took 070-467 with Prometric, in the UK.

    PS. I continue to have trouble accessing this page: it seems that my login is "lost" (and I see "Access denied") along some navigation paths, but I was able to navigate to this page and leave a comment this time. I note that the issue was raised with your colleague Friday afternoon. If Microsoft is indeed listening, it should also master the art of the status update.    

  • Samantha Lindsey-Ahmed (Microsoft)
    | |

    We are looking in to the UK delivery and why the remaining number of exam questions did not show up for you.  Thanks for bringing to our attention!  

  • Demyan1
    | |

    A month has passed since the issue was raised - let's look at what has been done.

    1. An "unofficial" investigation was attempted, thanks to a very kind Microsoft employee, but did not really produce any tangible findings or commitments.

    2. The "official" route was explored as well - I wrote to both American and European regional service centers.  I have had no response from the US. The European service center did respond - ineptly and bureaucratically. What can be considered the definitive response came on Dec 23, and said

    Please be informed that since Microsoft exams are being conducted by

    third party companies the form of those exams are solely responsibility

    of those companies.

    Microsoft can  intervene only when there are obvious mistakes in

    questions or the exercises  in the test are plainly wrong.

    With regards to your statement if you have doubts or suggestions on how

    the exam should look like please kindly contact the testing center where

    you have taken the exam directly.

    I would summarize this as "if it's not an item challenge, go away". (Incidentally, my item challenges have been with the European regional center for 6 weeks now - unless the resolution arrives today, it is going to be yet another time they break their promise of a 4-to-6-weeks turnaround.

    3. The login issue persists.

    You know, dear Microsoft, you may be listening, but if you are not *doing* anything about what you hear…

  • Liberty Munson (Microsoft)
    | |

    Hi Dimitri,

    I hope you've found our email exchanges helpful. Feel free to reply to me there if you'd like more information or follow up.

    Happy Holidays!