Previously on Lost… I mean “The Story of Exam Development,” we talked about the first three phases of exam development. In today’s installment, let me enlighten you on the next three phases of our exam development process.
Phase 4: Item Development The number of items to be written for each objective is based on the results of the blueprint survey. Items are written by external SMEs; internal Microsoft employees cannot act as item writers except in rare circumstances. Once written, the lead item writer reviews and revises all items to ensure that they are:
Items that meet these criteria are included in the initial item pool.
Phase 5: Alpha Review
The technical (or alpha) review ensures that the items are technically accurate, relevant to the real world, consistent with the experience level of the target audience, and map to the intended objective. The alpha review meeting includes the lead item writer, editor, and external SMEs. These experts become the primary arbitrators of any issues or concerns that are exposed. To minimize conflicts of interest, only the lead item writer can attend the alpha review; none of the other SMEs who participate at this phase have seen exam content prior to this meeting.
The reviewed and edited items are collected into a beta exam item pool; the beta exam is built and administered to an invitation-only audience in most cases. During the beta exam, each participant has the opportunity to answer and comment on the items in the pool.
Coming tomorrow on the final installment of this trilogy…the final steps of our exam development process. Find out what the mysterious final step is that doesn’t appear on graphic that was included in my initial post…
As always, let me know what your thoughts and any questions that you have.