A few months ago, we proposed a new item type--progressive case studies--to the community and asked for your feedback. We created a video describing this new item type and asked you for feedback via a brief survey, and the results are in!
As a reminder, progressive case study is a variation of the case study item type in which the case builds progressively. More resources are introduced to candidates as they answer questions. The progressive case study scenarios will include information in a format that reflects how candidates might receive it on the job. For example, the case may include emails, graphics, exhibits, and documents that candidates need to review and synthesize in order to answer questions.
So, what did you think? Highlights include:
Several respondents also expressed some concerns that:
Given the generally positive feedback on this item type, we expect to pilot test it sometime in 2014. Thanks to everyone who provided their feedback!
The one thing that I have to say about this (and I included it earlier in the survey) is that the more info that you put into an exam question, the more important it is that people review the question for clarity, especially clarity of language. I have taken many beta exams over the years and have seen more than my fair share of poorly worded or unclear/imprecise questions. I've even seen a few such questions on live exams, and I always make it a point to comment on them when they are encountered.
I have often felt that during the process of writing exam questions that many of the question authors have read, re-read, and discussed the question so often that the intent may be crystal clear to the community of authors, but unclear to someone who is reading the question for only the first or second time. I have noticed this problem particularly with Microsoft exams, and not so much with exams that I have taken from CompTIA or VMware. I think that you would be well-served by having another wave of review, before beta, that focuses purely on the clarity\comprehensability of the questions.
I think the telling number is 47 %. Less then half the People found it effective.
And as long as clarity of writing is an issue in each and every exam I take, this is a bad idea.
Do you know how the current exams are scored? If you miss a question in the case study, do you lose points for that question or the entire case study?