See what’s next! Check out the short answer code item type

Finally a step towards testing real skills, looking forward to seeing the feedback.

Nice and challenging. This would make it a bit more interesting and create more knowledge as guessing will be ruled out.

I just think that there should be provided some kind of "lab" where you could test/execute the code that you write, just to clear it of any spelling errors or other.

Krista

I think the concept of having to do "real world" exercises in exams is a great and noble idea, but the implementation always scares me especially when the words "finite list of correct answers" is included in the announcement--in fact this terrifies me!

The reason is this, unless the query as written is executed against an *actual* instance of the database on the platform specified there are a whole lot of assumptions being made, none of which can be verified unless the actual query is executed on that platform as written. As you know, assumption is the mother of all stuff-ups!

As an absolute minimum (failing getting Prometric to get a database engine *actually* working in their environments), the exam interface should give confirmation that the query as written parses correctly--this would make the scenario more "real world" as all queries written in T-SQL executed in any environment will provide feedback from the database engine's parser, and will not execute if they cannot be parsed.

As you are probably aware structured query language (SQL) is based on mathematical set theory. The way a query is formulated is based on the relational algebra E.F. Codd (en.wikipedia.org/.../Edgar_F._Codd) developed in the 1970s. There are a whole bunch of SQL professionals who are coming up with new and novel ways to answer any given question using set-based logic--I am very much afraid that these people are likely to fail these questions, due to the "finite list of correct answers", when in fact their query will solve the problem as stated. This will have disastrous effects on the value of the qualification.

This idea reminds me of the attempts MSL has had using Flash simulations in exams. They too had a "finitie list of correct answers" and even though there might be several ways of performing the task as detailed, many ways of doing it were missing in the interface presented in the exam. Therefore, in my opinion, the question failed.

I have test-driven the interface and I like it--it works well (though I would like more room for writing a query). I have some constructive feedback--in order to be able to formulate a query to answer the question posed, you need to know the data type and nullability of columns in the tables and a sample of the contents of the tables themselves; without this information there are even more assumptions being made and more propensity for stuff-ups!

Krista, I will send you an email independent of this post in case you wish to discuss my comments further.

Regards

Chris

I'll assume the following is something that will be changed in actual exam:

Allow the candidate to write question in textarea instead of textbox control. Multiline container is helpful from candidate because that can read the whole T-SQL at the same time, and hopefully can catch any spelling mistakes in between. It'll also help when the answer grow long by allowing them to break it into segments of code.

What can I say, Chris Barker nailed it.  The notion of a "finite list of correct answers" when dealing with any kind of coding is a bad, bad idea.  There's just too much room for differences in technique.

The best solution (though you'll hate to hear it) is an actual simulation-based exam where you are scored based on the results/final state of the environment, not based on your input.  I know that didn't work very well before, but if you want to test real-world skills you really need to have a "real world" environment to demonstrate them in.

I agree with everyone who voiced concerns over the 'finite list of correct answers' bit. In practice it might very well be a large list, but still I have seen people come up with approaches that actually worked, that I never would have even considered. I think a lot of pronlems can be avoided by giving more specific questions that prompt the test takers to solve the problem in a very specific way instead of just any way that works.

Another approach would be to give a preliminary score based on just the finite list, but later once the answers have been send to Microsoft, they should be run against a real database to see if they give the correct result too. Then a final score can be calculated based on that. Any additional succesfull answers found that way can then be periodically added to the list of correct answers that the test centers use.

I am also wondering which other areas besides sql this approach might get used. I guess powershell and server core commandlines usage might also have a low enough number of correct short text answers to be tested this way, but not much else.

@Lars: I wouldn't even put PowerShell on the list.  If you've seen what has been done in the PowerShell Scripting Games, you'll see 200 people responding to a particular task and there will literally be 200 different answers that all work.  When you get down to it, writing tools/commands in PowerShell shares a lot of similarities with software development.  But more to the point, unless you're someone like Ed Wilson you don't have many of the cmdlets and their syntax committed to memory.  There are literally hundreds of "stock" commandlets available that come with the OS, and depending on what roles and applications you have installed there will be additional cmdlets available.  The ability to type "get-command" and "get-help" is pretty much the key to being able to write useful PowerShell code.

I guess from the comments above it should be clear that the textbox should be multi-line in order not to annoy the candidate unnecessarily.

I also agree with Chris Barker's view: the only way that the query's correctness can be verified is by actually executing it on a real database.  If a finite list of strings is used, a lot of correct answers will fail.  I'll give a really simple example: what if, in my query, I use a table alias called "TheAliasThatNoOneEverBeforeCameUpWith".  I guess the problem should be clear...

Another remark, and that goes also for the current exams btw, is that questions should be clear. My first question was something like "create a list of customers who have open orders".  Based on the schema presented, I did not see any way to determine if an order is open or closed.  I did not see any Orders.Status field, nor a ShippingDate or whatever.  How can you create a query for open orders if nothing in the DB indicates that an order is actually open?

Last, small, remark: I think the time needed to answer this type of question may be a longer when compared to the multiple-choice questions.  Hope that will be considered as well?  It would be unfair for the time to run out just because the candidate had a lot of typing and proof-reading to do...

To end on a positive note: I am in favor of these types of question, given that the comments above are taken into account.

Best Regards,

Valentino.

We are proud to announce that short answer code questions will soon be available in an exam near you