It's easy to get into a discussion about quality. Everyone strives to deliver or wants be the recipient of the highest possible quality product, regardless of what it is - courseware, automobile, Internet service, t-shirt or sandwich. When I get into the discussions with educators, content developers, training partners and students, I've found that setting up the conversation by defining what we're talking about is one of the biggest challenges.
As you are most likely aware, perfection in anything is actually impossible, if only because what is perfect for you may not be perfect for me. Hence the need to provide guidelines, measures and, if at all possible, examples that can assist in the discussion. When the conversation is about courseware, this is really difficult sometimes, if only because some of the things that are part of a quality course are more art than science.
For example, doing a spell check and a grammar check is fairly simple....or so you'd think. If you type like I do (fast and with plenty of mistakes because I'm focusing on what I'm GOING to write instead of focusing on what I'm ACTUALLY WRITING RIGHT NOW) it's easy to type "that" when I mean "than". Spell check won't catch it, grammar check might not catch it and my eyes are unlikely to catch it. Unless I have an EXTREME editor go over my work, it will likely slip through. And that's supposed to be the easy stuff...
I read a lot of technical reviews of the Community courses on the Microsoft Learning Courseware library. About two thirds of them consist of some variation on "This course is OK" or "This course is horrible". Nothing else there; nothing actionable for the author to fix, so usually nothing happens.
Other reviews are quite granular and very, very helpful to the authors who use that feedback to put together the revision plan for their courses. There are other "middle ground" reviews that provide more feedback than "This isn't a good course", but not by much. The kinds of comments I read like this include (these are actual reviews):
....and so on. You get the idea.
In several large meetings with members of the technical training community this summer, we've spent a lot of time discussing the quality of Community courseware in particular and training content in general. Obviously, the better the working material, the more time the trainer can spend delivering a significant learning experience to the students and the less time she or he has to spend figuring work-arounds. Those conversations are the basis for the building of this foundational conversation on:
That's a start. I'll be posting in these areas and others throughout the year, but this is a conversation, not a monologue. Here is a good place to start.
I´m a MCT since 1999 and i can do some comparissons with the past and remeber NT4 materials (the best materials ever) and the evolution since Win2000 (a new approach, with LABs divided in tables where the lef is an "overview" and the right side, more "step-by-step style), WIn2003, Windows Vista Vista (Arrrghhh, horroble materials, 5115, the worst ever) and now finally the Win2008/R2 "era"!
One of the "mortal sins" of the Konah and similar initiaves was the "no-content" style. Tanks god, MOCs like 6425 were "revamped" and the things began to change.
Another "motal sin" of the MOCs today are the no correlation of the content of the MOC in its relation with the "Real-World" where a lot of content was introduced but not well used int the "real-world". A good example is the fantastic material 6425 (the new version) where two technologies with low usage is presented in the MOC (a lot of students look and say: i´ll never will use that) and materials like 6420, an introdutory material with a LAB using Network Load Balance, a really low-used technology (important, but information like that don´t have to presented in a introdutory MOC).
Of course, the best MOC will never exist and someone in some palce in sometime will complain about, but i really think MS can come back to the NT4 quality. What´s changed form these days?
Anotehr co-related issue are the CPLSs, whose sells whatever is recommend as "preparation materials" and often are under pressure of the customers who says: we want the MOC i see in the MS Web Site in the "preparation materials" section. I think the coordination with the content of the Learning Web Site it will be benefical for everyone.
What is quality? Something that doesn't embarrass me when I teach it.
Case in point, all of the SQL Server courseware produced by one certain vendor.
Here are the basics I expect from courseware.
- Has no technical errors. I consider this to be non-negotiable. There is no "cost-prohibitive" as far as technical accuracy - if you are not willing to pay for accuracy then don't produce the course in the first place! There is no excuse at all for a technical error.
- Has labs that work. I also consider this to be non-negotiable. Labs should be tested by someone with no skills in the product in question to make sure all the steps work. If a student has a failure in the lab and they don't know if it is their failure or the lab-writers failure then the lab is useless as a learning tool.
For it to be a quality course, I also expect the following.
- Is polished. Looks nice, uses consistent formatting.
- Follows good adult-education practices. Things liike parallell form, not using terms before they are defined.
- All the links work.
Regards
Craig
That the feedback you get on your courses is not good enough for you, is your problem. Not the community.
Food for thought: there are other coursematerial vendors than Microsoft delivering labs that work and a course environment without errors.
And also what Craig said before us :)
/J
As an MCT I expect:
- That the classroom setup be clear and correct and not to discover hidden gems about Hyper-V setup or lab inconsistencies.
- That demonstrations be documented in the Instructor Guide indication which VM and what specific information/files to use.
- I would prefer an actual printed Instructor Guide but can live with the One Note Trainer Packs but they need to be delivered with the courseware and not weeks later. I understand that there are some issues generating the One Note files so if that is an issue the courseware should be delayed in shipping until all of the materials are available. You wouldn't think about shipping a course on one date and the labs 3-6 weeks later.
-As Craig said, lab must work period.
-PowerPoint presentations without missing slides, all text slides, nothing but bullet point slides, etc. The visual aspect of the presentation is supposed to be designed to support the oration not be a reading assignment. Slides should be an impactful visual aid to help drive home the instruction not supplants the instruction.
-Give me the materials-courseware/VM's/Lab Instructions/Demo steps/Slides/etc. to setup, prepare, and deliver a home run for my students who deserve nothing less.
As a Student I Expect:
-Value. I paid upwards of $2500 for a 5 day class and I want an experience to remember not one I'm trying to forget.
-That means labs that make sense, work, and apply to real world situations (see 6294).
-That also means I want some take always. I expect the instructor give me the benefit of his years of knowledge and experience and share some cool tools that might fall out of Microsoft's realm.
-But I also expect to take home a book (digital or print) that I can use as a reference resource (see 6425)and not just a print out of the slide show.
- A reference resource should contain some detailed info in the chapters and point me to some other resources not just a list of links. Appendices like command line reference, white papers on implementation, engineer’s field notes, etc. are what I'm talking about.
- If printing such valued material is cost prohibitive then at least include it on the companion CD. Much of this material is available online through TechNet and other sources but the VALUE ADD is that I have at my fingertips and don't have to go searching for it.
-In short, I want to be blown away and leave the class thinking that it would’ve been worth twice the price and wishing I had another class the next week. The Instructor carries most of the burden on this one but the materials, takeaways, environment, all play a part too.
There have been some significant gains in Quality (thank you M. Roche) and the process has certainly seen some improvements. The community of MCT's is being involved more and earlier in the process and it just plain makes sense to take advantage of this group of passionate, knowledgeable, professionals who want to work with you to help us all shine. Vendors that deliver poor quality materials to Microsoft should not be used again for those types of materials or that subject matter. Vendors that don't meet deadlines should no longer be used, Vendors should eat the cost of doing the Technical and non-technical editing themselves as CW library authors do. It boils down to something very simple. Don't put out anything you wouldn't want to take into a classroom and stake your professional reputation on, because ultimately you are.
Quality courseware is technically accurate, professional in delivery, and properly timed. First, technically accurate. If the content is wrong, why are they here? Even if an MCT can point out what is wrong and explain why something is incorrect, it still leaves a bad impression for a student. "Who do I believe?" This is especially true for "best practice" type topics and slides.
Secondly, professional delivery of materials. I am all about humor in the classroom. However, sometimes students are not. The point is, it should be my perrogative as the instructor to add in humor. I understand that community driven courses are written, most frequently, by instructors. However, courseware exists to provide the outline and content, while the instructor exists to present that to students in an understandable and memorable way. If humor is introduced in the courseware, it should have a point beyond just cracking a joke. If it is part of an illustration (story) or an example, sure, I'll take it, but if it is just there to be there, get it out of my courseware.
Thirdly, courseware should be properly timed. I can mention many, many classes off the top of my head (2778. 6461/3/4, 6231, 50064, among others) that have more content than can possibly be covered in the number of days alloted. Some of this is a consequence of having few 4-day courses in the library (6461/3/4, etc). Other timing issues are due to there being no MCITP/MCPD courses for particular technologies, so more content must be shoved into the MCTS level classes (6231/2/4/5/6). Others are due to there just being to much content in one book (2778/50064). If it takes 75% of class time to run the labs alone and the class is not a workshop, how am I supposed to teach the content in 25% of the time? I shouldn't have to cut significant portions of content or pick what is "important" to fit time. To fit my students, I'll cut content, but I shouldn't have to cut content due to poor courseware timing.
I do agree with the article however. Typos (as long as it isn't in code) doesn't make or break courseware. And many of the reviews in the CWL are useless as well (whoever checks stars and says "good" is just wasting their time and mine). Also, I way too frequently see authors rate their own courseware. Does anyone else see a conflict of interest?
Anyways, there is my 2cents.
Paul
This is great! Keep 'em coming!