Remote Proctoring and Badging: ATP Recap

Remote Proctoring and Badging: ATP Recap

Kerri Davis - MSL Anti-Piracy PM

Two weeks ago you heard from my colleague Liberty Munson on the Innovations in Testing conference we attended in March, hosted by the Association of Test Publishers. I also attended and I look forward to this conference every year. This year attendance topped more than 1,000 attendees—clearly the testing field is alive and growing.

What I like about the conference most is the opportunity to meet with professionals from all the different components of testing programs; from content developers to exam delivery providers to pyschometricians and data forensics experts; from those in the academic fields to IT and to professional licensure entities. It is such a great place to bounce ideas of each other, learn how other programs craft their tests, and find out about new trends in the industry.

Naturally, I make my way through the security circuit. No matter what kind of testing program an entity manages, they all need to thwart cheating and fraud. I’d say more than a third of the conference sessions focused on security in some capacity. Those sessions were also standing-room only in some cases. I learned that some of my counterparts have very robust programs with a lot of process and a huge staff of people; others have found innovative ways to focus on security with more limited means. Some rely on technology and data forensics to combat infringement; some have more standardized ways of ferretting out cheaters. No matter how big or small the program, I always come away with good ideas on how to craft our own program, and what I should pay attention to next.

Some of the trends I saw this year in test security centered on two themes: remote proctoring and badging. Both of these raise good security questions: Test-takers desire more flexibility in how they test and when; with technology as it stands today, is it possible to create a test that can be taken in one’s living room? How do we ensure no one has notes, or a helper in the room? How do we verify identity without running into privacy limitations? These are all facets of remote proctoring that we in the security field will need to address, and I saw some innovative demonstrations at the conference focusing on all those questions. Security is definitely at the forefront of the remote proctoring field.

The other hot topic was badging—the practice of earning a digital “badge” that denotes a particular skill or achievement. Badges are meant to supplement, rather than replace, a certification. Again, from a security standpoint, how do we ensure that a badge isn’t “stolen?” How do I make sure a badge that shows up on someone’s resume is completely earned? What if they expire? All good questions, and another trend I will be keeping my eye on.

What do you all think? Does it make you nervous to take a test in your pajamas, or is this a convenience that outweighs the possibility cheating could increase? Does the idea of a digital badge raise any fraud questions for you?

Got a tip for me? Let me know at mlsecure@microsoft.com.


Comments
  • Mike Corkery
    |

    I like the flexibility of being able to take an exam more places, but I do worry about the exams becoming compromised. Perhaps there is some middle ground between just making exams available at Prometric/Certiport testing centers and "anywhere".

    I personally don't see any extra value in the badges. I already share my Microsoft transcript with colleagues and hiring managers as needed, and use certification logos where appropriate.

  • KevinM
    |

    Badges?  We don't need no stinking badges!  Seriously though, I have no idea what purpose they would serve beyond what is already served by transcript sharing, certificates, access to the MCP Logo Builder, MCP ID cards, and the MCP Virtual Business Cards.  How many ways can we bling ourselves out?

    As far as remote proctoring goes, I've only taken a single exam that did this.  It was to get certified on a complex product from a smaller company that was in a high demand.  Basically, in order to be able to take the exam you had to first take a training course from the company, then successfully pass a post-course assessment at the end of the class week.  Then you had 6 months to schedule a remotely proctored exam, and the proctor was one of the course instructors.  You would remote into a lab scenario that they set up, and you were required to have a webcam and microphone so that they could watch and listen to you while you worked through the exam.  There was 1 proctor per candidate, and it was frankly a little creepy.  I understand the need to prevent cheating, but I didn't care for the idea of people watching me in my home office while I was working.

    I suppose that the system worked for them, but it didn't scale well.  The exam was expensive, and since most people who took the class only needed to learn the software for internal use  only a small number of people (from partner orgs) ever bothered to take the certification exam.

  • OrinThomas
    |

    I prefer physical rather than virtual evidence of certification. My favorite sigil of certification was the (I guess now ancient) lapel pins. While I know they aren't coming back, another good outward identifier might be small high quality certification specific stickers - primarily to be put on laptops. Maybe as something that could be purchased once you'd reached a specific cert. Would also be something cool to hand out to the certified at conferences. A small MCSE: Server Infrastructure sticker would look great on the top corner of my laptop's lid.

  • John Deardurff (MCT Regional Lead - Eastern US)

    I am actually going back to college. (Tomorrow is my first day.) It is at WGU which is an online school. Apparently, they will be sending a web cam for me to set up in my house. So whenever I need to take a final exam, they will have a live proctor monitor me.

    The testing center I go to for my Microsoft exams has a camera where for some of there other exams the test taker has to have their picture taken while showing their ID. The testing center also get secret shoppers that visit at least 3 times a year to make sure that all security precautions are being observed.

    Has far as badges and label pins... I can understand if you only have 2 or 3 certs, but after awhile it would start looking like boy scout uniform. Like Mike, I pretty much just share my transcript in my email signature and on my website.

  • Sidney Andrews - MCT
    |

    "Badges" would be an interesting concept to me if there was a universal standard shared amongst all of the certification authorities.  I know the ITCC has started to work in this direction with a multi-vendor transcript.  What I would like to see in the future is that idea taken even further.  A trusted third-party syncs with each certification vendor's database and generates badges for an individual.  These badges could be used on resumes, LinkedIn profiles and other number of places.  Right now, the ways people display their certifications are so different and disjointed.  

    Large companies and hiring managers love resumes that are automated and easy to verify.  If the trusted certification exchange can be automated in their resume-sorting system, that would be an even bigger benefit.

    It's probably a long way off, but I wouldn't be surprised to see an organization or a startup in this space.