Thursday, March 12, 2009

Assuring security when giving tests

The requirements of online testing and absolute security/verification are generally incompatible. As noted, the user and their assistant simply need to be at the terminal at the same time in order to cheat the system. If absolute verification that the individual is responsible for every answer is required, the only guaranteed solution is proctored testing with periodic checks using a government issued photo-identification card.
If the user is trying to cheat the on-line system, there are many ways they can bypass all the security measures proposed by the various vendors:
1. The registered user and their assistant can both be at the terminal/computer taking the test. Any biometric or password-based verification can be completed by the real user, while at the same time their assistant provides them the answers.
2. The registered user can share their password with their assistant, and thus any password-based system can be fooled.
3. A camera taking pictures of the end-user's environment will not pick up the blue-tooth headset through which they are being fed the answers.

Instead of trying to increase security, the approaches we have found to work well include:
1. Give the required questions multiple times throughout the course rather than just at the end. For example, after a page in the middle of a chapter, give a short quiz with the required question. Feed the answer to the student if they got it wrong. At the end of the chapter, repeat the question. Finally, in the final exam, give the question. Make all tests required, but only the last one counts. If, by the third attempt, the user doesn't answer correctly, you have a bigger problem with that employee, and alternative remediation will be necessary.
2. Require the user/employee to sign and send in an affidavit that their responses were their own work. This provides legal/regulatory compliance that the organization has done their due diligence.

Most end-users (especially those who cheat) will opt for the easier alternative. If the tests are comprehensive, but easier (and less annoying) than the effort to cheat, they will generally choose to just take the test. We have seen that a 45 minute PowerPoint presentation followed by a required test is a really bad way to present content. Users walk away until the automated part is finished, and then come back to just take the test. Instead, make it so that the user can navigate anywhere in the course. If they fail an exam, send them back to the content so that they have to pass the material.

If you make a reasonable effort, you should be able to satisfy the regulatory requirements. If requirements cannot be met this way, consider proctored examinations in a controlled environment.

No comments: