Anyone who has ever had the “privilege” of interviewing a certified individual with purely theoretical knowledge appreciates the value of hands-on tests. The creators of certifications in the IT industry (including Cisco Systems) have responded by including more and more hands-on exercises in the certification exams. Unfortunately, Cisco decided not to use the real equipment, but rather simulations. While this is definitely better than relying exclusively on multiple-choice tests, students can still work their way through the simulations without having a decent level of hands-on experience.

This blog post has been written in 2009. In the meantime, almost all labs are performed in virtual environments. Hopefully nobody is pushing the simulation products any longer.

The simulations are—by definition—not equivalent to the actual devices. The few I’ve seen accepted only a limited set of configuration or monitoring commands, sometimes even dictating the order in which the commands have to be entered. It’s quite easy to find the right command if the simulation allows you to use five or ten different commands. Figuring out the correct command from a three-page list of keywords (each of them followed by another page of options) is a completely different story. In a very limited environment, someone who has only vague ideas about the configuration process and the necessary steps might be able to complete the simulation, but would not have a chance to do the same trick when faced with an actual device.

Interestingly, the solution to this dilemma has been available for over a decade: several companies on the market offer true remote lab exercises (not the kind of equipment sharing where you get access to a rack of boxes for eight hours and have to start by cleaning up the mess left by the previous student). Some of these companies offer complex scheduling options, including the ability to schedule an exercise days or even weeks in advance. The willingness of end users to entrust their e-mail or documents to a “cloud service” somewhere on the Internet also proves that the global Internet has reached the level of reliability where users have stopped wondering whether they’ll be able to access a remote service. Thus, there is no technological barrier to using actual devices in the certification exams. So why is everyone (apart from the CCIEs) still stuck with the simulations?