A Parallel Postsecondary Universe: The Certification System in Information Technology - October 2000

The Assessments: Think You Can Pass?

49. What kind of examinations are these? First, they all have cut-scores. Similar to licensure rules in law, medicine, architecture, and nursing, this is a "no pass/no play" world, though "no play" in this case means only that one does not have access to the higher salaries and the support services of the guild that certification brings. Like the Bar exams, too, one may retake the certification tests, with an appropriate wait between test administrations.

Second, the formats of the examinations range from 45-120 minute restricted response (multiple-choice, identification, one best correct answer, 2/3/4 correct answers), to restricted-response adaptive modes (that stop the testing process at the point in the adaptive curve at which a passing score could be predicted at a 95 percent confidence level), constructed response (e.g. drag and drop), and "essays" (e.g. "Write a program in X to do Y" or "Write a technical report/press release explaining M to audience N), to three-hour simulations, scenarios with "select-and-place" items, and case studies involving performance benchmarks.

Even within restricted response formats, some examinations are very creative. Cisco's Certified Design Associate (CCDA) examination, for example, presents stories about organizations that require a wide area network (WAN), and a combination of drag-and-drop, identification, and one best answer questions require the candidate to apply design knowledge in the context of the stories.

50. The Red Hat Certified Engineer Exam is a six-hour laboratory affair, including two and one-half hours performing an installation and configuration of a system, and two and one-half hours debugging system problems. A core Linux operating system firm, Red Hat was the only vendor-certifier I found that indicated passing rates on examinations: 60 percent for the performance-based Red Hat Certified Engineer exam, on which the cut-score is 80. As table 6 demonstrates, 80 is at the high end of passing scores on IT certification examinations.

51. The most demanding certification is probably the Cisco Certified Internetworking Expert (CCIE) assessment which, in addition to a two-hour written exam, requires a two-day lab exam that "pits the candidate against difficult build, break, and restore scenarios." Those in higher education who talk glibly about performance assessment don't really know what it means until they are up against the CCIE. On the Web site for current and prospective Cisco certified experts, one student advised others to work through the Internetwork Troubleshooting and Internetwork Design examinations before "attacking the real monster, the CCIE."

52. As in the case of high-stakes gateway examinations in the traditional higher education system, a considerable industry has arisen to assist prospective IT certification holders prepare for the examinations. Exclusive of the coursework provided by the vendors, industry associations, training-partners, and postsecondary institutions, a massive amount of printed exam preparation manuals, self-study CDs, and dedicated Web sites (certifyexpress.com, selftrain.com, exampractice.com, gocertify.com, cramsession.com, IT2002.com, netcraftonline,com, and others) form the core of support. The user should verify the credentials of the site. An on-line service that sells flowers and health advice in addition to certification preparation is probably one to avoid.

Unlike the competitive environment for access to elite undergraduate institutions or to "top" law and medical schools, the spirit of the IT certification industry involves cooperation and sharing. The most revealing indication of this phenomenon is the "braindump," a Web site at which prospective examinees seek--and others provide--sample questions and problems that will help them prepare. Here are two typical exchanges:

? I am preparing for the Sun Certified Programmer. Please send questions as many as you have . . .also tell me about recommended reading material.

> What will happen if you compile/run the following code?

1: public class Q21 2: {3: int maxElements; 4: 5: void Q21() 6: {7:maxElements=100; 8:System.out.printIn (maxElements); 9: } 10:11: Q21(int i) 12: {13: maxElements=i; 14: System.out,printIn(maxElements); 15: } 16: 17: public static void main(String[] args) 18: {19:Q21 a=new Q21(); 20: Q21(999); 21: } 22: }

Think about initialization [of variables], compilation errors, print output.

? Hi friends. I plan to write the Sun Java2 Certification. Please send brain dumps, related web sites, question papers. . .

> What will be the output [of the following]?

public class Test {public static void main(String args[] { StringBuffer a=new StringBuffer("One"); StringBuffer b= new StringBuffer("Two"); Test.swap(a,b); System.out.printIn("a is "+a="\nb is "+b);}static void swap (StringBuffer a, StringBufferb) {a.append("more"); b=a; } }

Check this site: http://www.software.u-net.com/javaexam/

The answers to the questions are not the point: rather both the sense of community implicit in the sharing, and the instructional talents of the respondents, who posed their questions as unrestricted response (open-ended) prompts. To be sure, sharing may be a form of self-interest, because the more people who acquire the level of competence required for certification the better the communication across the industry. The people involved in these exchanges may wind up on the same design team.

53. With very minimal knowledge and study, I have taken practice pre-tests on-line for the A+ certification, the MCSE, and Web Developer, and, with correspondingly little difficulty, managed to flunk all of them. The experience drove home the most basic principle of IT industry certification: content counts! One cannot get by on interpreting verbal clues in prompts. You either know the "stuff" or you don't. And you must study--hard. Responses to Microsoft's 1998 salary survey revealed that those who earned the MCSE in the previous year spent an average of 216 hours preparing for their certification exams. The Microsoft Certified Trainers (MCTs), who must be recertified annually, spent an average of 222 hours preparing.

54. Given the global reach of IT certification, some examinations are available in languages other than English, but that depends on subject matter. Java, for example, is not written in Japanese, Spanish, or Russian. One might prompt an examinee to solve a Java problem using a language other than English, but without a modicum of English, the examinee cannot read or write the Java text. On the other hand, for example, the field of document imaging is symbol-dependent, not language-dependent, and CompTIA's examination for Document Imaging Architect is available in some languages other than English (and when it is not available in a local language, candidates have an extra 30 minutes to complete the test). In between are simulations which, as Microsoft notes, "are used more often in English exams," but can be incorporated into exams in "selected languages." When languages are "selected," they tend to be major. CompTIA's exams, for example, are offered in Spanish, German, French, Brazilian Portuguese, Japanese, and both simplified and "traditional" [Mandarin] Chinese.

A global guild is sensitive to linguistic diversity, even when the technological supradialect of English is the language of its superstructure. From the perspective of traditional higher education, this awareness and concern with language environments should be one of the most attractive features of the information technology guild. It's not that certification requires bilingualism (though in many cases, it does), rather that the learning and assessment processes are linguistically inclusive. In the last half of the year 2000, Cisco's training partners, for example, are offering preparatory coursework for technical certifications in 19 languages other than English (see Appendix B).

55. The testing firms do not award the formal certifications. Rather, they report examination results to the vendors or industry associations that issue the documents of certification, and both vendors and industry associations may have requirements for certification beyond those of examination. Nor do the testing firms write the examinations or determine test content and specifications. The Computing Technology Industry Association (CompTIA), for example, maintains a board of Subject Matter Experts (SMEs) who write test objectives, new test items, and regularly conduct what's known in the psychometric trade as "content representativeness studies" to ensure that certification examinations are current with the state of knowledge and practice. Certification tests in the IT world are constantly being retired and replaced to meet the current state of vendor products and industry knowledge. As of January 2000, Microsoft had announced the retirement and replacement of 27 of its certification examinations. Compared to modal behavior in the world of higher education (the Graduate Record Examination Board's subject field test in Computer Science has not undergone a major content representativeness study since 1982!), these standards are rigorous.

56. I found a few exceptions to the firewall on test volume data, one of them perhaps inadvertent: a Prometric press release of March 10, 1999 indicating that, since the inception of the first IT industry certification a decade earlier, Prometric had delivered more than 2.1 million examinations for Novell alone. When a representative of Virtual University Enterprises estimated worldwide test volume for the entire industry in 1999 as 3 million (half outside the U.S.), the Prometric press release number did not appear outlandish.

57. Another exception came from the 8,500-member National Association of Communication Systems Engineers (NACSE). NACSE offers four generic certifications (two for technicians and two for "network specialists"). Its examinations, monitored by a board of standards, are delivered on-line by a small testing service firm called Web University, and administered by partner institutions (including colleges) which provide the computer labs and the proctors. As a non-profit organization, NASCE does not regard test volume data as proprietary information. Bob Kile, its Director, reported 1999 test volume in four "disciplines" as follows:

This distribution reflects larger industry patterns: networking certifications are more important than others as labor market entry and advancement points, and Web-related certifications provide entre to positions in a wide array of both commercial and non-profit organizations.

[Certification Agents][Beyond the Vendor]