![[Table of Contents]](/icons/up.gif){kind=icon}
Archived Information
The Quality of Vocational Education, June 1998This section of the report will review empirical evidence on the adequacy of occupational experience and competency for teaching in vocational and technical education. Examination of the extant literature and studies should provide evidence to enable informed decision-making about the use of occupational experience and/or competency assessment as alternatives to baccalaureate degrees and college preparation of vocational education teachers. Are vocational and technical education teachers better prepared in the workplace or in college?
At the onset, it may be important to note that all studies reviewed seemed to lack a collective theoretical framework -- which defines well the concepts under investigation, provides benchmark data for subsequent findings, and allows comparisons among related studies. Thus, these studies -- without a theoretical framework -- are eclectic; research questions, variables of interest, even definitions of teacher effectiveness were chosen because of the researcher's personal interests. There is no established relationship among these studies, they generally do not point to a common problem, and few connect data from one to another.
Further, empirical studies are scarce, quite dated, and those that are available are not particularly conclusive on the subject of occupational experience/competency and teaching effectiveness. Most studies collect data through surveys and provide only descriptive information. Both survey and empirical studies are fraught with design error, examine limited populations, tend to focus on trivia, are based in perception or opinion, and provide more in the way of descriptive information than m. --> data to answer substantive questions. Rating instruments apparently do not discriminate sufficiently between "good" and "not so good" teachers or "high achieving" and "not so high achieving" students. Further, there is a definite bias which undergirds the studies; the bias being that occupational experience is and ought to remain the primary determinant of who is to teach in vocational education. Generally, the empirical connections between occupational experience and various effectiveness measures do not match the rhetoric and bias that says years of trade experience can substitute for a college degree and professional education. Nevertheless - and despite all of the above negative noise -- some interesting work has taken place.
No studies of the relation between occupational experience and teaching in vocational education took place before 1954. In that year, Rumpf asked secondary vocational administrators to identify superior teachers on their staff. By studying the traits of the 236 teachers, he concluded that years of occupational experience had little or no positive effect on administrators' performance rating of teachers; in fact, greater amounts of occupational experience appeared to have a negative effect. He found only two slight and positive correlations: rated performance improved with number of college credits earned (r = .11) and increased teaching experience (r = .36). Nine years later, Storm (1965) found similar results. Using survey data from 138 technical school administrators in 38 states, Storm concluded that "high success" technical instructors had more advanced degrees in education and 5.1 less years of occupational experience than "low-success" instructors.
Four other studies, from 1968 through 1973, supported the conclusions of Rumpf and Storm. These studies, although with diverse populations and methodologies, reported on supervisors' or teachers' self-ratings of teaching success factors. As reported by Whitener (1981) and Wilson (1984), authors of these four studies concluded that work experience beyond the minimum requirements of two to four years had no positive effect on teacher success.
Only one study (Swartz, 1974) found a significant effect of trade experience with teacher competency as rated by administrators of Virginia's T&I teachers. Interestingly, though, Swartz found no significant differences in the ratings by administrators, supervisors, teacher-peers, students, and self by combining trade experience, teaching experience, and professional education. Most recently, Mullins (1993) concluded that the immediate supervisors' ratings of Virginia T&I teachers was not dependent upon the variable of years of trade experience (nor, for that matter, on such other variables as teaching experience and professional education).
Kapes and Pawlowski (1976) correlated several factors about T&I teachers with student achievement as measured by the Ohio Trade and Industrial Achievement Tests. "It is important to point out that [teachers'] industrial experience...yielded either no relationship or a small negative relationship with student shop achievement" (p. 10). The authors concluded, "since many years of teacher industrial experience do not improve student shop achievement...it may be necessary to minimize industrial experience as a criterion for teacher certification, or at least consider other competencies more important" (p. 11).
Two studies, both completed in Ohio in the 1960s, produced evidence to favor occupational experience as a positive factor in student achievement. The Industrial Materials Laboratory, Ohio State Trade and Industrial Education Service (1965), and Shoemaker (1971) found a relationship between student achievement in auto mechanics, machine shop, printing, and the number of years the instructor had spent in the occupation prior to becoming a teacher. It is interesting to note, however, that at least two subsequent reviewers of these studies have questioned their conclusions due to a lack of collaborating data. For example, Wilson (1984) comments: "No data or statistical evidence...was provided in the study" to warrant such a statement as "teachers' industrial experience was a significant, positive factor relating to student achievement" (p. 26).
Several studies (Welch & Garner, 1976; Covey, 1973; White, 1971) also found that some occupational experience did produce positive results in student achievement, but that increased occupational experience on the part of the teacher produced decreased performance on the part of the student. Thus, the relationship between the teachers' occupational experience and their students' performance is nonlinear.
The National Occupational Competency Testing Institute (NOCTI) is a nonprofit organization, headquartered at Ferris State University in Big Rapids, Michigan. According to Whitener (1981 and in personal communication, 1994), NOCTI provides high-quality occupational competency examinations on a national level. Each examination has two parts: a written test that measures technical knowledge required in the occupation and a performance test that measures skills typical of the occupation. The services at NOCTI include test development, revision, updating, and scoring, plus job and task analyses that lead to test development. NOCTI occupational competency examinations are the most widely used tests for trade and industrial teachers in the United States.
NOCTI administers 56 occupationally-specific exams, ranging from air conditioning, heating, and repair to welding. Others include, for example, the most frequently administered exams: auto mechanics, carpentry, cosmetology, industrial electrician, machine trades, electronics technology, printing, and quantity food preparation.
In addition to establishing occupational competency and meeting state requirements for teacher certification, colleges and universities grant credits for successfully completing NOCTI teacher tests. According to the most recent survey data (1989), 92 colleges and universities in 43 states grant from 9 to 45 semester hours of credit for persons who successfully complete a NOCTI teacher test(s). Thirteen states use the national mean as their cut-off score for "passing" the NOCTI test; 11 use the national mean minus one standard deviation, 4 grant "passing" to anyone who scores above the 40th percentile on the national norms, and other variations are used by 11 states. Norms are updated each time tests are scored. National data are not available on the number who "pass" the exam nor on the number of retakes. One large state (Michigan) reports a 17 percent retake group.
Eighteen states require NOCTI tests for either initial certification, to recertify within the first year of teaching, or for preservice teachers who lack work experience. In addition, two states use their own tests and two more are considering requiring NOCTI exams as a condition for continued employment. In most cases, a candidate pays the fee (an average of $238) to take the exams. A total of 14,576 persons have taken the test since 1975, an average of 767 per year.
As stated in interviews and correspondence with the director and assessment specialist of NOCTI (Whitener, personal communication, December, 1993 and January, 1994; Rupe, personal communication, January, 1994), no relationships between teachers' occupational test scores and subsequent performance of teachers or their students have been ascertained. There are no predictability studies. It is curious that so little research has been done on scores on NOCTI exams and any relationship they may have to quality measures of vocational education students and programs. NOCTI does collect information on each examinee about the number of years of work experience, grade or education level completed, and the number of years of teaching experience (if any) prior to taking the exam. No correlations on these variables with teacher scores, student achievement, or teacher performance have been done at NOCTI nor apparently at its area testing service centers.
In the early 1980s, Whitener (1981) and Stewart (1984) completed dissertations using NOCTI test scores. Whitener found differential results using scores of a sample of 1,556 persons who had taken the examination from 1974 to 1980 in four specialized occupational areas: auto mechanics, carpentry, machine trades, and quantity food preparation. His general conclusion, "It is apparent that occupational competency is related in different degrees to occupational experience, teaching experience, educational level, and their aggregate. There is evidence that these relationships vary from occupation to occupation" (p. 159). Some occupations had no relationship. Based on his data, Whitener suggests that these variables not be used for teacher selection and certification.
Stewart (1984) correlated occupational competency with such variables as job satisfaction, job satisfactoriness, demographic characteristics, and self-ratings by teachers. His population consisted of 155 Georgia secondary and postsecondary trade and industrial teachers who had passed NOCTI examinations as a condition of their employment during a two-year period, 1981-82. The teachers' average age was 39, and their average occupational experience included 11.5 years. The Minnesota Satisfaction Questionnaire and Minnesota Satisfactoriness Scale instruments were used to gather data from and about the teachers. Stewart found no reliable relationships among job satisfaction, perceived job satisfactoriness, written and performance occupational competency tests, age, teaching experience, teaching level, and occupational experience in the population. He concluded that the relationship between occupational competency and job satisfactoriness was not sufficiently strong to reliably predict job satisfactoriness with measures of NOCTI occupational competency nor occupational experience.
Wilson (1984) reported on three state studies conducted in Pennsylvania in the 1960s and 1970s. The authors investigated the relationship between years of occupational experience and scores on Pennsylvania's state-developed occupational competency examinations. One study investigated degreed teachers (with less than five years of occupational experience) and nondegreed teachers (mean of more than 14 years of occupational experience). All three studies concluded that years of occupational experience did not correlate with higher occupational test scores. In fact, in some occupational areas, there appeared to be a negative correlation between years of occupational experience and occupational competency examination scores.
Doerfort (1989) compared two groups of beginning T&I and agricultural education teachers, one prepared through an industry route and one prepared through traditional teacher education programs, with their results on the National Teacher Examination Core Battery Tests. Those teachers prepared through teacher education scored higher on the communication skills, general knowledge, and professional knowledge components of the test. The amount of professional education received accounted for the largest portion of variance in each of the dependent variables. As cited earlier, Rumpf (1954) and Storm (1965) found positive correlations between college credits and others' ratings of T&I teachers' success, although they did not find positive correlations between the ratings and occupational experiences of teachers. Similarly, Kapes and Pawlowksi (1976) found college credits of T&I teachers to be moderately related with student achievement.
Beidel (1993) extracted data from the 1989 report of the Public School Teacher Questionnaire of the Schools and Staffing Survey for trade and industrial education teachers (identified in Item 27a of the survey). He compared nondegreed and degreed teachers on their self-reported responses to information about current teaching load, perceptions and attitudes toward teaching, and incentives and compensations received. The population of 495 included 184 (44 percent) nondegreed teachers. There were some differences between nondegreed and degreed trade and industrial education teachers, but they tended to be low statistically and probably unimportant. Beidel did find that degreed teachers are more satisfied with their class size and felt they had greater control of the instructional process than did nondegreed teachers. "Degreed teacher may be better prepared for the average number of students in a class and develop curriculum reflecting these needs prior to teaching" (p. 76). However, both groups tended to "agree" and "disagree" with statements at about the same level. Study data might have been more relevant and helpful had it compared trade and industrial education teachers with the teacher population as a whole, and with other vocational education subject areas.
Johnson and Summers (1993) recently analyzed 17 studies that examined various characteristics of the schooling experience and subsequent labor market performance of those schooled. The studies all controlled for relatively "hard" data measures of labor market outcomes, labor characteristics of students after they left high school, and quality measures of the school and school experience. The studies used fairly sophisticated statistical techniques (multiple regression analysis, large sample size, and a range of control measures). The authors concluded that the most significantly positive coefficients describing the quality of teachers were descriptive of their education, simply "better-educated teachers produce more effective employees" (p. 13).
Two studies cited by Johnson and Summers (1993) seem particularly relevant for purposes of this paper: Card and Krueger (1992) and Wachtel (1976). Using multiple regression techniques, Card and Krueger analyzed several census, education, labor, and personal income data sets containing information on 1,019,746 white men born from 1920-49. All were working in 1979 -- approximately 21-41 years out of high school -- and kept a log of their weekly earnings during that year. An important finding for purposes of this paper is that the rates of return on labor market outputs were better for individuals from states with better-educated teachers and, incidentally, with a higher fraction of female teachers (Johnson & Summers, p. 31.)
Using several data sets and a recursive model (run with and without school quality measures, defined by expenditures per district), Wachtel analyzed the post high school achievement of 1,812 public school educated men whose mean age in 1969 was 47 and whose incomes in 1958 dollars were determined to be between $4,000 and $75,000. Regressions using alternative measures of school quality found "evidence that school quality affects the rate of growth of earnings" (p. 48) and one of the quality affects was that "better-educated teachers produce more effective employees" (p. 13).
Interesting is that, in general, Johnson and Summers found high quality vocational education as having a number of positive results for high school graduates who go directly into the labor force upon graduation. These authors suggest that policy groups focus on identifying the common characteristics of effective vocational education programs. Cumulative evidence in the Johnson and Summers analyses suggests there is a significant and positive relationship between better educated teachers, effective vocational education, and the subsequent earnings and performance of students. They conclude that better teachers do a better job of preparing students for the world of work and that the most important measurements for better teachers are increased education and salary (pp. 14-15).
As discussed earlier, Beidel (1993) outlined four types of instructor training programs which have historically been offered for vocational education teachers, primarily in trade and industrial education, who are alternatively certified. The fundamental intent is to provide "survival training" to those with limited or no formal teacher training, college education, or recent experience with public education. The challenge to state and local school administrators has been to design beginning teacher induction programs that will reduce the many problems confronting first-year teachers and to help them experience success in the classrooms. Has it worked?
Some of the more interesting and comprehensive studies with beginning vocational education teachers, including those prepared through traditional teacher education and those entering teaching without teacher education (i.e., but with considerable occupational experience), were conducted during the late 1980s and early 1990s by Heath-Camp and Camp for the National Center for Research in Vocational Education, The University of California at Berkeley.
Camp and Heath-Camp (1988 and 1989) and their colleagues originally laid out their research agenda to study the nature, dynamics, and scope of the induction process for beginning vocational education teachers. They define the induction process as a "culmination of experiences that lead the beginning teacher eventually to become successful, comfortable and confident in teaching or to exit the profession" (1988, p. 14). They were particularly curious about induction programs for vocational education teachers who are beginning to teach without benefit of teacher education.
Using beginning vocational education teachers as their subjects and focus, these two researchers and their colleagues analyzed the literature from 1933 to present, conducted in-depth focus group and individual interviews, analyzed daily/weekly tape-recorded logs, observed participants, reported qualitative data from five case studies, collected survey data from a national stratified random sample of 352 vocational education teachers in 15 states who began to teach during the 1989_90 school year, and examined and reported on exemplary induction programs. About 43 percent (N = 149) of the national sample of respondents were alternatively certified. Among their findings of relevance for purposes of this report and as reported in various monographs, journal articles, and reprints:
[Conclusions]