![[Table of Contents]](/icons/up.gif){kind=icon}
Archived Information
The Quality of Vocational Education, June 1998The first analysis examined the level of vocational coursetaking.Table 1 shows sample sizes for subgroups used as control variables in this analysis and presents the average number of vocational course Carnegie Units and the percentage of total Carnegie Units taken as vocational courses.[1] On average, this population spent over three-and-one-half Carnegie Units (or about seven one-semester courses) in vocational courses. Because public schools are more likely to have vocational programs, as expected, students in public high schools took more vocational courses than students in private high schools. Students in rural areas took more vocational courses than those in urban and suburban areas and those in the lowest three socioeconomic quartiles took more vocational courses than those in the highest socioeconomic quartile. Students of Asian/Pacific Islander ancestry were least likely to take vocational education courses. Students who never dropped out of high school took a greater number of vocational courses than did dropouts, and handicapped students took a greater number of vocational courses than non-handicapped students.
Table 1 also shows vocational coursetaking as a percentage of total Carnegie Units. Because the total number of units is more or less constant at about 24 across subgroups (except for dropouts), the subgroup percentage differences follow the pattern of the average Carnegie Units for vocational coursetaking. The student/dropout percentage difference is an exception. As noted, students show more Carnegie Units in vocational education than dropouts. However, dropouts show a higher percentage of total Carnegie Units spent in vocational education. This indicates that vocational education makes up a higher proportion of the dropout's educational experience. Therefore, it is particularly important to examine the relationship between vocational education and achievement for dropouts. It is also important to examine how vocational programs affect the dropout rate. Both of these issues will be taken up in a later section.
In the following sections, results are presented that assess the relationship between vocational coursetaking and achievement. First, achievement levels of students in different curricular programs are examined. Next, the relationship between vocational coursetaking and gain in mathematics, science, and reading achievement from tenth to twelfth grade are observed. In the assessment of gain controls for school, demographic, and background factors on the one hand, and coursetaking in academic and other non-vocational areas on the other, are included. Finally, dropout rates of students in vocational programs, compared to dropout rates of students in academic and general curricular programs, are examined.
One criticism of vocational education is that it is a repository for low-achieving students. To examine this, average achievement test scores of students in academic, vocational, and general high school programs were compared. Table 2 shows the mean 10th- and 12th-grade mathematics, science, and reading achievement scores[2] across the different curricular program types[3].
Achievement scores of students in vocational programs are substantially lower than those of students in academic programs, and are sometimes less than achievement scores of students in the general category. Statistical tests of differences in achievement between the programs were conducted using regression analysis[4]. Contrast variables examining the difference in achievement between academic and vocational programs, on the one hand, and a general high school program, on the other, were used as independent variables along with the control variables listed in table 1 in a regression of achievement scores. Results indicate that vocational program students score significantly lower than general program students in 10th-grade mathematics achievement (t=1.965, p=.05) and in 10th- and 12th-grade reading achievement (10th-grade reading, t=2.75, p=.006; 12th-grade reading, t=2.46, p=.014).
Ideally, vocational education courses should be structured to impart vocational skills and to reinforce or introduce standard academic content as it applies to a vocational area. This section examines the relationship between vocational coursetaking and achievement.Table 3 presents unstandardized regression coefficients between vocational coursetaking across the entire high school experience and 12th-grade achievement in mathematics, science, and reading[5].
For all three indicators of achievement most of the coefficients are negative. This suggests that vocational courses may have a negative impact on achievement in these three traditional academic areas. The exception is technical/communications, for which a positive coefficient is found. This positive coefficient is largest, relative to its sampling variation, for mathematics achievement, suggesting that technical/communications courses significantly influence math achievement.
However, the coefficients presented in table 3 may be misleading because the simple relationship between vocational coursetaking and achievement may be mediated by other factors, such as students' prior achievement levels. For example, with respect to the negative relationships, it may be the case that students who received low scores on their achievement tests early in high school were more likely to take vocational courses during the rest of their high school tenure. If this is so, then because 12th-grade achievement scores are related to earlier achievement scores, the zero-order relationships between vocational education coursetaking and 12th-grade achievement would be negative. If this were the case, it would be wrong to conclude from this negative zero-order relationship that vocational education courses depress the achievement of the students who take them, rather than recognizing selection bias as the main source of the negative association.
In addition, the simple relationships between vocational coursetaking and achievement do not take into account the influence of academic coursetaking, background characteristics, and other demographic factors on achievement. If taking certain vocational courses is associated with taking other non-vocational courses, and if the nonvocational courses are associated with achievement, any analysis that does not account for the non-vocational courses and their impact on achievement would give a biased indication of the influence of vocational education on achievement. Similarly, if taking vocational courses is associated with background factors, and these in turn are associated with achievement, then not taking the background factors into account would result in misleading conclusions about the relationship between vocational coursetaking and achievement.
Multiple regression is a technique suitable for controlling the effects of prior achievement, non-vocational coursetaking, and background characteristics, while assessing the impact of vocational coursetaking on achievement. Because other variables related to achievement are included in the analysis, multiple regression becomes a means of examining the relationship between coursetaking and achievement gain independent of the other variables. Two-stage least-squares regression (Johnson, 1984) was used to examine the effect of vocational coursetaking on 10th- to 12th-grade achievement gain, independent of academic coursetaking and independent of school and student background characteristics. This technique was used because when prior achievement is included in the regression analysis, negative bias in parameters may result due to measurement error in the 10th-grade achievement variable (Meyer, 1992).
The two-stage least-squares regression technique was used to examine the effect of vocational coursetaking on 10th- to 12th-grade achievement gain, independent of academic coursetaking and independent of school and student background characteristics[6]. In the first stage of the analysis, 10th-grade achievement in mathematics, science, and reading were predicted from composite indices of 9th- and 10th-grade vocational and academic coursetaking, on the one hand, and 11th- and 12th-grade vocational and academic coursetaking, on the other. Ninth and 10th-grade coursetaking were used as instrumental variables to create a predicted value of 10th-grade achievement that was uncorrelated with errors of measurement in 12th-grade achievement. Eleventh and 12th-grade vocational and academic coursetaking were included to control for student self-selection into vocational and academic tracks in their junior and senior year based on their earlier high school performance. Finally, student and school background characteristics were also included in the first-stage analysis.
In the second stage of the analysis, 12th-grade achievement was predicted from vocational and academic coursetaking in the 11th and 12th grade. Student and school background characteristics were included in the analysis along with 10th-grade achievement scores predicted from the first stage analysis. The results of the second stage of the analyses for mathematics, science, and reading, for the entire sample and for dropouts are presented in tables 4 through 9.
Gain in mathematics achievement. Results of the two-stage least-squares regression analyses for mathematics achievement are shown in table 4 and table 5. The best predictor of 12th-grade math achievement was 10th-grade math achievement. When the entire sample is considered, vocational courses do not explain significant variation in gain in mathematics achievement. Significant positive effects were found for coursetaking in Algebra I, Algebra II, Geometry, Precalculus, Calculus, and Physics.
Gain in science achievement. Results of the two-stage least-squares regression analysis for science achievement are shown in table 6 and table 7. As with mathematics, the best predictor of 12th-grade science achievement is 10th grade science achievement. For the entire sample, vocational courses did not have significant positive effects on gain in science achievement. Significant positive effects for science were found for Science Survey courses, Chemistry, Physics and Foreign Language. The effect of Biology was marginally significant.
Gain in reading achievement. Results of the two-stage least-squares regression analysis for reading achievement are shown in table 8 and table 9. As with math and science, 12th-grade test scores were best predicted by 10th-grade test scores. There were no significant effects of vocational coursetaking on gain in reading achievement in the overall analysis, although Consumer Education had a marginally significant negative effect while Specific Labor Market Preparation had a marginally significant positive effect.
Subsample analyses. Results from analyses of the dropout subsample also indicated little impact of vocational coursetaking on academic achievement. The only significant effects were a The second analysis regressed 12th-grade achievement on 10th-grade predicted achievement (obtained from the model specified in the first analysis), 11th- and 12th-grade coursetaking, and background characteristics. negative coefficient for health occupation studies and a positive coefficient for specific labor market preparation on reading achievement (see table 9). Other subsamples, including students in public schools, rural schools, vocational programs, special education, and members of various minority groups and socioeconomic levels, showed similarly small effects of vocational coursetaking (results not shown, but available from the authors upon request).
Vocational education and dropout rates. While vocational courses do not appear to have an appreciable effect on math, science, and reading achievement, it is possible that vocational programs may keep students from dropping out of high school. This possibility was examined by assessing the dropout rate (here defined as the ratio of students who had dropped out of school by the 12th grade to students who were not classified as dropouts in the 10th grade) by high school program type. For the data used in this report the overall dropout rate was 5.6 percent. For students in academic programs the dropout rate shrinks to a little more than a quarter of a percent (.26 percent). In sharp contrast, nearly one out of every four (24.58 percent) students in general programs drop out of high school during their junior and senior years. The dropout rate for vocational programs (4.01 percent) is larger than for academic programs, but is considerably smaller than for general programs.
Logistic regression was used to test whether these differences were significant. Because it is important to control for background characteristics when assessing the effect of curricular program on the dropout rate, background characteristics were included in the analysis. Prior achievement was also included. Two measures of prior achievement were available, 10th-grade achievement in math, science, and reading, and class rank. Both variables seem important: achievement because of the careful way it was measured in the NELS:88 survey, and because of its traditional importance in educational research, and class rank because it is a measure of relative performance and may be more salient to the student as a measure of his or her academic performance and motivation. Both are relatively independent measures of achievement; the measures correlate only about .34. Unfortunately, both measures have fairly high missing data rates and, when used together in the analyses, reduce the number of cases by about one-third.
Three logistic regression analyses were conducted to assess the effect of curricular track on the dropout rate. The first analyses used 10th-grade achievement, the second used class rank, transformed to a logit to improve the distribution, and the third used both achievement and class rank. Though the number of missing cases was higher when both achievement and rank were included, the results of the three analyses were essentially the same. Table 10 presents the results of the analysis using both 10th-grade achievement and class rank. The 10th-grade achievement variable used was the sum of the IRT-number right scores on the math, science, and reading tests. In this table a negative coefficient indicates that a variable decreases the dropout rate while a positive coefficient indicate that it increases the dropout rate.
Table 10 results indicate that the proportion of students who drop out in the 11th and 12th grades is significantly less in vocational programs compared to general programs. Results are supportive of similar analyses conducted on national studies in the past (Grasso & Shea, 1979, Perlmutter, 1982, Wagner, 1991) and summarized by Kulik (this volume). Dropout rates were lower in course pattern-determined vocational programs than in general high school programs. The coefficients for vocational program from each of the three models suggest that students in vocational programs are 8 to 10 times less likely to drop out of high school in their third and fourth years if they are in a vocational program, compared to a general high school program. Academic track students are much less likely to drop out than either general or vocational students. Using the transcript-defined program classification in assessing the effect of vocational education on dropouts presents a difficulty; students who drop out may not be in school long enough to qualify for membership in either the academic or vocational programs. By default they drop into the general high school program category, with the result that dropout rates in this category are artificially inflated while dropout rates in the vocational and academic categories are artificially depressed[7]. In fact, this would appear to be true in any research where program classification is used to determine a vocational concentrator, unless one assumes that students are tracked into vocational, academic and general programs very early during their high school tenure.
An alternative analysis was designed to attempt to overcome the confounding of curricular program categorization and dropping out. The alternative analysis examined the effect of vocational coursetaking in 9th and 10th grade on the dropout rate in 11th and 12th grade. From this analysis one can see the effects of taking vocational courses in the first two years on dropout rates in the latter years. Results are presented in table 11, table 12 and table 13.
Table 11 shows the results of a logistic regression analysis in which 11th- and 12th-grade dropout status is predicted from vocational and academic course Carnegie Units in 9th and 10th grade, background factors, special program participation (special education, bilingual, or gifted vs. no special program, as indicated on the students' transcripts), class rank (from transcripts) and the number of times the student reported skipping classes (from the NELS:88 Second Follow-up Student Questionnaire). When all of these variables are included in the regression analysis, the effect of vocational coursetaking on the dropout rate is not statistically significant.
Kulik (this volume), who has reviewed the literature on the issue of high school program participation and dropping out, has argued that it is inappropriate to include factors such as class rank and absence from class in assessing the impact of vocational education on the dropout rate. His reasoning (Kulik, 1994) is that these factors are themselves outcomes and should not be confounded with other outcomes such as the dropout rate in the same analysis.
He cites as defense of this argument the study by Wagner (1991) who analyzed data from the National Longitudinal Transition Study of Special Education Students (NLTS), and found a 2.7 percent advantage in dropout rate for vocational students compared to nonvocational students. Her analysis included absenteeism and course failure as predictor variables. However, when she left absenteeism and course failure out of the model, the decrease in the dropout rate due to vocational education rose to 8 percent.
In response to Kulik's argument, the dropout rate in the NELS:88 data was modeled using vocational and academic coursetaking and background characteristics, but leaving out other school-related variables. Table 12 shows the results when class rank, number of classes skipped or cut, and special program participation are removed from the model. The coefficient for vocational courses is negative, indicating that it is associated with a lower dropout rate, and is statistically significant. Each additional Carnegie Unit in vocational education courses in the first two years of high school reduces the dropout rate 1.14 times. For example, for the overall dropout rate of 5.6 percent, an additional Carnegie Unit of vocational education would reduce the rate to 4.9 percent. For the general program dropout rate of 24.58 percent, an additional Carnegie Unit in vocational education would reduce the dropout rate to 21.56 percent. In each case, the estimated reduction in the dropout rate is about 12 percent. The magnitude of the coefficient for vocational courses changes very little and remains significant when the special program indicators are added to the model (table 13).
Comparing across tables 11, 12, and 13 it appears that the variable most responsible for reducing the effect of vocational education is class rank. The question remains whether it is appropriate to include class rank in the model because of its status as an outcome variable that is affected by vocational coursetaking. It is appropriate to include class rank because rank is related to vocational coursetaking and to dropout status. Eliminating class rank from the model predicting dropout status would result in model misspecification. Bias due to model misspecification would be in attributing a direct effect of vocational education on the dropout rate. Rather, the results suggest that vocational education may have an indirect effect on the dropout rate, and that the effect may be mediated through class rank. This interpretation suggests further that vocational education may deter students from dropping out because it allows them to perform better in relation to other students, and it is this improved performance, perhaps because it represents a success experience, that keeps them in school. However, the results are not conclusive because of the inability to account completely for student motivations regarding coursetaking and the decision to drop out of high school.
The final analysis investigated whether coursetaking in individual vocational areas had direct effects on the dropout rates. Results are shown in table 14. Courses in agriculture and technical/communications taken in the 9th and 10th grades significantly reduce the likelihood of dropping out in the 11th and 12th grades. In contrast, courses in consumer economics significantly increase the likelihood of dropping out. Because class rank is included in the model, these effects cannot be attributed to their effect on class rank.
[Summary and Conclusions]