A r c h i v e d  I n f o r m a t i o n

Education Reforms and Students At Risk - October 1996

Assessment of the Outcomes of Reforms

In this section, we present evidence that the individual and collected group of schools/programs that we studied produced desirable effects. Among the elementary school programs, effectiveness is examined in terms of staff capacity and norm-referenced achievement. Among secondary programs, simple measurement of effects is necessarily more complicated and contextual.

Puma et al. (1993) found that within their carefully weighted, representative sample of U. S. third-grade students, the mean Normal Curve Equivalent (NCE)27 score on the Comprehensive Test of Basic Skills (CTBS) in reading was 53.4, corresponding to the 56th percentile on the norming tables provided by CTB/McGraw Hill. By contrast, the mean Total Reading NCE score for third-grade students attending schools that served over 75 percent free-lunch populations was 37.6, or the 28th percentile. In mathematics, the third-grade national average NCE score was 52.4 (52nd percentile). For students attending schools that served over 75 percent free-lunch populations, the mean NCE score was 36.6 (26th percentile).

Excluding the replication sites at the elementary level and those elementary sites that are aiming for less than full program implementation, the elementary schools in our study all served over 75 percent free-lunch communities. Not only did these elementary schools consistently perform above the levels found by Puma et al. to be average for very high-poverty communities, but they also consistently approached or exceeded national averages. For example, all grades at School J averaged above both local and national means. Similarly, School B's norm-referenced achievements were consistently averaging in the 50th to 70th percentile. School D produced achievements that exceeded local and state averages (the local achievement test is normed within the state, not the nation).

Schools K and L presented similarly impressive academic pictures by the time students reached third grade, and each of these schools also obtained higher-than-district-average levels of student attendance.

It is not inevitable that students living in poverty fall forever further and further behind. The schools we studied presented clear evidence that there are successful working models of elementary school excellence in America today. Most models have been replicated several times, and where the replications are well-supported over several years, the results are dramatic and impressive.

Our study also provides data on the effects of less-than-strong implementation, and on the effects of a reasonably strong implementation of a program with not highly academically focused gains. School G was operating with very mixed evidence of implementation of the Paideia program, and was achieving outcomes consistent with the scores found by Puma et al., and far below those at several other elementary schools serving very high poverty communities in our study. Although School H had made advances in several important areas, such as regarded relationships among adults and between adults and children, the lack of clear academic focus may have led to test results that, while acceptable, were not nearly as stellar as those of some other schools we studied.

The elementary school data collected during our site visits seem clear. Choosing an academically focused program, creating a strong sense of community, and using the program and other resources to create a high-reliability organization consistently led to powerful academic outcomes for children placed at risk of educational failure. Puma et al. 's national averages for schools serving large numbers of students placed at risk are in no way ordained; they simply reflect current realities. When placed in positive environments that support academic skills development, America's elementary grades children placed at risk achieve at and above national levels.

The high school data from our study are more ambiguous in several regards. First, and most problematic, fewer people accept norm-referenced tests as a valid measure of a school's "effectiveness." Second, high schools do not control the first several years of schooling received by students, so that relatively low levels of achievement may be, in part, the result of prior, unsuccessful experiences. Third, by high school, students' peer groups begin to have very strong influences on students' actions. The United States has produced fewer models or designs for improving high schools, and those that have been forwarded have been slow to conduct carefully controlled studies of their effects. Studies focusing on high schools have tended to look for common themes within and across schools, such as indicators of school restructuring, rather than evidence of implementation of a "program" (see Newmann and Wehlage, 1995).

In our study, including replicate sites, we visited four "restructuring" high schools. These schools were engaged in efforts to provide greater attention to individual students needs by creating smaller learning communities within their campuses. All the schools had experienced difficulties along this journey, and none perhaps can be regarded as fully "restructured." However, three of these schools clearly were achieving higher levels of student engagement and rates of student graduation than was the fourth. At this fourth school, the restructured "communities" were largely identifiable only by name; that is, whatever team teaching and integrated curricula had ever been achieved were no longer in evidence. Not surprisingly, many classes seemed poorly prepared and poorly taught, and the students in these classes were more often off task than on. Even at this high school, however, selected classrooms gave evidence that able, creative, and interested teachers were making a difference in students' learning. These teachers were not less demanding; in fact, they demanded more of their students than was observed in other classrooms precisely because they appeared to believe strongly that their students could do the work.

We also visited two high schools as part of our study that had previously received federal funding as demonstration programs under the School Dropout Demonstration Assistance Program (SDDAP), authorized under Title VI, Part A, of the Augustus F. Hawkins-Robert T. Stafford Elementary and Secondary School Improvement Amendments of 1988. One of these sites was an alternative high school serving students from surrounding districts deemed most in need of special services. The other site combined a school-within-a-school model with mentored, paid work experience for students at risk. These sites had participated in the national evaluation of the SDDAP from 1989 through 1992 and were found to have achieved lower student dropout rates (and to have kept students in school longer prior to their dropping out), higher grade averages, and lower rates of absenteeism than schools serving comparably at-risk populations in the area (see Rossi, forthcoming). In our study, both sites were found to be quite alive, well, and, apparently doing better than ever. Eight districts are now supporting the alternative school, which continues to feature a strong school-community orientation, and the school-within-a-school/work experience program remains a model for the district and its private sector constituencies.

In summary, data collected from the schools we studied suggest the following:

 

27.Normal Curve Equivalent scores (NCEs) are normalized standard scores matching the percentile distribution at values of 1, 50, and 99, with a standard deviation of 21.01 (Tallmadge & Wood, 1981).


-###-

[Cross-Site Analysis Part 4] [Table of Contents] [Assessment of the Resources Required to Impliment Reform]