For nearly four decades, magnet schools have been an important element of American public school education. They have offered innovative programs not generally available in local schools and provided opportunities for students to learn in racially diverse environments. Magnet schools have been particularly important in districts that are trying to desegregate.

Congressional support for desegregation first came in the form of the Emergency School Aid Act (ESAA), enacted in the spring of 1972 to "encourage the voluntary reduction, elimination, or prevention of minority-group isolation."¹ Legislation specifically authorizing grants to support the planning and implementation of magnet programs in school districts attempting to desegregate was passed in 1976 as an amendment to ESAA,² and again in 1984, with the enactment of the Magnet Schools Assistance Program (MSAP). MSAP grants are intended to support magnet schools that are part of an approved desegregation plan and that are designed to bring students from different socioeconomic, ethnic, and racial backgrounds together. Beginning in 1985, MSAP has offered multiple-year grants to school districts through a competitive process administered by the U.S. Department of Education (ED).

In 1998, the American Institutes for Research (AIR), with the McKenzie Group serving as subcontractor, was awarded a contract to evaluate the MSAP. This is the final report for our evaluation. It is based on data collected from the 57 projects that received three-year MSAP awards in the summer of 1998.

During the period covered by this study, the Magnet Schools Assistance Program was authorized under the Improving America's Schools Act (IASA) of 1994. Under this legislation, the program had four purposes: to support, through financial assistance to eligible school districts or consortia of school districts, the following:

• The elimination, reduction, or prevention of minority group isolation in elementary and secondary schools with substantial portions of minority students.

• Courses of instruction within magnet schools that will substantially strengthen the knowledge of academic subjects and the grasp of tangible and marketable vocational skills of students attending such schools.

• The development and design of innovative educational methods and practices.

• The development and implementation of magnet school projects that will assist local education agencies (LEA) in achieving systemic reforms and providing all students the opportunity to meet challenging state content and performance standards.³

In each school district receiving MSAP funds, MSAP projects are developed to support these four purposes in one or more magnet schools. In this report, we refer to the MSAP (the U.S. Department of Education source of funding and assistance), the 57 districts receiving MSAP grants in 1998, the projects that the districts developed with MSAP funds, and the MSAP schools and programs supported by the projects.

This report examines the progress MSAP projects made in meeting the legislative purposes of the Magnet Schools Assistance Program. Particular emphasis is given to program outcomes in reducing minority student isolation and improving student achievement. The results show that program outcomes varied within and across school districts. While MSAP schools adopted innovative practices and worked to align their programs with state and district systemic reforms, overall they made only modest progress in reducing minority group isolation and improving student achievement during the three-year funding period. A major factor contributing to these findings may have been the length of the grant period: three years may not allow sufficient time for MSAP projects to fully implement their programs and show substantial change in school enrollment and achievement patterns.

In the following sections, we outline the data sources that informed the evaluation and provide general information about the operation of the program during the 1998-2001 funding cycle. In four subsequent sections, we examine the extent to which the 1998 cohort of MSAP grantees fulfilled the program's legislative purposes of reducing, eliminating, or preventing minority isolation; increasing student achievement; promoting innovative practices; and supporting systemic reforms. In the last section, we discuss strengths and limitations of the study, as well as implications of our findings for the administration of the MSAP.

Data for the National Population of MSAP Projects and Schools

To assess the progress made by MSAP projects and schools with respect to the four main purposes of the program, AIR collected survey data from the full set of projects and schools funded by MSAP in 1998-2001. During 1999-2000, 2000-2001, and 2001-2002, we collected and analyzed data on all 57 projects through telephone interviews with the MSAP Project Directors; in 1999-2000 and 2001-2002, we also conducted mailed surveys, completed by the Project Directors. In addition, we gathered data on the 292 MSAP schools through Principal Surveys in both 1999-2000 and 2000-2001. Finally, we obtained school-level data from the National School-Level State Assessment Score Database (for use in comparative analyses of student achievement in MSAP and non-MSAP schools), from the Common Core of Data of the National Center for Education Statistics (for use in describing characteristics of MSAP schools and districts and analyzing desegregation outcomes), and from the 1999-2000 Schools and Staffing Survey.

Case Studies of Eight Projects

In both spring 2000 and spring 2001, we visited eight MSAP projects selected for in-depth case studies. Although too few to be a representative sample of the national population, the case studies were chosen to include major dimensions on which MSAP districts and schools vary: regions of the U.S., urbanicity, grade levels served, and whether the district's desegregation effort was voluntary or required by a court or other agency. In each case study district, we conducted interviews at the district level and observations and interviews in four MSAP schools, as well as in one or two comparison schools. We also surveyed a sample of teachers in each case study school. Case study data were used to contextualize the findings of our more quantitative analyses of data from the national sample.

In 1998, three-year MSAP grants were awarded to 57 districts and 292 schools within those districts. Some magnet programs did not become fully operational during the first year of the grant, and a few schools discontinued their programs before 2000-2001. Descriptive information about the MSAP-funded districts and schools follows:

• MSAP-funded magnet schools. A total of 285 of 292 magnet schools funded by the 1998 MSAP grants operated magnet programs in 2000-2001.⁴ These MSAP-supported schools represent about 9 percent of all magnets in the United States promoting desegregation.⁵ MSAP-supported magnet schools are similar in terms of grade level distribution to the pool of all magnets in the U.S. promoting desegregation. (Approximately 60 percent of the MSAP programs are in elementary schools.)

• MSAP funding for the magnet schools. While on average MSAP-supported schools received about $300,000 per year, grant amounts varied considerably across projects. Support for personnel constituted the largest budget expenditure.

• Whole school programs and programs-within-schools (PWSs). Most MSAP-supported schools (88 percent) operate whole school programs that offer the magnet curriculum to all students in the school who are in the grades served by the program. PWSs, which offer magnet curricula to some, but not all, students in a school, comprise a small proportion of MSAP's magnet programs (12 percent). This pattern differs somewhat from the national pool of magnet schools promoting desegregation, in which about one-third of the schools operate PWSs.

• Characteristics of students in MSAP-supported schools. MSAP magnet school enrollments include a high proportion of minority students (73 percent on average) and students in poverty (an average of 60 percent are eligible for free or reduced price meals). Approximately 70 percent of the MSAP schools operate Title I programs.

A major objective of the MSAP is to prevent, eliminate, or reduce minority group isolation in MSAP schools. Minority group isolation (MGI) refers to schools in which minority group students constitute more than 50 percent of school enrollment.

• Most of the 294 MSAP schools targeted for desegregation⁶ (77 percent) sought to reduce MGI, rather than prevent or eliminate it.

• Less than one-tenth (8 percent) aimed to prevent MGI by keeping the school's minority enrollment from exceeding 50 percent.

• Less than one-sixth (16 percent) aimed to eliminate MGI by reducing their minority enrollment to 50 percent or less.

This pattern reflects the fact that most of the targeted schools are in districts in which minority students constitute more than 50 percent of public school enrollment.

Overall, the MSAP-supported magnets had a modest impact on preventing, eliminating, or reducing MGI at the 294 desegregation-targeted schools from 1997-1998 (the year prior to the initiation of the three-year MSAP projects) through 2000-2001 (the final year of the MSAP grant), with some variation by project, program, and school features.

• Adjusting for districtwide demographic trends, the MSAP-supported magnet programs for 57 percent of the desegregation-targeted schools succeeded in preventing, eliminating, or reducing MGI, while 43 percent did not show progress.

• The proportion of targeted schools in districts with voluntary desegregation plans that prevented, eliminated, or reduced MGI was slightly higher than the proportion of such schools in districts with desegregation plans required by a court or other agency (60 percent vs. 53 percent).

• A larger proportion of elementary schools prevented, eliminated, or reduced MGI compared with middle schools or high schools (60 percent vs. 54 percent and 48 percent, respectively).

• Whole school programs, which are offered to all students in the school, were more likely to prevent, eliminate, or reduce MGI than were programs-within-schools, which are offered to only some of the students attending a school (59 percent vs. 49 percent).

Analysis of the amount of progress desegregation-targeted schools made in reducing MGI indicated that a small proportion of targeted schools prevented or eliminated minority group isolation, while the successful schools mostly reduced minority group isolation.

• One in six (17 percent) of the targeted schools experienced a reduction of 5 percentage points or more in MGI relative to the district. About one-quarter (28 percent) experienced a reduction of 1 to 5 percentage points, and one in fifteen (7 percent) experienced a reduction that was less than 1 percentage point.

• An additional one in twenty (5 percent) of the targeted schools prevented or eliminated MGI.

• In the remaining 43 percent of schools that did not make progress, MGI increased or remained constant.

Analysis of the factors that influence the ability of targeted elementary schools to reduce minority isolation pointed to several school features:

• Schools are more likely to experience decreasing minority isolation when the school has a racially and ethnically mixed group of minority students.

• Schools are more likely to experience decreasing minority isolation when parents are involved in school events and activities.

• Schools with larger numbers of students per teacher are more likely than those with lower student-to-teacher ratios to experience increases in minority group isolation.

Some of the challenges facing MSAP projects that may explain the modest impact that MSAP magnet programs have had on MGI include such factors as the decreasing number of nonminority students in many districts, a need for more effective recruitment, the need to support parents and retain students in the magnet programs, limitations on factors that are used in the selection of students, and inability to conduct recruitment efforts for the first year of the project because of the timing of grant awards. Our case study data suggest that districts are experimenting with strategies to address some of these challenges. Additional research into these and other challenges is needed to better understand the processes that influence the success of programs in meeting their desegregation objectives.

We examined student achievement in MSAP schools using two types of data: information provided in MSAP projects' annual reports concerning the progress that the schools had made toward meeting achievement goals set by the project, and data obtained from statewide testing programs for MSAP magnet schools and a matched sample of non-MSAP schools.

Progress Toward Meeting Achievement Goals

MSAP projects were required to establish goals for student achievement for each funded school. MSAP projects set a wide variety of achievement goals for their schools.

• Most achievement goals related to student performance on standardized tests in English language arts and mathematics.

• Overall, the MSAP-supported schools were most successful in meeting or making progress toward goals set for the first year of magnet program operation, but continued improvement over longer time periods proved more difficult. Overall, about 51 percent of the schools met half or more of the benchmarks set for the final year of the grant in language arts, and approximately 36 percent did so for mathematics.

• The availability of achievement data was a concern in these analyses. MSAP projects' annual progress reports provided usable school-level data for about one-third of the general achievement objectives that had been described in MSAP applications. For each grant year, outcome data were available for between half and two-thirds of the specific school-level goals we tracked. A major reason for the lack of outcome data was that the state had revised or discontinued the assessments upon which the projects had based their objectives.

Comparative Analysis of Achievement in MSAP and non-MSAP Elementary Schools

In addition to analyzing objectives and data provided by the MSAP projects, AIR used school-level state assessment scores to examine gains in reading and mathematics performance of MSAP-supported magnet elementary schools and a sample of matched comparison schools without MSAP funding.

• Both the MSAP-supported magnet schools and the comparison schools made noticeable progress in reading and mathematics during the grant period. However, when changes in the demographic composition of the schools were controlled for, the gains exhibited by MSAP schools were not significantly different from those exhibited by non-MSAP schools with similar characteristics.

• The measure most strongly associated with achievement growth in both reading and mathematics in MSAP-supported schools was the overall strength of the professional community of the school (i.e., the extent to which teachers in a school are reported to share a common set of goals and beliefs, and to have frequent opportunities for collegial interaction). Additional factors were associated with growth in only one of the subjects. Greater progress in reading was associated with professional development related to standards-based reform. Greater progress in mathematics was associated with stronger influence of state or district standards and frameworks on curriculum and instructional decisions, and with longer periods of magnet operation (i.e., magnet programs that had been in operation prior to the 1998 MSAP grant).

Influence of Implementation Time and District Context on Achievement Results

In interpreting the similarity of achievement results for MSAP and non-MSAP schools, it is important to consider the conditions under which dramatic differences might occur. One prerequisite for improved achievement is sufficient time for a program to be fully implemented, for teachers to change their methods, and for students to respond with improved performance on achievement tests. Information from surveys and case studies suggests that these conditions were not always met. Implementation time was particularly problematic for new MSAP-supported magnet schools, which had three years to design programs, acquire materials, train teachers, and implement new methods well and consistently enough to affect student performance. Due to late notification or funding, the use of a planning year, or the opening of a school during the second or third year of the grant, some schools had even less than three years in which to produce measurable results.

Furthermore, for the performance of MSAP and non-MSAP students on standardized tests to differ appreciably, one would expect instructional programs addressing content covered by the tests to differ in the two types of schools. Given the context of high-stakes accountability in which all schools operate, case data indicate that non-MSAP as well as MSAP schools focus their efforts on improving teachers' practice and student performance using a combination of regular and special funding sources (of which MSAP is only one), thus diminishing the differences between MSAP and non-MSAP instructional programs.

Magnet schools are expected to adopt distinctive themes and innovative programs, designed to promote a positive school climate and professional community among teachers. These conditions, in turn, are expected to lead to effective instructional practices and ultimately to improved student achievement. Findings, based on interview and survey responses, include the following:

• Magnet themes. MSAP schools have adopted a wide variety of themes. Over one-third of MSAP schools include technology among their themes, and more than a quarter of MSAP schools include a science theme. Arts, communication, and mathematics are also common themes.

• Adoption of comprehensive school reform models. More than half of the MSAP schools have adopted comprehensive school reform models, such as Success for All. This is a substantially higher proportion than is observed among the full national population of Title I schools. It is also higher than among comparable non-MSAP schools in the MSAP districts.

• School climate and community. MSAP schools differ in some organizational and instructional features from comparison schools in the same district. MSAP schools on average have a somewhat more positive sense of professional community than comparison schools. For example, principals of MSAP schools are more likely to report that staff members are supportive and encouraging of each other than are principals in non-MSAP schools. MSAP principals are also more likely to report that administrators and teachers collaborate to help make the school run effectively than are principals in non-MSAP schools. MSAP schools also have a somewhat more positive school climate (i.e., fewer student disengagement and behavior problems), although there is considerable variation among schools.

• Instructional practices. According to teacher survey responses, MSAP schools make somewhat more use of technology in instruction than do comparable schools, and place more emphasis on instructional methods designed to elicit higher-order thinking skills, such as open-ended projects and presentations.

While MSAP schools are intended to adopt innovative themes and practices, they are also expected to ensure that these programs are aligned with state and district standards and assessments. We hypothesized that magnet schools would be more likely to flourish if their themes and programs were aligned with state and district standards and assessments, but we also anticipated that innovative magnet themes might at times conflict with the emphases in many state and district assessments. Findings, drawn from surveys and in-depth case studies, include the following:

• Respondents to the MSAP Principal Surveys report a high degree of familiarity with standards and assessments, and indicate that the content of state standards and assessments match the goals of their magnet programs.

• The case data support the conclusion that magnet themes are generally consistent with the content emphasized in state standards.

• Case data also provide some evidence of tension between the goals of innovative instruction and systemic reform. Staff in some MSAP schools reported feeling pressured to learn how to teach a new theme/curriculum while simultaneously being mindful of state content standards and assessments. It also appears that some MSAP schools altered their initial plans in order to bring the curriculum more in line with standards and assessments, or reduced their emphasis on novel programs to increase the time for work more directly related to state standards and assessments.

The results we have obtained must be understood in terms of the strengths and weaknesses of the evaluation design. Two strengths of the design are:

• The study made use of data on the full population of MSAP-funded projects and schools. We conducted telephone interviews with all of the local MSAP project directors in the first year following implementation and during the second and third years of implementation. We also conducted surveys of MSAP principals during the second and third years of implementation, achieving more than a 90 percent response rate each year. These survey data for the full population are accompanied by data on school-level demographics and achievement, drawn from national databases and MSAP annual progress reports.

• Detailed case study and teacher survey data were collected in a sample of MSAP and matched comparison schools in eight in-depth case study districts. This information allowed us to contextualize findings from our more quantitative analysis of desegregation and achievement outcomes.

The evaluation data, however, have some important limitations:

• Our results on school practices in the full population of magnet schools are based entirely on principal self-reported data. Analyses comparing principal and teacher reports for schools in the case study districts indicate that conclusions based on principal and teacher reports are reasonably similar for questions that were asked of both sets of respondents. Nonetheless, it would be preferable to have data from teachers for a larger sample of MSAP schools.

• Our data on achievement are restricted to aggregate school data. We had initially planned to conduct a more powerful study to gauge the impact of magnet schools on student achievement using linked-longitudinal student-level data. This approach would have allowed analyses of student growth in magnet and comparison schools that took account of prior achievement and other student background characteristics. We were prevented from doing so by a Department of Education moratorium placed on the collection of individual student-level data pending resolution of issues pertaining to the Family Educational Records and Privacy Act (FERPA). Given the increasing emphasis on providing evidence on the achievement effects of school interventions, it would be useful to explore ways in which individual-level data might be made available for evaluations without compromising important privacy concerns.

• The evaluation was not able to make as much use as planned of student achievement and enrollment data collected from the annual performance reports of the MSAP projects. The information on student achievement provided usable school-level data for about one-third of the general achievement objectives described in MSAP applications. Some of the reporting problems included lack of clarity of the objective and its measurement, changes in the assessment or the metric in which it was reported, changes in objectives, and other inconsistencies in reporting. Mathematics and English language arts outcomes could be assessed for about half of the schools in the first year of the grant and just under two-thirds for the latter two years. Additionally, the enrollment data provided by grantees was not sufficiently standardized for the purposes of analyzing progress in reducing minority group isolation, so we made use of the Common Core of Data from the National Center for Education Statistics.

Overall, our results indicate that MSAP projects and schools achieved mixed results over the three years that are the focus of our evaluation. In particular, most MSAP schools appear to have adopted innovative themes, and there is some evidence, based on both survey and case study data, that MSAP schools on average were able to establish a somewhat more positive school climate and level of professional community than other schools serving similar students in the same districts. In addition, our survey and case study data indicate that MSAP schools made efforts to align their programs with state and district standards and assessments.

At the same time, MSAP schools made only modest progress in preventing, reducing, or eliminating minority student isolation and improving student achievement. There is some indication, however, that MSAP schools with more positive program features (e.g., school climate) outperformed comparison schools in student achievement.

These results suggest a number of approaches that might be taken to enhance the prospects that MSAP schools will produce positive desegregation and achievement outcomes.

• Narrowing the focus. It may be useful to permit districts to prioritize the goals or purposes of MSAP projects to achieve one or two focused outcomes rather than attempting to pursue multiple, sometimes competing, goals with limited resources. Case study data suggest that projects sometimes had difficulty balancing the four goals required of them under MSAP's authorization under IASA. As reauthorized under NCLB, MSAP projects are now expected to focus on six goals, potentially making it even more difficult to achieve these goals. Narrowing the focus--and evaluating the grantees on this narrower focus--might improve the chances for positive results.

• Re-examining the definition of minority student isolation. Federal regulations currently define minority group isolated schools as those in which more than 50 percent of the students are minority group members. In light of the high proportion of minority students in urban school districts, federal policymakers might wish to re-examine the meaning and utility of equating minority group isolation with a single fixed percentage. Given the high percentages of minorities in the large urban districts that MSAP typically serves, it was not surprising to find that the desegregation objective of targeted schools was overwhelmingly to reduce, rather than eliminate or prevent, MGI. As the proportion of minorities in schools generally continues to rise, there would seem to be a diminishing opportunity for schools to prevent or eliminate MGI as it is currently defined without adversely impacting other schools in the district. Policymakers might wish to examine more broadly the meaning of minority group isolation in an increasingly diverse population.

• Awarding MSAP grants in a more timely manner. The timing of a grant award is critical to the first year implementation of recruitment efforts and magnet programs. Inability of districts to begin recruitment efforts and delays in first year programming are particularly likely when funds are not received until June or later of the year the program is expected to begin operation. If funds were awarded by March, or even earlier, of the calendar year the project were to begin, districts might be able to implement more effective recruitment efforts for the first year of magnet programs, and schools would have more time to secure materials, training, and personnel for the first year of the program.

• Extending the period of funding. The MSAP program might provide more than three years of funding. Three years may not be sufficient to plan, develop, and implement curriculum and expect to see change in enrollment and student performance around a new theme. It may take several years for magnet school programs to build a strong reputation before they can attract students from outside the immediate neighborhood. Whether or not the period of funding is extended, it would be helpful to continue to examine school outcomes for a period longer than three years.

• Improving the use of annual performance benchmarks. Districts receiving MSAP funds are required to set annual performance benchmarks for improvements in minority student isolation and student achievement, and to report on their success in meeting these benchmarks. The benchmarks districts set for their schools varied considerably in ambition and plausibility. If districts are required to set benchmarks, additional technical assistance may be required to ensure that the benchmarks are meaningful and that outcomes are monitored consistently.

As we have indicated, the data collection for this evaluation was conducted prior to the enactment of No Child Left Behind (NCLB). Our results do, however, suggest that the provisions of NCLB may have special implications for magnet schools.

• Providing assistance in the use of disaggregated achievement data. By 2005-2006, all states must conduct annual testing in grades 3-8, and at least one grade from 10-12. In addition, achievement scores must be reported disaggregated by ethnicity and other subgroups. The availability of these data will enhance the capacity to examine achievement outcomes, and, in particular, to determine whether magnet schools are effective in closing the achievement gap between minority and nonminority students. As such data are increasingly available, it may be useful to provide technical assistance to MSAP grantees to encourage appropriate uses of disaggregated achievement data in evaluating magnet schools.

• Supporting district choice systems. Under the provisions of NCLB, districts are expected to offer a choice of schools to students enrolled in schools that fail to meet adequate yearly progress standards for two consecutive years. Magnet schools may be particularly attractive options for families with students in underperforming schools, and thus MSAP projects may have particular strengths in assisting districts to build coordinated district choice systems.

• Providing support for magnet schools that fail to meet adequate yearly progress requirements. Under NCLB, schools that fail to make adequate yearly progress for two consecutive are expected to revise their plan for the use of Title I funds and engage in other interventions. Magnet schools confronting a failure to meet adequate yearly progress standards may face the additional challenge of maintaining the continuity and integrity of the school's distinctive mission or theme while incorporating changes in curriculum and instruction to improve test scores.

[1] P.L. 92-318, Title VII, Sec. 701-720, June 23, 1972.

[2] ESAA was eliminated as a separate program in 1981, when it was consolidated along with more than 30 other programs as part of a block grant program under the Omnibus Budget Reconciliation Act of 1981.

[3] 20 U.S.C. 7202

[4] Initially 292 schools were funded, but the magnet programs in six schools were not operational and two MSAP elementary magnet schools combined into a single school. Of the six schools in which the magnet programs were not operational, two schools were closed, two schools have programs that never became operational during the 1998-2001 funding cycle, one school was removed from the district and taken over by the state, and one had its magnet status removed by the district after the first year for low performance. Four of the schools were elementary and two were middle schools.

[5] The estimates of magnet schools are based on data from the 1999-2000 Schools and Staffing Survey of the National Center for Education Statistics.

[6] In most cases, the school in which an MSAP-supported program is located is also the school that is targeted for desegregation. In a few cases, the magnet program is designed to draw students out of one or more other desegregation-targeted schools that may or may not receive MSAP funding. Therefore, there are 292 MSAP-funded magnet schools but 294 targeted schools.

Return to Report Contents