A r c h i v e d I n f o r m a t i o n

Mapping Out the National Assessment of Title I: the Interim Report - 1996

Mapping Out the National Assessment of Title I: The Interim Report

Executive Summary

Background

As the largest single federal investment in schooling, Title I of the Elementary and Secondary Education Act (ESEA) [P.L. 103-382] provides almost $7 billion to school systems across the country to improve education for children at risk of school failure who live in low-income communities. It reaches more than 6 million children annually, primarily in the early elementary grades; one in five first graders participates. Typically, Title I supports supplementary instruction in reading and math.

Although Title I has operated for more than 30 years, its reauthorization in 1994 redesigns the program in fundamental ways. (See summary of key changes.) To evaluate the progress of the redesigned Title I in achieving its aim of helping children at risk of school failure to meet high standards, Congress mandated a National Assessment of Title I (NATI). Reauthorization of the Chapter 1 program as Title I relied heavily on the findings of the previous national assessment to help inform its redesign. Evaluation of the reauthorized Title I will concentrate on two major questions:

Impact -- What is the impact of Title I on student performance for students at risk of school failure?

Management improvement -- To what extent are changes in Title I supporting state and local efforts to improve the quality of education for students at risk of school failure?

Congress has mandated an independent review panel of leading researchers, educators, and concerned citizens to advise the Department on evaluation questions and research strategies to address significant areas of concern. The panel brings a wealth of experience in Title I and educational improvement to the National Assessment. The National Assessment of Title I will draw heavily on the Panel's expertise in the design, implementation, and reporting of evaluation studies and their results.

Challenges in Evaluating Title I

The redesign of the Title I program offers several challenges for evaluating the program.

Collecting information on end-result and intermediate performance indicators. The mandate in Title I requires the NATI to examine both student and system performance at the national level. This involves mapping back from the key objective of the program--to improve student performance--to the kinds of supports that will be needed at the classroom, school, district, and state levels to achieve improvement.

Indicators are needed that measure both end-results and intermediate changes in policies and practices that affect end-results. These performance indicators, as required under the Government Performance and Results Act (P.L. 103-62), will be used to report the program's progress in achieving its objectives. Moreover, the NATI will need to understand how important changes in policy and program operations support improved student learning in schools and classrooms. Improved student performance will demonstrate the success of Title I, but improvement takes time and the support of sustained policy and programmatic changes at the federal, state, local, and school levels.

Seeing Title I operate within a system of support for improvement. Title I is no longer conceived of as an isolated supplement, operating separately from regular instruction and state and local reform efforts.

Evaluations also cannot be viewed as separate from a system that provides feedback on how well components are operating, where problems are being encountered, and where solutions have been developed.

Providing continuous, timely information. The extent to which changes in the Title I law will translate into changes in practice is largely unknown. Adding uncertainty is the unfolding state and local context in which Title I operates.

Evaluations of Title I that summarize outcomes at the end of a five- or six-year reauthorization cycle come too late to affect changes in program implementation and operation that can have an impact on results. Consequently, evaluations of Title I must provide timely information throughout the reauthorization cycle so that policymakers and practitioners can make midcourse corrections to improve program performance.
Coordinating Title I evaluations with other data collections at the state and national levels.
This will ensure that the National Assessment of Title I adds maximum information value while avoiding duplication.

Key Indicators of Title I Performance

Critical indicators of performance include the following measures that will be tracked to assess the impact and the implementation of key components of Title I.

The impact on improved student performance--

Students in high-poverty schools will show gains in reading/language arts and math at least comparable to those of other students in their state, reversing the downturn in disadvantaged communities in recent years.

The link between Title I and the establishment of challenging standards and assessments--

By 1995-96, states will show progress in developing and implementing high standards to improve teaching and learning for all students in at least reading/ language arts and math.

The content and performance standards will be challenging according to accepted benchmarks, such as voluntary national, international, or recognized state standards.

Title I support for enriching curriculum and instruction--

Students served by Title I will be exposed to challenging subject matter and the effective use of instructional time and resources in Title I schools.

Title I-supported professional development and assistance for improved teaching will be integrated with other resources to address school and classroom needs for improvement.

Flexibility, in providing Title I-supported services, coupled with increased responsibility for student performance--

An increased number of schoolwide programs will indicate that principals and teachers are aware and make use of the flexibility available through Title I and other federal programs.

Waivers that are approved by the Department will show their potential to improve learning and not simply be requested for administrative convenience.

Title I parental involvement that promotes partnerships with families, schools, and communities to support learning--

Evidence will show that families are becoming increasingly involved in school decisions covered under Title I and that schools are more open and responsive to their involvement.

School-parent compacts will address academic and communication goals to improve children's learning.

The effective targeting of Title I resources--

Fewer low-poverty schools will receive funds and funds will be concentrated on a smaller number of schools. Conversely, high-poverty schools will receive a significant increase in funding and more high-poverty high schools will be served.

Evaluation Agenda for the National Assessment of Title I

The evaluation agenda for the NATI focuses on tracking the performance indicators to address questions raised by Congress and by the Independent Review Panel. The agenda intends to answer questions about the extent to which key changes in Title I help improve student performance and support educational reform. Multiple evaluations have been planned to report on performance indicators at various levels of governance. Evaluations, along with information from state plans and progress reports, are designed to feed back into an indicator system useful for assessing progress to improve the program.

Impact evaluations -- What is the impact of Title I on student performance for students at risk of school failure?

The Longitudinal Evaluation of School Change and Performance, as separately mandated by Congress under ESEA, will provide the best possible way to answer the question of impact on student learning of the standards-based features in Title I. As shown with the earlier Prospects study, a longitudinal design permits analysis of the cumulative impact of Title I and other federal support on student learning over time. Moreover, while tracking student achievement, it can point to what is and is not working in schools and classrooms to support improved student learning. Being able to examine both performance and processes simultaneously has great explanatory value.

At the same time, longitudinal studies are expensive. Questions of costs are necessarily involved in decisions over whether to concentrate on one grade level or to examine different levels, whether to test students independently or to rely on state and local assessments, and whether to focus on particular types of Title I schools--including Title I schoolwides and schools that have been chronically identified for improvement--or to aim for a more nationally representative sample.

The NATI will also rely on national and state-level reporting on trends in student performance from the National Assessment of Educational Progress (NAEP). The National Assessment of Title I is working with NAEP staff on collecting and analyzing student results useful for tracking progress in high-poverty schools.

Analyses of state assessments to identify the progress of students in Title I schools on each state's own standards of performance. Initial pilots supported through the National Assessment will evaluate and develop strategies for improving the quality, scope and usefulness of state-reported performance information.

Management improvement evaluations -- To what extent are changes in Title I supporting state and local efforts to improve the quality of education for students at risk of school failure? Are the key provisions of the new Title I being implemented as intended in the legislation? Are they working well both individually and collectively as a system? What are the hurdles to overcome in effectively implementing the new law? The management improvement evaluations are designed to provide Title I administrators, teachers and policymakers with the kind of practical information that they can use to address these issues.

Monitoring the performance of Title I operations at local, state and federal levels will address the question of how well Title I is supporting improvement. Current and planned evaluations and analyses will collect baseline and follow-up information nationally--from the classroom level through the district and state to the federal level.

Fast-response surveys of school principals and teachers provide the first indicators in the information system, offering a current snapshot of school-based perceptions of federal, state, and locally supported reforms and the extent to which reform efforts have begun to influence changes in curriculum, professional development and classroom practice. High-poverty schools will be oversampled in these national surveys to help gauge the awareness of change in schools eligible for the schoolwide option.

The Study of States' Planning and Early Implementation focuses on how the legislative framework and federal resources are incorporated into the context of state school improvement efforts. The study will be replicated midway through the reauthorization cycle to assess full implementation across all states. A subsequent Study of Local Implementation will analyze districts' efforts to implement Title I and other federally supported efforts.
The Evaluation of Federal Efforts to Assist in School Reform, as separately mandated by Congress, will collect indicators--from the customers' perspective--of the federal government's processes and performance in promoting improved state-, local-, and school-level practices under Title I and other federally supported efforts. Review of the process and purpose of waivers requested under the new authority is a particular focus.

The Study of Intercensal Poverty Data, as separately mandated by Congress, supports the examination of district-level data developed by the Bureau of the Census that will be the basis for Title I allocations beginning in 1998.

strong>In-depth management improvement studies will examine issues of particular concern to Congress and the Independent Review Panel. For each topic, these evaluations will evaluate implementation issues and identify good practices to support the activity.

A Study of Intradistrict Targeting and Resource Allocation will examine resource issues, including how districts allocate Title I funds to schools, the poverty data used to determine eligibility, and exceptions made to the rules governing allocations.

The Study of the Impact of Title I Schoolwide Programs will examine how well the extension of the schoolwide option under the new Title I is being used to support fundamental program improvements. A special Study of Migrant Children in Schoolwide Programs, as mandated by Congress, will examine the extent to which schoolwide programs affect the learning opportunities afforded to migrant students.

The Study of Family Involvement in Title I, as separately mandated by Congress, will draw on two special studies, one focused on state policy and the other on school strategies to engage parents in their children's learning.

An Evaluation of Title I Participation of Private School Students will survey a nationally representative sample of districts to examine the impact of changes in allocation procedures on participation of private school students.

Budget Requirements

The National Assessment of Title I is collaborating with other data collectors to ensure that the work is conducted as efficiently as possible. Duplication of data collections will be avoided. However, it cannot proceed with planned studies without sufficient funds. The National Assessment of Chapter 1 and Prospects--the separately mandated longitudinal study used in the Assessment--were funded at about $6 million annually, or approximately one-tenth of one percent of the funds for Part A. The previous National Assessment of Chapter 1 showed the value of evaluations for informing legislative action and program improvement.

Reporting on the National Assessment of Title I

This first report on the new Title I focuses on essential features of the local education agency grants program (Part A); subsequent reports will concentrate on other parts of Title I, including the Even Start, Migrant Education, and Neglected and Delinquent programs. The ongoing NATI will attempt to show where these programs link up and how they work together.

Congress has mandated that the Secretary of Education submit a final report on the NATI by January 1998. The NATI intends to report findings to Congress and to the field throughout the Title I authorization cycle in order to meet the need for continuous performance improvement and for timely reporting. Feedback on performance indicators contributes to continuous progress by suggesting areas and practices that are not working and need correction, and those that are most promising and worth sharing.

-###-

[Forward]