A r c h i v e d  I n f o r m a t i o n

Planning and Evaluation Service
Analysis and Highlights

Living in Interesting Times:
Early State Implementation of New Federal Laws

"Living in Interesting Times: Early State Implementation of New Federal Laws," provides a baseline evaluation at the state level of the Improving America?s Schools Act [IASA] (P.L. 103-327) and the Goals 2000: Educate America Act (P.L. 103-227) in the first two years of implementation. The study analyzes the ways in which the state officials who administer each of several funded programs initially responded to the new legislative provisions regarding flexibility, accountability, and capacity building, and provides data, which will serve as a baseline for follow-up state-level evaluations. The programs included in the study were: Title I (Helping Disadvantaged Children Meet High Standards), Title II (Professional Development), Title IV (Safe and Drug-Free Schools) and Title VI (Innovative Grants) of the Elementary and Secondary Education Act (ESEA), the McKinney Education for Homeless Children and Youth Program, and Title III of Goals 2000.

The evaluation is informed by surveys administered to state education agency (SEA) managers of the federal programs in all 50 states and the District of Columbia in late 1996 and early 1997. A core set of questions about program procedures was administered to the managers of all the programs. Tailored questions about targeting, assessment, and program-specific topics were added for most programs.

The purposes of the evaluation were derived from a larger framework of data collection and analysis for the National Assessment of Title I and a comprehensive analysis of federal assistance to states, local districts and schools, mandated under Secs. 1501 and 14701 of the ESEA. A follow-up state-level evaluation, which will build on the findings reported here, is underway. The study will analyze the ways in which administrators of federally funded programs have continued to respond to the procedural provisions in the legislation—such as consolidated planning and cross-program communication—and have begun focusing on the task of aligning program services and operations with state content and student performance standards, so as to improve student achievement.

Key Findings

Flexibility: Do States and Districts Experience New Latitude in Meeting the Laws? Challenges?

• State administrators offered varied perspectives regarding the extent to which their own flexibility had increased in the first two years after ESEA reauthorization.
• Title I was the only program for which most administrators reported greater flexibility as administrators of other programs, particularly Title VI, Title II?Eisenhower, and Title IV?Safe and Drug-Free Schools and Communities argued that their programs always had offered flexibility.
• In describing newfound administrative flexibility, program administrators pointed to the opportunity to coordinate and collaborate with other federally funded programs, the opportunity to approve a wider range of local program designs, and the pooling of state staff and other resources.
• Perceptions of flexibility were also related to key program changes. Indeed, the more areas of administration in which program managers reported making key changes since the reauthorization (i.e., in procedures for local applications, monitoring, professional development, or technical assistance), the more likely they were to report finding that the legislation gave them more administrative flexibility.
• Consolidated planning was a key factor in influencing SEAs? changes to their procedures for implementing the programs studied.
• Almost all state administrators participated in the development of their state?s consolidated ESEA plan in 1996. While the actual level of their involvement cannot be determined, they all report having been "at the table."
• It was not clear based on administrators? comments however, how consolidated planning was translating into coordinated program services and operations; many administrators seemed just to have planned for planning?s sake.
• Very few state administrators described consolidation of administrative funds as useful in supporting comprehensive planning.
• In only two of the nine states, which reported an across-the-board consolidation of state administration funds, did administrators report that consolidation had affected their own work "to a considerable extent," and many perceived the effects as negative (e.g., more steps needed in coordination).
• Among administrators across all states who believed their agency had consolidated administrative funds, 55 percent reported at least some effect on their own work. Often, they cited the lifting of a requirement to keep "time and effort" logs; some said that this helped them work more effectively and efficiently.

Accountability: To What Extent are States Using Performance Data to Inform their Efforts?

• With the exception of following some mandated procedures?such as identifying Title I schools in need of improvement?most program administrators gave only limited evidence that they were attending to student performance.
• Very few program administrators reported any systematic way of evaluating the success of their program; almost none referred to student performance data. Most administrators judged their successes based on feedback from subgrantees, their own observations, and other anecdotal information.
• Despite the infrequent use of program performance indicators as an evaluation tool, they seemed to have held value among those program administrators who did report using them.

• Many program administrators, across the majority of states, said that assessment was the area in which their state had the farthest to go in meeting its own reform goals.
• While some administrators seemed to anticipate reasonably smooth progress in assessments, others were concerned that their states would lack the will or the capacity to put aligned assessments in place. Indeed, the ESEA legislation reflected a more ambitious plan for standards and aligned assessments than the states were prepared to undertake.
• States had made progress in dismantling their old systems of monitoring for compliance with program provisions. They had far to go, however, in building monitoring procedures that would send a clear message to districts about a new, standards-based accountability framework.
• State program administrators reported that program effectiveness had become the focus for monitoring. However, program managers were most likely to report conducting on-site monitoring visits in districts that invited them in. Very few program managers (13 percent) reported that monitoring visits were triggered by information about student performance.
• Integrated monitoring visits appeared to be the coming trend among federally funded programs; already, about 27 states conducted some form of integrated monitoring visits. However, many of the states? integrated monitoring teams did not include some of the smaller discretionary grant programs (e.g., Even Start, Migrant Education, and Homeless Education).

Technical Assistance and Professional Development: Are States Working Strategically to Build Capacity?

• Programs varied a great deal in their attention to building local capacity around standards, assessment, or whole-school improvement; these topics were reported in Title I but less so in other programs.
• Agency downsizing in many SEAs had reduced the technical assistance capacity in federal programs. With limited resources, program administrators focused primarily on assisting districts that asked for help. A much smaller percentage gave priority to districts with low achievement.
• The content of technical assistance varied by program. Most noted their efforts focused on "meeting the needs of special populations." However, more than two-thirds of Title I and Goals 2000 administrators did show coordination in the content of assistance they provided?particularly standards, assessment, whole-school improvement, and data-driven decisionmaking.
• Variation existed within any given state, regarding the content of professional development.
• Virtually every state administrator of Title I and an overwhelming majority in Eisenhower reported that they encourage districts to pool funds for professional development across federal programs.
• However, federal program administrators seemed more inclined to encourage professional development on a broad range of topics rather than a tight focus.

Conclusions

In the first few years of implementation of Goals 2000 and the reauthorized ESEA and homeless programs, SEAs took major steps toward cross-program communication in their own agencies, and many program administrators communicated a message of broad program changes to their districts. Where implementation fell short of the original federal vision was in accountability for results. Indeed, having dismantled their old monitoring systems, few state agencies were sending a strong message to districts that accountability for student performance would replace compliance monitoring of the past.

Copies of this Planning and Evaluation Service report are available by contacting the U.S. Department of Education at 1-877-4ED-PUBS.

[Return to Planning and Evaluation pages]