A r c h i v e d  I n f o r m a t i o n

Chapter 9: Conclusions

Lessons Learned About Evaluation of the Even Start Program

A number of valuable insights about evaluation design have emerged as a result of the long-term study of the Even Start program. One consequence of the flexibility of program design has been that local projects can tailor service delivery on the basis of locally available resources to meet the needs of individuals within their own communities. However, the same flexibility has consequences for the national evaluation, because the design, content, organization, and staffing of services all vary across local projects. Not only is there variation across local projects, but staffing and service delivery patterns may change within projects from year to year. These changes, coupled with the indicators of greater variability in demographic characteristics, mean that an evaluation must be able to assess progress for participants who speak different primary languages, who participate for differential amounts of time, and whose children may be too young to participate in the assessment measures used in the current evaluation.

Further, variation across these dimensions affects the capacity of the national evaluation to identify effective programs and practices. As described above, the definition of appropriate services has evolved over the past several years (vis-à-vis the legislation). It has become increasingly clear, for example, that the definition of appropriate services requires projects to serve both as a provider of educational services and as the "glue" that meaningfully connects and supports families' educational experiences. Some elements of the comprehensive management information system (or ESIS) data collected as part of the national evaluation changed to reflect refinements and emphases defined by the changes in the relevant legislative language. Even though the ESIS data have changed somewhat over the course of the second national evaluation, the data collected are not yet specific enough to assess accurately the content of services provided. Nor is the ESIS, even after modifications, fine-grained enough to be able to identify effective practices and program elements, or to be able to document how Even Start functions both as a provider of educational services and as the "glue."

-###-

[Lesssons Learned About the Even Start Program]

[Insights into Evaluation of the Impact of the Even Start Program]