[Chapter 9: Conclusion]
[Lessons Learned About Evaluation of the Even Start Program]
Chapter 9
It is clear that the Even Start program has indeed begun to achieve some of its overarching goals. Even Start has been able to provide unified family literacy experiences for children and their families through an integrated program of early childhood education for children, parenting education, and adult literacy and adult basic education for adults. Participation in Even Start has led to improvements in children's school readiness and language development and in adults' literacy skills, as assessed by various outcome measures across two four-year evaluations. During the first national evaluation of Even Start, the administration of the program also moved from the federal to the state level as the number of funded projects increased substantially. Further, as time went on, and the administration of projects shifted to state control, the federal share of the projects' total operating costs was designed to decrease annually while the non-federal share increased.
We also learned about the early impact of the Even Start program. The first national evaluation indicated that children participating in Even Start made greater gains on some measures than children in a comparison group, and that Even Start adults were more likely to complete the requirements for attaining a General Educational Development certificate (GED) than were adults in a comparison group. We learned that participants progressed on some of the educational and developmental measures used, but that the progress of Even Start participants was not consistently better than the progress of individuals in a comparison group.
Some of the findings from the first few years of the program and its evaluation led to changes in the legislation re-authorizing the program, including strengthening the targeting of families most-in-need in their respective communities; providing intensive instructional services in all three core areas; offering services on a year-round calendar; providing services to children in at least a three-year age range; allowing projects to serve teen parents within the mandatory school age (i.e., those not previously eligible for adult education); and continuing to demonstrate coordination, not duplication, of available services. At the federal level, the guidelines governing the Even Start program continued to change, reflecting the increasing awareness of the need to specify some elements of service provision while retaining the flexibility of locally-determined service delivery, and while ensuring that the Even Start program continued to serve those most in need. Some aspects of the Even Start program design have purposefully remained unspecified, such as establishing minimum amounts or duration of instruction to be offered, or identifying and/or recommending particular curricular or pedagogical approaches to instruction.
Many of the programmatic trends observed earlier have persisted. At the national level, the growth of the program has continued; the number of projects has increased from 439 in 1993-94 to 637 in program year 1996-97. This reflects the continued support for the Even Start program at the federal level. Although the amount of instruction offered and received in core instructional areas has increased over the average levels of the first four years, the changes in the second four years appear to be leveling off. The qualifications and experience levels of Even Start staff have remained stable. The patterns of service delivery, whether provided by Even Start paid staff or staff from collaborating agencies, appear to have stabilized.
There have been changes to the populations being served by the program as the numbers of projects and participants have increased. The Even Start program is now serving more teen parents, greater proportions of Hispanic/Latino families, and more families with greater evidence of disadvantage (as reflected in the composite need index described earlier in this report). The increase in the Hispanic population in Even Start comes at a time when other federal programs (e.g., Head Start) have been much slower to experience increases in the participation rates of Hispanic populations (Administration for Children, Youth and Families, 1992, 1997). At the national level, Even Start has clearly widened its reach to include more diverse populations, including greater numbers of those families most in need.
Some of the changes in the demographic characteristics of participants have consequences for the nature of services offered and received. Teenage parent participants, whose children are younger, on average, than the children of other participants, may have different needs both for early childhood education services and for adult education services. Non-native English speakers may have different needs for adult literacy education—in their own languages as well as in English—than native English speakers.
Changes in state-level requirements for recipients of public assistance are beginning to translate into changes in demand for employment-related or vocational education, and may also lead to changes in parents' availability to participate in Even Start services. Along with changes in the composition of the participant population, the patterns of participation have changed as well. Families with teen parents do not remain enrolled in the program as long as older parents, on average; those with higher (or needier) average scores on the need index participate intensively for limited amounts of time. Families whose primary language is not English are more likely to remain active participants for longer periods of time.
These patterns clearly reflect the variations in program participation by different participant groups. They also highlight the recurring tension between providing services to families most in need while sustaining their active involvement. Additionally, these differential participation rates raise some provocative questions about how to design program services to serve the majority of families who participate only for a limited time. To what extent are local projects designed to provide modules of instruction, for example, that are sufficiently intensive and flexible in timing to serve families who participate for three or four months?
Participants in Even Start did make progress on the outcome measures used in the second evaluation. The availability of longitudinal data, which allowed us to investigate children's growth over time, has been one positive feature of an evaluation design fraught with limitations. The fact that we have been able to observe growth over time within individual children on both the PSI and the PLS-3 suggests that participation in a program like Even Start might help spur accelerated learning, as measured by these outcomes. Further, our analyses indicate that children progress at the same rate regardless of family need, although children from families with greater needs consistently score lower, on average, than children from families with fewer needs. It is also clear that the longer children participate in Even Start (and the younger they are upon entry) the greater the gain, or the steeper the growth rate. By contrast, the performance of children who enter Even Start at a later age suggests that the older the child upon entry, the lower the score, on average.
One of the paradoxes we face, however, is that while we have indeed observed changes in outcome measures—particularly for children—the current evaluation design does not allow us to attribute progress to participation in Even Start. When we consider the findings from the first national evaluation in tandem with the current findings from the second evaluation, the value of a strong research design (as existed in the In-Depth Study) becomes even more critical. The first evaluation's In-Depth Study, which examined progress throughout the duration of the study for a much smaller number of participants, offered significantly greater explanatory power about the gains that could reasonably be attributed to the Even Start program because of the random assignment design (i.e., In-Depth Study participants were randomly assigned either to the Even Start program or to a control group).
-###-
[Chapter 9: Conclusion] | |
[Lessons Learned About Evaluation of the Even Start Program] |