A r c h i v e d  I n f o r m a t i o n

Planning and Evaluation Service

Analysis and Highlights

Even Start:
Evidence from the Past and a Look to the Future

Background

The Even Start Family Literacy Program focuses on the educational needs of low-income families with young children. The goal of Even Start is to help break the cycle of poverty and illiteracy by improving the educational opportunities available to low-income families with limited educational experiences. To reach this goal, all Even Start projects build on existing community resources to integrate adult education, parenting education, and early childhood education services into a unified program.

Federal law mandates an independent evaluation of Even Start projects. The U.S. Department of Education?s Planning and Evaluation Service (PES) has commissioned three four-year national evaluations since the program?s inception in 1989; two have been completed. Data from the second national evaluation have been reported in three interim reports, and are updated with 1997 data in a final report. These volumes have been geared toward a technical or research audience. This volume, prepared by Abt Associates, synthesizes information from national studies and evaluations of the Even Start program and is geared toward a policy maker and practitioner audience. The following findings are organized around the program?s performance objectives and indicators that the Department has established in its strategic plan. Implications for practice and future directions are also included. A companion final report that examines the program in greater detail is being released with this document.

Scope of the Program

• The Even Start program has grown significantly since its inception.
  • In the 1989-90 program year, the Even Start program had $14.5 million in funding and the U.S. Department of Education administered the program?s 76 grantees. In that year, Even Start served over 2,700 children and 2,900 adults. In the 1992-93 program year, with $70 million in funding, the program converted to State administration of 340 Even Start projects serving over 28,000 children and over 23,000 adults. In 1996-97, Even Start?s funding reached $102 million supporting 637 local projects; and serving over 48,000 children and almost 36,000 adults.

Program Performance Objectives and Indicators

Objective 1: The literacy of participating families will improve.

• In most of the areas examined in the Even Start national evaluation, children and parents in Even Start make gains; however except for one small-scale experimental study in five projects in the program?s first few years (the In-Depth Study), there is no control group to gauge whether or not gains are due alone to participation in Even Start.

• Indicator 1.1: Adult literacy achievement. Increasing percentages of Even Start adults will achieve significant learning gains on measures of math and reading skills.

  • Each year the national evaluation assessed progress using measures of adult literacy, Even Start adults achieved significant gains on the reading and math Tests of Adult Basic Education (TABE) and the Comprehensive Adult Student Assessment System (CASAS) reading and math tests. Further, the gains made by Even Start adults are comparable or larger in size than those observed in other studies of adult education programs. However, where data are available on adults not in Even Start, they too make gains, probably because they too take part in adult education programs.

  • In 1995-96, 24 percent of the adults who took the TABE math achievement test achieved a moderate- to large-sized gain between pretest and posttest. In TABE reading achievement, 20 percent of adults achieved a moderate- to large-sized gain. For the CASAS math and reading tests, the figures are 55 and 24 percent of adults, respectively.

• Indicator 1.2: Adult educational attainment. Increasing percentages of adult secondary education Even Start participants will obtain their high school diploma or equivalent.

  • Since its inception, Even Start has helped many adults attain a GED. Depending on the year in question, 8 to 15 percent of all of the adults who entered Even Start without a GED or diploma attained one. In the In-Depth Study, 22 percent of adults in Even Start attained a GED compared with 6 percent in the control group.

  • In 1995-96, 18 percent of all adult secondary education and GED participants for whom the GED was a goal earned a GED.

• Indicator 1.3: Children?s language development and reading readiness. Increasing percentages of Even Start children will attain significant gains on measures of language development and reading readiness.

  • After one year of participation, Even Start children in the In-Depth Study scored significantly higher on the PreSchool Inventory (PSI), a test of school readiness, than children in a randomly assigned control group. However, control group children caught up in the next year, when they entered preschool or kindergarten. A similar pattern was found on the Peabody Picture Vocabulary Test (PPVT), a test of receptive language.

  • An analysis of growth rates for children who remained in Even Start for more than one year shows that children who remained in Even Start longer may grow at a faster-than-expected rate both on the PSI and on the Preschool Language Scale (PLS).

  • In 1994-95, 66 percent of Even Start children made moderate- to large-sized gains on the PSI. In 1995-96, this figure was 81 percent. Forty-four percent of children achieved moderate to large gains on the PLS in 1994-95. In 1995-96, this figure was 50 percent.

• Indicator 1.4: Parenting skills. Increasing percentages of parents will show significant improvement on measures of parenting skills, home environment, and expectations for their children.

  • Parents made significant, moderate-sized gains on the Home Observation for Measurement of the Environment Screening Questionnaire (HSQ), which is intended to measure the quality of cognitive stimulation and emotional support parents provide to their children. By comparison, parents from a control group in a study of the Comprehensive Child Development Program did not achieve gains on the HSQ.

  • During the second national evaluation, depending on the age of the child and the year in question, between 36 and 54 percent of parents scored 75% or higher correct on the HSQ posttest.

  • Even Start families did not exhibit any measurable change on the adequacy of family resources or social support, and there was no measurable change in the self-efficacy of Even Start parents as assessed by self-report measures of sense of mastery and depression. The income of Even Start families rose over time, as did the percentage of families with an employed adult, but in neither case were these changes greater than those seen for control group families in the In-Depth Study.

Objective 2: Even Start projects will reach their target population of families that are most in need of services.

• Indicator 2.1: Recruitment of most in need. The projects will continue to recruit low-income, disadvantaged families with low literacy levels.

  • Family income has been consistently low. At least 90 percent of Even Start families in 1996-97 had incomes at or significantly below the federal poverty level.

  • Forty-five percent of 1996-97 new enrollees had reached at a maximum only 9th grade before enrolling; 42 percent had reached 10-12th grades but had not graduated.

  • The percentage of parents in the Even Start program under the age of 20 has increased from 9 percent in 1994-95 to 13 percent in 1996-97.

  • The percentage of primarily Spanish-speaking parents has risen from 15 percent in 1989-90 to 32 percent in 1996-97. Of 1996-97 new enrollees, three-fourths of those who spoke languages other than English at home were unable to read and/or speak English well or at all. This translates to about 30 percent of all 1996-97 new enrollees.

Objective 3: Local Even Start projects will provide comprehensive instructional and support services of high quality to all families in a cost-efficient manner.

• Indicator 3.1: Service hours. Increasing percentages of projects will offer at least 60 hours of adult education per month, at least 20 hours of parenting education per month, and at least 65 hours of early childhood education per month.

  • Over the years, Even Start projects have significantly increased the amount of instruction offered in all three core service areas. The amount of adult education services offered increased by 25 to 30 percent over the past four years. The amount of early childhood education offered increased by ten percent over the past four years. The amount of parenting education, while generally increasing over time, seems to be stabilizing in the past two program years at around 200 hours per year.

  • In 1995-96, half of projects offered 32 hours or more of adult education per month, 13 hours or more of parenting education per month, and 34 hours or more of early childhood education per month.

• Indicator 3.2: Participation, retention and continuity. Projects will increasingly improve retention and continuity of services.

  • Even Start families participate in core services at higher rates than they would have without the program. About 90 percent of Even Start adults participate in adult education compared with an estimated 30 to 40 percent without the program. About 90 percent participate in parenting education compared with an estimated eight percent without the program. About 95 percent of the children in Even Start participate in early childhood education compared with an estimated 60 percent without Even Start.

  • Families? actual participation in the services offered has varied over the years. However, there is a clear, consistent relationship between the amount of services projects offer and the amount participants attend. The average number of hours in adult education has been holding quite steady at about 95 hours per year for several years. Participation in parenting education has declined from an average of 58 hours per year in 1992-93 to about 30 hours per year for 1994-95 through 1996-97.

  • Of all families participating in Even Start in 1994-95, 38 percent stayed in the program for more than one year. Of new families entering in 1995-96, 41 percent stayed for more than one year.

• The per-family cost of Even Start services from Federal funds has decreased from 1989-90 to the present, but has remained stable across the last four program years.

  • During the first national evaluation, the federal cost per Even Start family declined, from $5,894 in 1989-90 to $3,669 in 1990-91, and to $2,503 in 1991-92. During the second national evaluation, this figure has been in the $2,700-$2,800 range.

Implications For Practice and Future Directions

• In addition to information on the overall effectiveness of Even Start, the national Even Start evaluations have provided data on the effectiveness of selected programmatic practices; specifically that intensity of services, service location, parent/child time, and literacy-based parenting education matter.

  • A substantial body of research on the effectiveness of early childhood education programs shows that gains are enhanced by a high level of exposure to a high-quality, center-based program. Research on Even Start supports this finding in that adults and children with high levels of participation in Even Start's core services had larger learning gains than those with low levels of participation.

  • Children in projects that emphasize center-based programs had larger learning gains than children in projects that emphasize home-based services.

  • Families in projects that have large amounts of time for parents and children together had better home environments (e.g., more materials in the home, parent/child learning activities, approaches to discipline) than families in projects that have smaller amounts of parent/child time together.

  • Early on, projects received strong messages from the federal level to focus on literacy-based parenting education. Findings from the first national evaluation showed that a positive relationship between the amount of parenting education received and children?s vocabulary test scores. More recently, there is anecdotal evidence that parenting education has become less literacy-focused. If this is the case, it is one possible explanation for the disappearance in the second national evaluation of the relationship between amount of parenting education and child test gains.

  • The size of the project does not seem to matter; children and adults seem to do equally well in small and large projects. In addition, the number of staff, the type of degree held by staff, and years of experience are not related to learning gains.

• Even Start has a useful reporting system for all grantees that informs Congress and the national program, but the evaluation also needs to provide data to help local projects improve their services. Future research and evaluation options include:

  • Focusing on literacy outcomes for all projects. Information about Even Start participants? progress toward literacy skills is key at all levels. The national evaluation could designate a common set of rigorous and objective outcome measures, and give local projects flexibility by allowing them to choose from a set of approved measures.

  • Strengthening local evaluation for continuous improvement. A recent review of a sample of local Even Start evaluation reports calls into the question the usefulness of these efforts to local projects. One way to strengthen local evaluation is to reframe local evaluation as a continuous improvement effort. What the continuous improvement model could add to the evaluation process is the refinement of clear, concrete program outcome goals that can be used as a framework for projects to identify promising strategies for service provision and to use in critically examining what they do. Projects would assess participants? outcomes and use this information for program improvement.

  • Examining the quality of Even Start services. Options for assessing program quality include: a self-assessment examination of the inputs, a comparison of program outputs with some set of performance standards, and a study of the collaborations that Even Start projects arrange.

  • Examining long-term effects of Even Start. Given the extreme level of need of Even Start families and the fact that most families leave within their first year of participation, many of the potential outcomes of Even Start may occur only after the period of Even Start participation. A study of families over an extended period of time and an examination of their success after being in the program would provide information on the long-term value of the short-term gains measured in the national evaluation.