[Acknowledgments]
[Chapter 1: Introduction]
The Even Start Family Literacy Program addresses the basic educational needs of parents and children of low-income families by providing a unified program of (1) adult basic or secondary education and literacy programs for parents; (2) assistance for parents to effectively promote their children's educational development; and (3) early childhood education for the children. All Even Start projects are required to provide services in each of three "core" areas corresponding to the broad programmatic goals of Even Start: adult education and literacy; parenting education; and early childhood education. Projects provide direct support for services and also build on existing community resources by collaborating with other service providers.
The Even Start program was initially authorized by 1988 amendments to the Elementary and Secondary Education Act of 1965 (ESEA), then amended by the National Literacy Act of 1991 (P.L. 102-73). In 1994, the Even Start program was reauthorized by the Improving America's Schools Act (P.L. 103-382) as Part B of Title I of the ESEA.
Even Start was first implemented as a federally-administered program in fiscal year 1989. Since 1992, the program has been primarily state administered. The states award subgrants to partnerships, each consisting of at least one local education agency (LEA) and at least one community-based organization, institution of higher education, or other public or private non-profit agency. A few types of projects remain under federal administration, including: special set-aside programs for migrant agricultural families, Indian tribes and tribal organizations, and insular areas; discretionary grants for statewide family literacy initiatives; and a family literacy project in a prison that houses women and their preschool-aged children. Since 1989, the program has grown from seventy-six projects serving approximately 2,500 families to 637 projects serving approximately 34,400 families eight years later in 1996-97.
Two types of evaluation are required of all Even Start projects by law: an independent local evaluation arranged for by each project and a national evaluation conducted by the U.S. Department of Education. The first national evaluation documented the program's early development from 1989-90 through 1992-93. Continuing the same objectives from the first national evaluation, the second national evaluation, covering the program years from 1993-94 through 1996-97, addressed these questions:
Since the inception of the Even Start program, data on program participants, implementation, and progress indicators have been collected annually. In the second national evaluation, approximately 95 percent of all local projects provided data on their participants and program implementation each year. In addition, a sample of fifty-seven projects also provided data on program outcomes.1
Building upon key findings from the 1994-97 evaluation as a basis, this report reviews what has changed and what has remained stable over the first eight years of the Even Start program. Rather than serve as a final, summative account of program accomplishment, the report is intended to serve as a springboard for continuing program improvement and evaluation efforts that are currently underway.
The second national evaluation assessed programmatic and participant progress through two chief strategies: one was to collect information on all projects about all participants, and this was called the Universe Study. The second strategy, called the Sample Study, was to collect more detailed educational and developmental outcome data on a subset of participants from approximately 10 percent of the Universe Study. The Sample Study collected outcome data using measures designed to relate participation in Even Start services to specific, measurable educational and developmental outcomes for both adults and children. As in the first national evaluation, the Sample Study measured three types of educational/developmental outcomes: child cognitive development, adult education, and parenting. The second evaluation did not include a control group, and program outcomes were assessed in terms of the differences between participants' pretest (i.e., pre-Even Start) and posttest scores on a battery of standardized tests.
The outcomes presented in this report are based primarily upon data from those participants who remained in Even Start long enough to participate in at least two rounds, or waves, of data collection. For most participants, this translated into an assessment at entry into Even Start, and one additional assessment during the same program year, and for a few participants, another round of assessment in the subsequent year. However, when we contrasted demographic and other characteristics of families for those with only pretest data (or those with one wave) versus those with two waves, we observed systematic differences between the two groups. Essentially, families with more complete data are more likely, on average, to be employed, have higher incomes, speak languages other than English at home, and have completed more schooling. This means that the results we describe from the Sample Study—based upon participants who remain in the program—represent a biased subgroup of the Even Start population. This critical caveat should be kept in mind when reviewing the discussions about educational and developmental outcomes that follow.
Children also showed progress on the Preschool Language Scale (PLS-3), a measure that assesses language development.
We examined children's growth on these two measures by using individual growth modeling to investigate two related questions.
These analyses allowed us to capitalize on the truly longitudinal data collected on the same children over time. For both the PSI and the PLS-3, children with two or more waves of data made clear progress on the outcome measures—more progress than one might expect on the basis of development alone.
The fact that we have observed this pattern in the PLS, a measure designed to have the same standardized average score of 100 no matter what the child's age, provides positive evidence of the benefits of participation in Even Start. Because standardized scores on the PLS do not increase as children develop (e.g., the scores are standardized to remain the same at different age intervals), the growth we observed suggests that participation in Even Start has a demonstrable and positive effect on children. 3
The educational progress for Even Start adults, however, has consistently been more modest than was the case for children. Adults were assessed with either the Comprehensive Adult Student Assessment System (CASAS) or the Tests of Adult Basic Education (TABE) in both reading and math subject areas. Over the course of the Sample Study, adults made gains on these measures.
The TABE was not used in the first four-year evaluation of Even Start, so we cannot compare results on this measure to results from the earlier national evaluation.
The differences between adults of varying educational backgrounds were more pronounced on the math tests; adults who pretested below a 7th-grade level gained less than one grade level by posttest, while those who entered with skills at or above the 10th-grade level gained over two grade levels.
Participating adults' educational progress was also examined in terms of GED attainment.
The rate of GED attainment for adults who entered in 1994-95, for example, who also participated in 1995-96, increased to approximately 11 percent, and it increased to 14 percent for those who continued to participate in 1996-97. However, only about 40 percent of adults continued to participate into the second program year, and only 14 percent continued into 1996-97. This pattern of slowly increasing percentages of GED attainment appears to persist for adults who entered in the 1995-96 program year, although we only have data on GED attainment for one additional program year.
In addition to educational assessments for children and adults, the outcomes included a measure of the home environment, called the Home Observation for Measurement of the Environment (HOME) Screening Questionnaire. The HOME measures the quality of cognitive stimulation and emotional support provided to the child(ren) by the family.
While the second Even Start evaluation did not include a control group, a control group of low-income families participating in a national evaluation of another federally-funded "two-generation, family support" program showed no change in HSQ scores over time. This finding suggests that there is no "normal" or "developmental" growth associated with this measure. This further suggests that the positive changes observed among Even Start families on this measure may be due to the program rather than to other factors.
The findings about progress on outcome measures are modestly positive, yet we must emphasize that the Sample Study component of the second evaluation, unlike the In-Depth Study (IDS) component of the first evaluation, depended upon local projects administering child and adult tests and collecting and submitting data on outcome measures. Sample Study project staff were initially trained in August and September of 1994. Although there were no other meetings for the Sample Study projects, the projects received refresher training manuals in September 1995 and September 1996. While many projects worked hard to ensure that annual data submissions were accurately completed, both the quality and quantity of data submitted by the local projects were extremely variable. We believe this was due to a combination of factors, including changes in testing and administrative personnel at the project level. As a result, we believe we must interpret our findings with considerable caution because the data may not reflect participants' performance as accurately as we had hoped.
Also, the Sample Study, like the In-Depth Study component of the first national evaluation, was intended to collect three waves of data: the first at pretest, the second at posttest (at the end of a participant's first full program year), and the third wave at the conclusion of the following program year, in order to assess the impact of participation in Even Start over a longer time period. What we learned, however, is that most participants in the Sample Study remained in Even Start long enough to participate in one wave, and in some cases, two waves of data collection, but only about 10 percent of the Sample Study participants were still enrolled in Even Start at the time of wave three. Consequently, we can assess changes in performance from wave one to wave two for many of the Sample Study participants, but our estimates of changes from wave one to wave three are based upon a potentially biased and limited subsample of participants (e.g., those 10 percent or so of participants still enrolled in the program at the end of their second full program year).
Over the past three years, we have collected data on a number of outcome measures for children and adults participating in Even Start and have found the same trend from year to year.
In the first national evaluation, for example, we observed a relationship between service intensity and educational outcomes for children and adults (e.g., the greater the level of participation in Even Start services, the greater the increase in children's gains on the Peabody Picture Vocabulary Test). During the first year of the second national evaluation, we observed some relationships between service intensity, such as hours of adult education or parenting education, and outcomes such as adult scores on the CASAS, TABE (math only), or the HSQ. For the past two program years, however, the data do not indicate any consistent relationship between service intensity and educational outcomes.
Additionally, the variability in data quality increased over time for the Sample Study, and our confidence in the data diminished as a result. Because the relationships are inconclusive and in fact differ somewhat across years, we can only report that despite the gains participants clearly made on all the outcome measures, the explanations for those gains have not been consistent across the several years of the second national evaluation.
To understand the extent of program outcomes achieved by Even Start, the evaluation documents in detail the characteristics of participating families, the extent of their educational needs, the services that are offered by Even Start projects, and the extent of their participation in these services.
Even Start is designed to serve low-income families with parents who have limited educational experiences and/or literacy skills. To participate in Even Start, a family must have (a) a parent who was eligible for adult education services under the Adult Education Act, and (b) a child younger than 8 years of age. Beginning in 1995-96, teen parents who were within the state's compulsory school attendance age range, and who were not eligible for adult education services because of their age or school attendance, became eligible for Even Start.
Consistent with the purpose of the program, Even Start projects are serving families who are disadvantaged economically and educationally.
In the 1996-97 program year, approximately 34,400 families, comprising 35,800 parents and 48,200 children, participated in Even Start across 637 projects, based upon data submitted by 95 percent of all projects. The average age of adult participants in 1996-97 was 28 years, one year younger than the average two years earlier. This reflects a steady increase in the enrollment of teen parents from 9 percent in 1994-95 to 17 percent among the 1996-97 new enrollees. The average age of Even Start children in 1996-97 was 4.2 years. Mirroring the increase in the number of teen parents, the number of infants and toddlers served by Even Start also has increased, from 29 percent of all children participating in 1994-95 to 38 percent of new children enrolling in 1996-97.
The family structure of Even Start families has remained fairly consistent over the years—roughly one-half being two-parent families, nearly 40 percent headed by single parents, and the remainder being extended families. As in previous years, the average size of Even Start families was between five and six persons in 1996-97. The typical Even Start family was a couple between the ages of 20 and 39, with two to four children. Another large group of families was headed by single parents with two to four children.
More than 80 percent of Even Start families in the last three program years had annual incomes under $15,000; 41 percent of 1996-97 enrollees earned and/or received annual incomes under $6,000.
However, only 26 percent of participating parents were employed at the time of intake in 1996-97. Of these, slightly more than half had full-time jobs. This suggests that family members not participating in Even Start were the primary wage earners in many families. For 28 percent of new families, their annual income of $6,000 or less came primarily from government assistance.
A slight decline in receipt of government assistance was consistent across all parent age groups. If the family income level remains consistently low but the receipt of public assistance declines, one explanation could be the impact of welfare reform. Because the changes in data are fairly slight and welfare reform is still relatively new in most states, these trends need to be monitored closely in the third national evaluation.
Repeating the patterns from the previous three years, 45 percent of 1996-97 new enrollees had not gone beyond 7th to 9th grade; 13 percent of new enrollees had not progressed beyond the primary school grades.
For roughly 60 percent of Even Start parents, Even Start represented their first experience with adult education programs. Similarly, only a small percentage (7 percent or less) had participated in employment or vocational training, either before or at the time of enrolling in Even Start. Fifty-eight percent of children enrolling in 1996-97 had received no early childhood education services prior to their association with Even Start. This reflects, at least in part, the increasing percentage of infants and toddlers among new families.
The racial/ethnic composition of Even Start participants has been changing since 1992-93.
The Hispanic and Asian parents' educational experiences averaged between the 8th and 9th grades, with most of their schooling completed outside the United States. Since 1992-93, projects have enrolled higher percentages of parents with no high school diploma or GED as well as parents with limited English proficiency. Although the majority of Even Start parents lack a few years of high school education, many older parents who are Hispanic or Asian present much greater needs for services.
Multiple disadvantages characterize most Even Start families. The neediest families participating in 1996-97 were identified based on the following seven indicators:
Families relying on government assistance as the primary source of income and/or receiving government assistance at the time of enrollment (59 percent of families);
Projects are required to recruit and serve families most in need of Even Start services in their communities. While the general level of need is likely to vary in different communities, the average Even Start family had three of the seven need indicators.
When asked for one primary reason for enrolling in Even Start, the most frequently cited explanation by parents was to improve their own educational status, followed by the desire to learn English and improving their child's chances for academic success. The primary reasons for enrollment have remained essentially the same since 1994-95.
Even Start projects are indeed recruiting and serving very needy families with multiple disadvantages. What resources are available to provide services to these families?
The patterns of funding at the project level appear to be shifting toward increasing contributions of non-Even Start funds supporting Even Start services. This is consistent with the legislation, which sets a minimum level of non-Even Start contributions by stipulating that the portion of the total budget supported by Even Start funds must decrease by 10 percent each year. This means that the non-Even Start ("local") matching funds must constitute at least 40 percent of annual operating budget by year four. Further, projects that receive a subsequent grant or grants must obtain at least 50 percent of their budget from non-Even Start sources in all years of these grants. In 1996-97, the eighth year of the Even Start program, roughly 30 percent of projects had received federal Even Start funds for five or more years. Aside from funding patterns, many aspects of program operations (e.g., staffing and interagency collaboration) have remained largely unchanged over the past several years.
With all federal, state, and local resources combined, the average annual budget for Even Start projects in 1996-97 was $250,267, nearly $5,000 per project more than in the previous program year. While the federal Even Start funds per project have decreased in the last two years, the average local contributions5 increased—from $87,161 in 1994-95 and $108,718 in 1995-96 to $115,960 in 1996-97.6
These results suggest combined effects of (1) a slight reduction in the average number of families served by each project (from sixty families per project in 1994-95 to fifty-five in 1995-96 and 1996-97), and (2) the increasing local contributions.
The Even Start staff composition, qualifications, experience, and training have remained essentially unchanged since 1993-94. The average project had a staff of ten Even Start-paid persons that included one project administrator, three to four teachers, one to two teachers' aides, one family specialist, one support service provider, one evaluator, and one administrative support person. Three-quarters of the Even Start-paid instructors had at least a bachelor's degree; and 25 percent had a master's degree. Ninety-six percent of aides had at least a high school diploma or GED, and 10 percent had a four-year degree. Over half of all Even Start instructors had six or more years of teaching experience. On average, in 1996-97, Even Start staff received five to eight days of inservice training, depending on job category.
Even Start is intended to serve as the "glue" that facilitates coordination of existing services and resources available for low-income families in the community. While a variety of agencies and organizations collaborate with Even Start projects, Even Start staff provide direct services in educational areas that are less frequently served by other sources in the community. The collaborative efforts described below are based on the 1996-97 data which closely represent the findings for the previous three years.
For approximately one-third of project sites, collaborating agencies were solely responsible for providing educational services in all levels of adult education; Even Start and collaborating agency staff shared responsibilities in about one-quarter of project sites. Public school departments (other than the specific departments sponsoring Even Start) on the one hand, and colleges and universities on the other, served as primary providers of adult education services for 34 percent and 21 percent of project sites, respectively.
Agencies serving as secondary providers of parenting education included: community groups (59 percent); public school departments (51 percent); government agencies (51 percent); and Head Start (50 percent).
Projects relied on public schools (33 percent), Head Start (25 percent), and other preschool and daycare programs (18 percent) as primary service providers.
Annually, projects are asked to prioritize areas in which they need technical assistance. Since 1993-94, funding or fiscal issues and increasing participant involvement and retention have been cited as areas needing the most assistance. More than half the projects reported some need for technical assistance in the following areas: participant retention; staff development; identifying effective programs; use of computers; approaches to adult and parenting education; sharing information with other projects; balancing program resources between educational and support services; and handling the social or health problems of participants.
Over half of the Even Start projects were located in rural areas; 35 percent were in urban areas. In many communities, adult education as well as early childhood education programs for children ages 3 to 5 years were available through other auspices. However, family literacy programs, educational services for infants and toddlers, and parenting education were available in fewer than half of the communities. In these community contexts, what are the services that Even Start provides to participating families?
For potential program participants, Even Start service begins with recruitment and screening. Projects used a wide variety of recruitment methods. Throughout Even Start's program history, word of mouth and referrals through collaborating agencies and other community agencies were the most commonly used recruitment strategies. Older projects with several years of Even Start experience used methods that target individual families (e.g., home visits, telephone contacts, and walking the neighborhood) more than first-year projects, while first-year projects used methods that reach many potential participants (e.g., mass mailing, mass media, posters and flyers, and making presentations in community agencies) more than projects with four or more years of experience.
In recruiting participants, many projects target families with specific characteristics and needs. Criteria most often used for targeted recruitment were (a) parents with no high school diploma, and (b) families with preschool-aged children. The proportion of project sites targeting teen parents increased from 44 percent in 1994-95 to 58 percent in 1996-97. On the other hand, relatively fewer projects (41 percent) used family incomes below specific levels as targeting criteria in 1996-97 compared to 1994-95 (56 percent). Judging from the pervasive poverty reported by program participants, projects may be expanding the needs assessment to include various criteria beyond low income to ensure that families most in need of services are recruited.
For example, for adult education, the average increases from 1993-94 to 1996-97 ranged from thirty-five to ninety-two hours per year, depending on educational levels. During the same four years, the increase in the average hours of educational services offered to children was 200 or more hours per year depending on the age of children.
Annual service hours rose from an average of 280 hours in 1993-94 to 406 hours in 1996-97, an increase of ten to eleven hours per month. The number of project sites offering services to infants and toddlers also increased from 71 percent in 1994-95 to 84 percent in 1996-97. Thus, projects are accommodating to the increased enrollment of infants and toddlers by expanding services for this age group—both in terms of availability as well as intensity. This also suggests that more children are benefiting from the Even Start services (e.g., early childhood education, parenting education services for their parents, early intervention services) from an earlier age than was common in previous years.
While the intensity of services has increased generally, every year we have found great variation across projects in the amount of services offered. Several project characteristics were related to the variation in service intensity.
Even Start projects are required to provide participants with services in three core areas (adult basic or adult literacy education, parenting education, and early childhood education) as well as home-based instruction, parent-child joint activities, and support services. However, there is no single Even Start model. A majority of project sites designed their own educational programs, incorporating features of existing approaches.
The proportion of projects using vocational materials for the beginning, intermediate, secondary, and English-as-a-second-language (ESL) classes increased between 5 and 13 percentage points between 1994-95 and 1996-97.
In parenting education, a majority of projects addressed topics related to child development; the development of language, thinking, motor, and social skills; and ways to ensure a child's safety and well-being. Almost all projects included parent-child literacy activities as part of their parenting education curriculum. On average, in 1996-97 a family was offered three hours per month of structured parent-child activities through home visits, eight hours in a center-based environment, and five hours of special events such as field trips. Topics addressed through these activities included: reading and storytelling; language development exercises; child's social development, independence, self-discipline, and self-help skills; health and nutrition; early academic skills; arts and crafts; sensory stimulation; and gross motor skills.
The integration of instructional activities across the three core service areas is one of the key elements of Even Start. Parenting and early childhood education, as well as parenting and adult education, were most commonly integrated through parent-child joint activities.
To facilitate families' participation in Even Start educational activities, projects provided a wide range of support services, either directly or through referrals. The support services most often received by Even Start parents and children were child care, transportation, family support (e.g., counseling and support groups), and meals. As we discuss below, families receiving more support services participated more in the educational services offered by Even Start.
Even Start projects implement many strategies to address the diverse educational needs of participants and provide support services to enable the families to benefit from the educational opportunities.7
A number of participant and project characteristics were associated with various measures of participation rates. Three factors that consistently emerged as factors contributing to families' participation were service intensity (i.e., hours of educational services offered), amount of support services that families received, and integration of instructional activities across core service areas.
Since 1994-95, we have found a strong relationship between the number of contact hours that projects scheduled per participant and the hours that parents and children actually participated. For example, in 1996-97, projects offering more than forty-four hours per month of adult education services reported an average of 144 hours per year of participation while the average participation hours were reduced by half among projects offering less than fourteen hours per month of adult education services. Similarly, for parenting education, parents in projects that offered less than fourteen hours per month of parenting education averaged only eighteen to twenty hours annually. In contrast, parents in projects offering more than twenty-three hours per month of parenting education averaged forty-two hours per year of participation.
The number of support services received by families was consistently associated with higher levels of participation in adult, parenting, and early childhood education programs. As a group, families who received five to nine types of support services were also more likely to participate in all three core educational components than families receiving fewer support services.
Integration of the three core educational services was associated with greater participation in adult and parenting education. Children enrolled in projects with highly integrated educational programs were significantly more likely than those in less integrated programs to have received ten to twelve months of early childhood education. These findings may suggest that integrated curricula meet the families' multiple educational needs more effectively than the traditionally compartmentalized approaches.
Again, referring to the 1996-97 data, the completion rate was 9 percent for families that continued participation from previous years and 5 percent for new enrollees. Among families who continued from previous years (who were more likely to complete the program than new enrollees), teen parents who enrolled with at least a 10th-grade education had the highest rate of completion (18-33 percent).
Parents who were native English speakers were three times more likely to complete the program than were parents with limited English proficiency (9 percent versus 3 percent). Even among families with limited English proficiency, however, parents who entered Even Start with at least a 10th-grade education and had continued participation since previous year(s) had completion rates of 10-12 percent.
Closely repeating the findings from the previous two years, 8. among the 1996-97 new enrollees, parents 30 years or older were more likely to be continuing (69-71 percent) than younger parents (59-62 percent). Families in which parents had either 6th-grade or less education or high school diploma or higher were more likely to be continuing (72 and 69 percent, respectively) than families with mid-level parental education (61 percent). New families that received many types of support services were more likely to be continuing (70 percent) than new families that had received little or no support services (59 percent).
Since 1994-95, the breakdown of families that were continuing participation at year-end versus those who left before the year-end has been roughly 60/40 percent. However, each year, we observed that some families that were reported to be continuing did not return to the program in the following year. For example, only 41 percent of new families who enrolled in 1995-96 actually continued into 1996-97. Data collected in the third national evaluation will allow much more in-depth analyses of Even Start participation patterns.
Sixteen percent of all families that were in the program in 1996-97 terminated due to a combination of factors such as poor attendance, lack of interest, and family problems.
To help families continue participation in spite of multiple difficulties, projects provided more support services to families with five or more needs than to families with few or no special needs. How well were these very needy families able to participate in Even Start educational services?
In terms of participation rates, the results are encouraging. Despite greater disadvantages, the very needy families participated in all core service areas at least as much as less needy families. Data on the year-end status, however, shed further light on the plight of the very needy families.
Somewhat higher percentages of families with four or more needs were terminated due to low motivation and attendance. This pattern of findings has been replicated each year since 1994-95.
Over the past eight years, the Even Start program has clearly begun to achieve some of its overarching goals. Even Start has been able to provide unified family literacy experiences for children and their families through an integrated program of early childhood education for children, parenting education, and adult literacy and adult basic education for adults. The first national evaluation indicated that children participating in Even Start made greater gains on some measures than children in a comparison group, and that adults were more likely to complete the requirements for attaining a GED. Participants progressed on some of the educational and developmental measures used, although the progress of Even Start participants was not consistently better than the progress of individuals in a comparison group.
Some of the findings from the first few years of the program and its evaluation led to changes in the legislation, including the requirements for local projects to recruit and serve families most-in-need in their respective communities; provide intensive instructional services in all three core areas; offer services on a year-round calendar; and demonstrate coordination, not duplication, of available services.
Many of the programmatic trends observed earlier have persisted. Although the amount of instruction offered and received in core instructional areas has increased over the average levels of the first four years, the changes from year to year appear to be leveling off. At the same time, there have been changes in who participates in Even Start. The Even Start program is now serving more teen parents, greater proportions of Hispanic/Latino families, and more families with greater evidence of disadvantage than during the early years of the program.
Along with changes in the composition of the participant population, the patterns of participation have changed as well. Families with teen parents do not remain enrolled in the program as long as older parents, on average; those with higher (or needier) average scores on the need index participate intensively for limited amounts of time. Families whose primary language is not English are more likely to remain active participants for longer periods of time. These patterns highlight the recurring tension between providing services to families most in need while sustaining their active involvement. Additionally, these differential participation rates raise some provocative questions about how to design program services to serve the majority of families who participate only for a limited time as well as how to evaluate service effectiveness.
Participants in Even Start did make progress on the outcome measures used in the second evaluation. The fact that we have been able to observe growth over time within individual children on both the PSI and the PLS-3 suggests that participation in a program like Even Start might help spur accelerated learning, as measured by these outcomes. One of the paradoxes we face, however, is that while we have indeed observed changes in outcome measures?particularly for children?the second evaluation design did not allow us to attribute such progress to participation in Even Start. When we consider the findings from the first national evaluation, which did have a random assignment component, in tandem with the current findings, the value of a strong research design becomes even more critical.
From the long-term study of the Even Start program, several key issues have surfaced concerning the directions for future evaluation. The issues include: identifying educational or other progress indicators that are appropriate for the diverse program population; ensuring consistency and quality of evaluation data collection and test administration; use of comparison groups; and matching data collection to actual participation patterns. Many of the lessons learned from the past eight years are being considered in the third national evaluation currently underway.
Footnotes:
1 Exhibit 2.1 in Chapter 2 summarizes the components of the second evaluation and data collected in each component.
2 Key evaluation findings are highlighted by bullets in the Executive Summary.
3 The gain scores discussed here for the CASAS and the TABE include scores for adults who were pretested either in 1994-95 or 1995-96 and posttested either in 1994-95, 1995-96, or 1996-97.
4 The percentages of all 1996-97 participating families identified as having each of the seven need indicators are shown in parentheses. Some need indicators are based on data collected at individual adult and child levels, summarized to the family level for families with multiple adults and children participating (i.e., at least one adult or one child reported the "need" condition). Further, the need index for "families receiving welfare" was based on answers to either one of two questions related to receipt of welfare. Due to these steps taken in creating the need index variables, some of the family percentages differ from percentages reported elsewhere in this report for adults, children, and one variable at a time.
5 "Local contributions" can include state, local, and non-Even Start federal funds. In submitting fiscal data for the national evaluation, some projects may have underreported the amount of local contributions due to omissions of some in-kind resources.
6 The analysis of change in the average cost per family does not account for inflation over time.
7 Approximately 15 percent of participating families and adults in 1996-97 had missing data regarding their participation in adult and/or parenting education. The participation rates cited above are based on participants with data, excluding those with missing data. The precision of data and method for computing participation rates have been refined every year during the second evaluation. Thus, the changes in participation rates over the years partly reflect these methodological changes. Computing participation rates by treating records with missing data as non-participants yielded 89, 91, and 78 percent for adult education, parenting education, and all core services, respectively, for 1996-97; compared to 85, 88, and 75 percent for 1995-96. In all years, well over 90 percent of children participated in Even Start early childhood education services.
8 Data on reasons for program exit were not collected in 1993-94. Data reported in the final report of the first evaluation are not directly comparable to the recent data due to changes in the data collection form and to a large percentage of families (30 percent) who exited the program in 1992-93 whose reasons for termination were not reported.
-###-
[Acknowledgments] | |
[Chapter 1: Introduction] |