![[Table of Contents]](/icons/up.gif){kind=icon}
When trying to assess the impact that Even Start has had on families, it is not sufficient simply to measure pre/post "gains" on test scores, employment, income, or parenting skills. To understand the extent to which Even Start is helpful to families, we need to compare gains for families who participate in Even Start with gains made by families who are do not take part in Even Start services, but who can, of course, obtain child and adult services on their own. Research has shown consistently that children who do not participate in any special program make gains on developmental tests, that some adults make gains on tests of functional literacy without being in adult education,(8) and that low-income families have incomes which increase over time, even without federal intervention (e.g., St. Pierre & Layzer, 1998; Doolittle & Robling, 1994). The question is whether Even Start can provide a boost, change the trajectory, or alter the normal developmental pathway of families. Even Start seeks to do this in three ways:
Questions of great interest to Congress, researchers, policy makers, and program practitioners are whether increased participation in the three Even Start core service areas, an integrated approach to providing core and support services, and increased parenting skills, help families more than the fragmented set of services that families obtain on their own.
A simple model of the types of effects that Even Start projects hope to produce is presented in Exhibit 2. Produced with the assistance of the Expert Work Group for the national Even Start evaluation, the model summarizes the hypotheses that underlie Even Start. The following text describes the model.
Hypothesized Direct Effects. It is reasonable to assume that Even Start will have short-term positive effects on child development due to increased participation in early childhood education; on the literacy skills of adults due to increased participation in adult education; and on the parenting skills of adults due to increased participation in parenting education and parent/child together activities, as well as enhanced literacy skills. All of these direct effects should be apparent in a 1-year time frame, and should increase in size over time. Even Start also should have direct, but longer-term, effects on the economic self-sufficiency of adults due to increased participation in adult education and mediated by subsequent enhanced literacy skills. These effects should occur within 2 or more years.
Hypothesized Indirect Effects. Even Start should have longer-term positive effects on child development caused by continued early childhood education and mediated by earlier effects on parenting, enhanced literacy skills of adults, and enhanced economic outcomes for the family; longer-term positive effects on the development of nonparticipating children mediated by earlier positive effects on the parenting skills, literacy skills, and economic self-sufficiency of adults; and longer-term positive effects on parenting skills and literacy skills of nonparticipating adults mediated by earlier positive effects on parenting skills, literacy skills, and economic self-sufficiency of participating adults. The time frame for all of these effects is 2 or more years.
Even Start families participate in core educational services at higher rates than they would have had they not been in the program. This finding affirms the hypothesis that Even Start is able to secure core services for families at a higher rate than they would have obtained for themselves. About 90 percent of Even Start families participate in adult education compared with an estimated 30 to 40 percent without the program, about 90 percent participate in parenting education compared with an estimated 8 percent without the program, and about 95 percent participate in early childhood education(9) compared with an estimated 60 percent in the absence of Even Start (St. Pierre et al., 1995, p.129-131).
Perhaps the most important objective in the Even Start performance indicator plan is that the literacy of participating families will improve. In almost all of the areas that have been measured in the Even Start national evaluation, children and parents in Even Start make "gains" (see Exhibit 3). The difficult question is: How do we know if the gains that we see are due to participation in Even Start or are larger than expected in the absence of Even Start? We do this by judging the gains made by Even Start families against the gains made by some other group, for example, the gains of a randomly assigned control group, the gains of families in other social programs, or the gains of families in norms groups. Much of the data in this section addresses the four end outcome indicators in the Department's Even Start performance indicator plan.
Adult Literacy Achievement. One of the outcome indicators in the Even Start performance indicator plan is that increasing percentages of Even Start adults will achieve significant learning gains on measures of math and reading skills.
Each year in which the national evaluation assessed progress using measures of adult literacy, Even Start adults achieved statistically significant gains of 4 to 7 points (between .25 and .50 standard deviations)(10) on the CASAS reading or math tests (St. Pierre et al., 1995, p.188-189; Tao, Gamse & Tarr, 1998, p.152). Other adults achieved gains of 20 to 25 points, equal to about .25 standard deviations, on the TABE reading or math tests (Tao, Gamse & Tarr, 1998, p.157). However, in a small-scale experimental study in five projects (the In-Depth Study), families in a control group achieved similar gains on the CASAS, suggesting that the gains for Even Start families may not be due alone to participation in the program (St. Pierre et al., 1995, p.185-187).
Adult Educational Attainment. Another outcome performance indicator is that increasing percentages of adult secondary education Even Start participants will obtain their high school diploma or equivalent.
In the past decade, Even Start helped many adults attain a GED. Depending on the year, between 8 percent and 15 percent of all of the adults who entered Even Start without a GED or diploma attained one (St. Pierre et al., 1995, p.195-197; Tao, Gamse & Tarr, 1998, p.162). In the In-Depth Study of a subset of five Even Start projects, significantly more adults in Even Start than in the control group attained a GED (22 percent vs. 6 percent; St. Pierre et al., 1995, p.193-194).
Based on these data we are faced with uncertainty about Even Start's effects on adult literacy. Adults who participate in Even Start make gains on all of the measures that have been used, and gains in math appear to be larger than gains in reading.(11) Further, the gains made by Even Start adults are comparable or larger in size than those observed in other studies of adult education programs (e.g., CASAS, 1992; Darling & Hayes, 1989). However, where data are available on adults not in Even Start, they too make gains, probably because they too take part in adult education programs.
Even Start helps adults get a GED – a useful credential which can open doors to employment options and to continued education. Recent research by Murnane, Willett & Boudett (1995) shows that attainment of a GED is better in an economic sense than not having a GED, but that it is not as beneficial as having a high school diploma. And, Quint, Bos & Polit (1997) concluded that there is little evidence that a GED, or a high school diploma for that matter, can be equated with any particular level of literacy performance or gains. Finally, data from the first national Even Start evaluation showed that no adults who entered with less than a fifth grade education were able to attain a GED, and only 5 percent of adults who entered having completed grades 5-8 attained a GED (St. Pierre et al., 1995, p.196). This presents a challenge for many Even Start projects because about half of adult enrollees enter with less than a tenth grade education and 15 percent enter with less than a sixth grade education. For adults with little or no high school experience, GED attainment remains a long-term goal.
Children's Language Development and Reading Readiness. The performance indicator for children's outcomes is that increasing percentages of Even Start children will attain significant gains on measures of language development and reading readiness.
Both the first and second national evaluations showed that the universe of Even Start children learned school readiness skills such as colors, shapes, and sizes (as measured by a .90 standard deviation gain on the PreSchool Inventory) significantly faster than would be expected on the basis of normal development. After one year of participation, Even Start children in five projects scored significantly higher on the PSI than children in a randomly assigned control group. However, control group children caught up in the next year, when they entered preschool or kindergarten (St. Pierre et al, 1995, p.160-165).
A similar pattern was found on the Peabody Picture Vocabulary Test (a test of receptive language). When they entered Even Start, children scored quite low on the PPVT, at the 9th percentile nationally. During Even Start, children in low to moderate intensity programs gained a significant amount on the PPVT so that they scored at the 19th percentile nationally, while children who participated in higher intensity programs scored at the 26th percentile nationally. However, children in a control group gained a similar amount (St. Pierre et al, 1995, p.165-174).
Finally, an analysis of growth rates for children who remained in Even Start for more than one year shows that children who remained in Even Start for longer periods of time may grow at a faster-than-expected rate both on the PreSchool Inventory and on the PreSchool Language Scale (Tao, Gamse & Tarr, 1998, p.140-146).
It appears from all of these measures that children get a "boost" in cognitive development when they first are exposed to an organized school setting (either preschool or the public schools). Enrollment in Even Start ensures that such an exposure occurs at an earlier age, and so Even Start children get an earlier boost than control group children. The question to be addressed by future research is whether that early boost translates into other types of benefits for Even Start children.
Parenting Skills. The outcome performance indicator in parenting is that increasing percentages of parents will show significant improvement on measures of parenting skills, home environment, and expectations for their children.
In the second national evaluation, Even Start families gained a substantial amount (3.5 points, or about .50 standard deviations) on the Home Screening Questionnaire (HSQ), a measure of the quality of cognitive stimulation and emotional support provided to children by the family (Tao, Gamse & Tarr, 1998; p.147). This appears be larger than would be expected without Even Start, because data from the national evaluation of the Comprehensive Child Development Program show that low-income families in the control group did not make any gains on the HSQ (St. Pierre, Layzer, Goodson & Bernstein, 1997).
Several additional aspects of the home learning environment were assessed, and positive gains ranging from .23 to .63 standard deviations were observed in areas such as learning activities, story reading, books in the home, play materials, talking with child, and teaching child. However, in the In-Depth Study, the control group also made gains in each of these areas. The one area where Even Start families gained more than control group families was on different kinds of reading materials in the home (e.g., books, magazines, newspapers), something explicitly targeted by many Even Start projects (St. Pierre et al., 1995, p.202-210).
Economic Self-Sufficiency. Even Start families did not exhibit any measurable change on the adequacy of family resources or social support, and there was no measurable change in the self-efficacy of Even Start parents as assessed by self-report measures of sense of mastery and depression. The income of Even Start families rose over time, as did the percentage of families with an employed adult, but in neither case were these changes greater than those seen for control group families (St. Pierre et al., 1995, p.221-227; Tao, Gamse & Tarr, 1998, p.162-163).
Even Start seems able to improve the home learning environment for low-income families. Positive changes were observed on the Home Screening Questionnaire as well as several other related scales. While positive gains were made on income, and employment, control group families made similar gains. Even Start families made few changes on measures of adult self-efficacy, social support, and family resources.
In addition to information on the overall effectiveness of Even Start, the national Even Start evaluations provide data on the effectiveness of selected programmatic practices.
Intensity of Services Matters. A large body of research on the effectiveness of early childhood education programs shows that gains are enhanced by intensive exposure to a high-quality, center-based program (Barnett, 1995; Yoshikawa, 1995). Research on Even Start supports this finding in that adults and children with high levels of participation in Even Start's core services had larger learning gains than those with low levels of participation (St. Pierre et al., 1995, p.180, 189-191). While this finding was not replicated in the second national evaluation, there were concerns about the quality of the outcome data in that study (Tao, Gamse & Tarr, 1998, p.129).
Data from the first national evaluation also showed that the extent to which parents took part in parenting education is related to gains in children's vocabulary (as measured by the PPVT), over and above gains in vocabulary that result from children participating in early childhood education. Families who participated in a relatively low-intensity or moderate-intensity program had children who scored at the 17th to 19th percentile, while families who participated in a relatively high-intensity program had children who scored at the 26th percentile (St. Pierre et al., 1995, p.177-180). This finding goes to the heart of the Even Start model, showing that an intervention directed at parents may have an effect on their children. Unfortunately, this finding was not replicated using data collected from the second national evaluation, calling into question either the validity of the basic finding, the quality of the test data collected in the second evaluation (this possibility is raised by Tao, Gamse & Tarr, 1998, p.129), or the content of the parenting education programs implemented during the mid and late 1990s as opposed to the early 1990s.
Literacy-Based Parenting Education is Important. If Even Start's approach of training parents to be their children's first and best teachers is to work, then projects need to implement a high-quality, literacy-based parenting education component. In Even Start's early years, projects received strong messages from the federal level to focus on literacy-based parenting education. Once the responsibility for administering Even Start was transferred to the states, technical assistance became less focused and there is anecdotal evidence that parenting education has become more diffuse – a catch-all for a variety of parent-focused services including health education, nutrition education, and life skills. If this is so, it helps explain the disappearance in the second national evaluation of the relationship between amount of parenting education and child test gains. The need for a refocusing of the parenting education component of Even Start has prompted the Department to fund a Parenting Education Improvement Initiative, in conjunction with an observational study of 12 Even Start projects. The initiative will culminate in a "Guide to Improving Parenting Education." This guide will be based on a conceptual framework linking different aspects of parenting education and children's learning. This guide will be useful for projects in improving their parenting education services and for State Coordinators to use in providing technical assistance on how to implement literacy-based parenting education.
Service Location Matters. Children in projects that emphasize center-based programs had larger learning gains than children in projects that emphasize home-based services. This is probably because center-based projects can more easily provide larger amounts of instruction.
Project Size Does Not Seem to Matter. The number of families served and the grant amount of the project are unrelated to learning gains (Tao, Swartz, St. Pierre & Tarr, 1997, p.184). This means that children and adults do equally well in small and large projects.
Parent/Child Time Matters. Families in projects that have large amounts of time for parents and children together had better home environments (e.g., more materials in the home, parent/child learning activities, approaches to discipline) than families in projects that have smaller amounts of parent/child time together (Tao, Swartz, St. Pierre & Tarr, 1997, p.184-185).
Footnotes:
(9) These figures could reflect families participating under continuing eligibility.
(10) Converting gains to "standard deviation units" instead of leaving them as raw scores, puts those gains in a common metric so that it is possible to compare the progress of Even Start families on different measures. A rule of thumb for interpreting the magnitude of gains expressed in standard deviation units is that a gain of .25 standard deviations is considered "small," a gain of .50 standard deviations is "medium-sized," and a gain of .75 standard deviations is "large" for social science interventions.
(11) This may reflect the fact that math instruction is most likely to occur in formal educational settings which emphasize test-related material, while opportunities to practice basic literacy reading skills exist outside such formal learning environments and may not contribute as much to the skills measured by standardized tests.
-###-
[Future Research Directions]