A r c h i v e d I n f o r m a t i o n

Making Information Work For You - 1997

Getting Started

This chapter will help you plan around two questions that are central to your comprehensive strategy and evaluation:

What are our community's goals for its children and families?

What are we doing now and what could we do to achieve those goals?

Although there is no single right way to approach these questions, this chapter suggests four steps to get you started:

Seek broad community involvement

Develop or reconsider goal statements

Establish specific, measurable objectives for each goal

Spell out the assumptions that connect activities to goals and objectives

These steps will start the process of continuous evaluation, regardless of how far along you are in establishing a comprehensive approach to serving children and families.

Seek Broad Community Involvement to Build Support

Community participation will help you build consensus around the shared values and beliefs that drive your collaborative effort and will energize your activities. Your strategy will benefit from a concerted effort to involve all relevant stakeholders in the community, including those who have historically played a limited role in the decision-making process. For example, parents have unique knowledge of their children's behavior and needs and can reinforce an initiative's efforts in the home; parent volunteers also are important resources. Yet parents may not participate in school events because of language barriers, transportation or scheduling problems, or a perception that their opinions are not valued.

Activities to Encourage Parent Involvement

Recruit door-to-door in the community

Provide child care, refreshments, and language interpreters at meetings

Actively encourage parents to voice their views

Hire program staff from the same cultural backgrounds as the people you serve

Provide parent training on meeting procedures

Provide staff training on multicultural programming

Schedule meetings at convenient times and locations

Send home bulletins informing parents of meeting times and locations in relevant languages

Encourage active parents to recruit others

Develop Goal Statements that Specify Shared Views

A goal statement is your community's long-term dream for children and families put into words. When goal statements represent the views of all relevant groups, they express what you are striving for as a community and capture deeply held values and ideals. Even partnerships that have well-defined goals can benefit from reconsidering them periodically and revalidating them with community review.

Goal statements can be broad and might reflect concerns in such areas as education, health, employment, and safety. At the same time, these statements should be clear enough that the community readily understands what you intend to achieve.

The following sample goals pertain to a countywide strategy. The list is not prescriptive or exhaustive, but it gives an idea of the types of goals you could select.

Sample Goals for Greenville County

Child-level Goal Statements

Our children will demonstrate early academic success.

Our children will be physically healthy.

Family-level Goal Statements

Families will provide their children safe and nurturing homes.

Families will achieve and maintain economic self-sufficiency.

Community-level Goal Statements

Our community will provide a safe and healthy environment for children and families.

Our community will treat all residents fairly.

System-level Goal Statements

Services for children and families will be easy to use and understand, culturally sensitive, and responsive to families' needs.

Although goals vary from community to community, the following issues are likely to arise as you formulate and refine your goals:

Target audience. Although comprehensive strategies affecting many people often have broader community support than more targeted approaches, they also can be more expensive. Decide whether your goals apply to a particular group of children and families, an entire school, an entire community, or some combination. If they apply to an entire community, you need to determine the boundaries of that community.

Levels of goals and relationships among them. A goal may address changes for children, families, a school, the community as a whole, or service systems. Try to distinguish among goals that apply to different levels of impact because strategies for reaching goals often depend on the level of impact that is sought. At the same time, goals for children and families are likely to be powerfully interrelated, as are goals for families and communities and for communities and systems. Meeting a goal for one level may be a necessary step in achieving a goal for another. For example, the sample goals for Greenville County assume that, in order for children to achieve academic success, they must be physically healthy. And in order for children to be physically healthy, families must provide safe and nurturing homes.

Scope of the strategy. Be selective. Striving toward too many goals at once may diffuse your efforts and reduce your chances of success. Among all the things that matter in your community, focus on what matters most. The broad participation of all community stakeholders can help set priorities among goals and focus efforts where they will meet the greatest perceived need and make the best use of a community's strengths.

Writing goal statements. Goals are written as statements. Child goal statements can start with the phrase, "Children will..." Family goals can start with the phrase, "Families will..." Community-level goals can begin with the phrase, "Our community will..."

Ultimately, the statements will need to generate widespread community support, but that doesn't mean everyone must be involved in the first draft. You can select a small work group or task force to consider issues related to goals, objectives, and evaluation design. This group's first task would be to develop a set of goal statements based on the community's most pressing issues. The group would then take their work back to the partnership for discussion and approval and subsequently to the community.

Alignment. Independent efforts on behalf of children and families are proliferating in many communities, threatening to continue the fragmentation and duplication of efforts that often characterize the systems they seek to improve. Aligning the goals of your strategy with those of other efforts in your community, county, or state can foster a sense of shared mission and increase the chances that children and families will benefit.

A Checklist for Developing Goal Statements

Define the target audience for your strategies

Identify the levels of impact you seek and the relationships among goals and impacts

Does a goal address changes for children, families, a school, the entire community, or a service system?

Does success in meeting one goal depend on progress at another level of impact?How will that affect your strategy?

Select a limited, feasible scope for the strategy's goals

Translate goals into goal statements

Identify other community efforts that can and should be aligned with your strategy and identify ways to foster alignment

Establish Objectives for Meeting Goals

Although goal statements provide the overall framework for a strategy, they are not sufficiently focused for structuring the strategy's day-to-day activities. Therefore, it is crucial to develop specific, measurable objectives by which to assess progress in achieving each goal. By measuring objectives, you should be able to know if you have achieved your goals.

Objectives should be realistic reflections of your community's conditions and strengths and should reflect long-term results that are relevant to your goals. Objectives can be the ultimate benchmarks by which the partners and participants in a comprehensive strategy are held accountable for their efforts.

If you are just starting a comprehensive strategy, the first step in establishing objectives is understanding the conditions of your community. Even if your partnership is fully under way, it is still essential to reassess how far you have (or have not) come and whether resources are available to help achieve your goals and objectives. A thorough community assessment or "scan" can tell you:

Where you are in relation to where you want to be (an assessment of conditions and the root causes of those conditions)?

What resources your community can bring to bear in moving toward its goals?

Where are we now in relation to where we want to be? For each goal statement, you can engage the community to determine conditions and assumptions about the root causes of those conditions. This assessment involves asking the right questions and knowing where to look for answers.

Using as many sources as possible and representing a variety of perspectives and viewpoints will help ensure that you don't miss important information. It also will inform you about priorities in each sector of the community.

As you work through this process, construct a matrix for each goal statement that charts the goal, the community's current understanding of the issue, related questions, and potential sources of additional information. Below is a sample matrix for the goal, "Our children will demonstrate academic success."

The partnership in this example generated the information in the matrix by asking such questions as:

What are our priority concerns related to this goal?

How many people in our school/community are affected by this problem?

Who is most affected by this problem?

Why do these conditions exist?

What are the consequences of this problem?

In completing the matrix, the partners drew on information from school records (hospital, police, and clinic records are other possible sources), interviews with school staff, parents, and students (interviews or focus groups with service providers, political leaders, religious leaders, and business leaders are other possible sources), and a review of documents. Groups can also draw on data from human service agencies, census reports, and news reports, depending on their information needs. (Lodge & Hart 1994).

Based on the information gathered in this process, you can construct a community profile as a context for your work. A good profile summarizes information on the current status of each of your goal areas and highlights areas of greatest need in your community.

What resources can we use to move toward our goals? In addition to assessing conditions, your community scan can document your community's resources. These are the formal and informal assets that serve as building blocks for the activities that make up your strategy. Assets may include programs, organizations, or other funding sources already operating in the community. Remember to include in your inventory of strengths the "human factor"--strong community leaders, an active citizenry in one neighborhood, a powerful sense of extended family in another, high levels of parent support for education in the school community.

In conducting this part of the community assessment, you may want to expand the goal matrix presented below. As you identify formal and informal supports you can use to achieve your goals, think in terms of categories such as community, parents, children and youth, school, and service providers, so that you include potential contributions from all stakeholders. (See the Tool Kit for an expanded matrix that relates to the absenteeism component of the matrix presented below. Note that the existing activities of the partnership are included as resources on this matrix.)

Sample Goal Matrix

Goal	Current Understanding Where are we in relation to where we want to be?	Related Questions What else do we need to know?	Potential Sources of Information Where and how can we get the needed data?
Our children will demonstrate academic success.	Fewer than half of students read at grade level by third grade. More than one-third of students score in the bottom quartile on standardized test at each tested grade level.	Which students perform poorly? What improvement programs currently serve students? Who participates in them? What instructional improvement efforts are in place? Are parents aware of students, academic problems?	Student records; interviews with parents and teachers
	Absenteeism rates are high. Unexcused absences are common. In some schools, older students often miss school to care for younger siblings when parents are unable to do so.	What students have the highest absenteeism? Is their academic performance also poor? What situations cause parents to keep older siblings out of school to provide family child care? What are district policies regarding unexcused absences? Are they being implemented?	School records; interviews with students and parents; review of district policy manual; discussion with principal(s)
	Many parents are not involved in or seemingly supportive of educational activities at school or at home. Students' reports of time spent watching television are high. Homework is often late or inadequately prepared. Parent attendance at PTA meetings is low.	What are parents' opportunities for involvement in school activities? What outreach activities have been tried? What are the barriers to increased participation (e.g., need for child care)? How do parents perceive their responsibilities for providing educational support at home?	Surveys or focus groups among parents and students; review of PTA attendance sheets and notes; interviews with principals and teachers on attitudes toward parent involvement and steps for increasing it
	Many children arrive at school hungry. Teachers report that hungry children cannot concentrate on their classes.	Why are students arriving at school hungry? Do students who seem hungry truly perform at a lower level than their peers? Which schools in the community offer breakfast programs? What do the programs cost to implement? What community resources are available to serve hungry children/families?	Surveys, focus groups, or interviews with parents, interviews with teachers; teacher logs of student behavior and academic progress; interviews with cafeteria staff and food suppliers; assessment of community resources

Adapted from Lodge and Hart (1994), p. 9.

One Community's Child and Family Profile

After articulating shared community goals, a rural children's initiative created a profile to highlight the needs of children in their community. The partnership defined its target community as the entire county. In order to gain an understanding of the issues and problems facing children and their families in relation to the community's goals, the community assessment process involved community forums, focus groups, and surveys. The community forums were held in four regions of the county. More than 600 community members attended the forums and completed needs assessment questionnaires. Focus groups and individual interviews were conducted with parents, teachers, business owners, social service providers, recreation workers, health care providers, and church leaders. Existing data were assembled from grant proposals, agencies working with families, and statewide reports.

The collaborative partnership produced a Child and Family Profile to present the findings from the community needs assessment. An executive summary offered highlights for those who did not wish to read the entire report. The profile contained an introduction to the partnership and its mission, a description of the community, the methods used in the assessment, and the findings. The description of the community was the longest section of the report and included census data, statewide statistics, county agency data, school district data, local hospital data, and resource mapping. Statistics included the number of children under age six, child abuse rates, percent of children receiving special education, percent of children living in poverty, childhood injury rates, number of children in foster care, number of children on waiting lists for child care, number of dentists in the community, and the number of family support agencies. The last section summarized the issues facing children and families that community members regarded as most urgent or serious.

The main purpose of the needs assessment and the profile was to enable the partners to find out what the community wanted to accomplish for its children; it was a strategic planning tool. The process of creating the profile also brought community members together and generated ownership. The process helped stakeholders become invested in the idea of improving results--and in the partnership. Other benefits included networking among people interested in children's issues; two women who had never met before decided to pool their resources and start a drama club at the junior high school. In addition, the community profile became a useful document for fund raising, educating people about the community, and demonstrating the partnership's capabilities.

Adapted from Cherner 1995.

Choosing objectives. After constructing a matrix for each goal, you are ready to derive a set of realistic objectives. Keep the following points in mind:

Link the objective to the goal. Look at each of your goal statements and brainstorm what would constitute evidence that the partnership has made a difference in this goal. Of the list you brainstorm, select only those objectives that are clear and realistic measurements of the goal.

Base objectives on conditions and priorities. The objectives you choose should reflect the community assessment you just conducted. For example, in the sample matrix presented earlier, it would make sense to track third-grade literacy because this was perceived as a problem. However, it would not make sense to track high school dropout rates for a goal focused on early academic success.

Be specific. Although goal statements can be somewhat broad, objectives must be specific. For example, if your goal is "children will be physically healthy," discuss whether objectives should focus on such subjects as infant mortality rates, teen suicides, substance abuse, and dental problems.

Be clear about your desired outcomes. Your outcomes could take the form of increases or decreases. For example, if you are looking at mental health referrals, an increased rate could indicate either a growing mental health problem or an improved intervention strategy.

Choose objectives for which data are available. You may not have access to some data, and some data may require more resources to collect than you have. These issues may require some additional research in your community.

The exhibit on Greenville County offers some sample objectives that relate to the sample goals listed earlier. The selection of the target audience--all children and families in a county--assumes that a large proportion of children and families will be exposed to the strategy. This list is not intended to be prescriptive or exhaustive but suggests the types of objectives you might aim to achieve. Don't expect to address as many objectives as we have listed in this sample, however--it would be an overwhelming task. The number of objectives you set should reflect the needs and priorities of your community as well as what your community can feasibly achieve with available resources.

Sample Objectives for Greenville County

Goal	Example Objectives
Child Level
Our children will demonstrate early academic success	Increase the number of third-grade students who are literate Increase the number of students performing at grade level or higher Increase grade point average for all grades Increase scores on the Iowa Test of Basic Skills given at the end of second grade Increase scores on state assessments given at the end of fourth grade
Our children will be physically healthy	Decrease the incidence of illness from diseases preventable by vaccinations Decrease incidence of infant mortality and child death rates Decrease of suicide among children and youth Decrease blood lead levels Decrease drug, alcohol, and tobacco use by children and youth Decrease incidence of dental cavities
Family Level
Families will provide children with caring and nurturing homes	Decrease number of substantiated cases of child abuse/family neglect Decrease incidence of domestic violence Decrease the number of families with insufficient food for their needs
Community Level
Our community will provide a safe and healthy environment for children and families	Decrease incidence of crime Decrease incidence of domestic violence Decrease the number of families with insufficient food for their needs
System Level
Services for children will be easy to use and understand, culturally sensitive, and responsive to families' needs	Increase the number of hours and locations of services Decrease the amount of paperwork to be filled out by clients Increase ethnic and language diversity among service providers

Spell Out the Assumptions that Connect Goals to Results

Once you have established goals and objectives, you can turn your attention to the question of how your comprehensive strategy will achieve its desired results. Your community scan will have identified the building blocks of your strategy--the resources you have to work with and the activities you already have in place that address problem areas. Your planning activities also will have identified barriers to achieving results that must be addressed in your chain of assumptions if they are to achieve your desired results.

Your next step is to develop a theory or a logical series of steps that explain the intended impact of your activities on your goals and objectives. In other words, you need to think about the assumptions you are making about what will and will not work in your community.

Your assumptions about how things work are grounded in past knowledge and experience. They are your best guesses as to why things are the way they are and how circumstances can be improved. For example, the sample goal matrix suggests several reasons why many young students do not perform at high levels. These include:

A high rate of absenteeism. The assumption is that the more students attend school the more they will complete assignments and learn the curriculum. Although there may be many causes of absenteeism, one cause is older children staying home from school to care for younger siblings.

Low parental involvement. The assumption is that the more parents reinforce what students learn in school the more children will demonstrate academic achievement.

Hungry children. The assumption is that hungry children cannot concentrate in class. The more children can concentrate on their school work the better they will perform.

The assumptions underlying problems should drive the activities that you choose to implement. In assessing existing resources, you identified the activities that already are in place. Ask yourself if these activities are consistent with your assumptions about how to achieve your project's goals. For example, if one of your activities for improving early academic achievement is a tutorial program for at-risk students, then this activity would not be consistent with any of the three assumptions described above. However, if one of your activities is outreach to parents to eliminate unexcused absences and promote parent involvement in their children's education, then this activity would be consistent with the assumption that a high rate of absenteeism and low parent involvement in education are contributing to poor academic performance.

As you examine existing activities, ask yourself how they can be modified, expanded, or even replaced to become more consistent with your assumptions about why a particular problem exists. (See the Tool Kit for sample criteria to use in choosing activities that can help you refine your understanding of the contributions, constraints, consequences, and costs of various activities.)

Once you have examined alternative activities and services and modified existing ones, update your goal matrix again to include your revised activities. (See the Tool Kit for a revised matrix based on the absenteeism component of the matrix).

Think of Assumptions as a Chain of Dominoes

"Assumption Dominoes." One way to connect your activities to your goals is through a technique called "assumption dominoes" (Hebbeler et al., 1996). The idea is to identify everything that needs to happen if the activity is to exert an impact on the goal, and the sequence in which activities need to happen. These assumptions are like a chain of dominoes in which each domino is linked to the one before and after it. For the activity to be effective, the sequence of steps in the assumption chain has to be right, and each step has to be carried out. Just like a chain of dominoes that fall over one on the other on the other, if you pull one out, you've broken the chain and thus the chain reaction.

For example, suppose the partnership that identified absenteeism as a contributor to poor academic achievement then decided to implement a program to address the problems caused by absenteeism. Using the method of assumption dominoes, partners would take the following concrete steps:

Step 1.	Write down a child goal that the partnership intends to address. Write it out on the far right side of a piece of paper, blackboard, or flip chart. Child goal: Our children will succeed in school.
Step 2.	Consider what barriers stand in your way of achieving that goal. Ask, "What has to change for us to achieve our goal?" Write down these changes to the immediate left of the goal. One of them would be: Absenteeism, due to older children missing school to care for younger siblings, is reduced.
Step 3.	Ask, "What conditions must be created for these changes to occur?" Write down those conditions to the left of the changes. Some of the answers to this question might involve programs or activities that are already under way. Others may be entirely new. Parents are more aware of the difficulties caused by student absenteeism.
Step 4.	Identify the practical steps that must be planned and taken to create the needed conditions. School administrators send home information on absenteeism and school policies.

Continue this process until all the links or "dominoes" are identified. Stop when you can answer the question, "What do we each do tomorrow to get this chain of events started?"

Consider using this process for all other activities the partnership chooses to implement, whether the activity is day care for young children (in order to reduce absenteeism of older children) or a free breakfast program (so children will not start the school day hungry).

In addition to this visual model, partners could generate a written account of the chain of events that will lead to the attainment of a particular goal. This written account could specify who is responsible for taking which steps along the way. At the end of this chapter you will find an example of a written account for a program that seeks to reduce absenteeism by improving parental understanding, increasing day care, and improving supports.

What Does All This Have to Do with Evaluation?

By setting goals, establishing objectives, and spelling out the assumptions connecting activities to goals, you have set into motion the process of continuous evaluation. Essentially, evaluation is the process by which you test the accuracy of each of the assumptions you have identified along your chains:

Was the step carried out?

Did the next "domino" then fall into place? If not, why not?

If your chains of assumptions are based on sound logic, and if each event along the way actually occurs, you should see improved results. If you do not, either one or more of your assumptions is faulty or incomplete or an anticipated event did not occur.

The rest of this guide book will show you how to ask questions and collect data to test the accuracy of each of the assumptions in your assumption chain. If something breaks down in the chain, you'll be able to pinpoint the weak link. And more importantly, you will be able to use your data to modify your strategy and increase the likelihood of improving the lives of children and families.

A Checklist for Connecting Goals to Results

Identify the assumptions you are making about what will (and won't) work in your community

Make sure that your proposed activities are consistent with your assumptions

Find ways to modify, expand, or replace existing activities to make them more consistent with assumptions

Update your goal matrix to include the revised activities

Chain of Assumptions

Goal: Our children will succeed in school.

Objective: Reduce absenteeism as a barrier to improving school performance.

Activity: Involve all partners (parents, students, teachers, other school staff, agency partners) in (1) reducing absenteeism that occurs when older children miss school to care for younger siblings because their parents cannot and (2) reducing the resulting poor school performance.

Parents often need child care for young children when the children are sick and parents must work and when parents have appointments and their children cannot accompany them. Parents have few options for meeting their child care needs when are unable to care for children themselves.

Parent volunteers from the PTA will visit families in their neighborhoods and discuss ways to increase informal supports to families with unmet child care needs, such as babysitting pools and shared child care on designated days.

The partnership will designate a member to contact child care providers. He/she will work with providers to identify possibilities for expanding sick-child and drop-in care options in the community and to develop a process for realizing the most feasible possibilities.

Parents are unaware of the academic challenges faced by their children when they miss school; increased awareness would motivate parents to seek available options for child care or to develop new options instead of relying on older children to care for younger siblings.

School administrators will send home information in multiple languages on the importance of children attending school and the district policies regarding unexcused absences. Parents will be invited to discuss their situations with school staff.

School counselors will personally contact (or will designate an appropriate person to contact) parents of children with high absenteeism due to child care demands in order to better understand their situations and to help parents identify other options for child care.

School administrators will institute a "homework hotline" through which students who miss school can call for recorded messages by teachers about each day's assignments.

Teachers will regularly update messages on the "homework hotline" to update absent students on assignments. Teachers will identify ways they can help students keep up with assignments (e.g., making assignments for a full week at a time, offering one-to-one time with students on designated days after school to discuss topics missed when students were absent) and will implement feasible strategies.

Student council members will hold focus groups or other discussions with students to identify strategies to help students keep up with missed work (e.g., assigning "buddies" to take notes, establishing homework clubs after school) and to discuss ways to implement feasible strategies.

Partners from public service agencies (e.g., departments of social services, health, employment) will discuss ways to increase parents' access to services (e.g., by out-stationing staff to neighborhood sites two days per month) so that service-related appointments are less often a reason for parents to experience unmet child care needs.

Parents will have increased options for child care and will less frequently require older children to miss school to care for younger siblings. Absenteeism will decrease. Students who miss school will have more help in keeping up with their assignments and will take advantage of those supports. School performance will improve for these students.

-###-

[A Framework For Continuous Evaluation]

[Documenting Progress and Results]