An effective program evaluation provides information on the following questions:

• To what extent are the purpose, goals, and objectives of the program achieved?
• What proportion of the target population is served by the program?
• What is the intensity of participation in program services by those served?
• What are the outcomes of the program -- whether or not they were intended?
• What is the evidence of the program's effectiveness in meeting its stated goals and objectives?
• How can the program be improved? And what "facilitates" success?

A less effective evaluation does not address these questions.

The two key questions an evaluation must address are:

• What claims or questions or hypotheses do you want to be able to answer or test about the program or its impact?
• What data do you need to answer those questions or test those hypotheses?

First, and this is the essence of the art of evaluation, it is hard to create effective programs. Many things can go wrong and not every program, no matter how well designed and supported, will be successful. Second, some programs serve so many participants that relatively few benefits accrue to each participant. In this case, it is hard to see the overall benefits of a program. If it is a very small program, it may be tailored to the particular environment in which it functions and not be suited for adoption by other sites. Last, bad news is news while good news is frequently unreported and uncommented upon. Sometimes, if the evaluator is only looking at/for goal attainment, evidence of good or positive impact may be missed.

n many cases evaluations do take a long time and cost more than you expect. There are many steps in designing and conducting an evaluation. You have to define the questions you are addressing, formulate the design you will use to gather the data, develop the instruments to be used, test the instruments to ensure you collect the information you want and need, allow the program to proceed (you can't evaluate the program's outcomes before it's been operating for a while), collect the data, analyze the data, consider the implications, determine your findings, and write the results of the evaluation. This is a series of sequential tasks and, in general, each step must be completed before you can move to the next step. It is a time consuming process.

Some types of evaluations don't have to cost a lot. There is a trade-off, however between cost and how much you can learn and how accurately you can measure outcomes In planning an evaluation it is important to remember the value of time: if staff will be devoting their time to the project their time has a price. If outside evaluators are brought in, their time costs. In most educational surveys, the largest cost element is staff time for data collection and analysis. All of the tasks mentioned above cannot be performed by low-paid staff. Evaluation techniques are an area of professional expertise like any other profession.