A r c h i v e d  I n f o r m a t i o n

VIII. Presentation of Findings

Following data collection, the next steps in the evaluation process involve data analysis and preparation of a report. These steps require the expertise of an experienced evaluator who is objective and independent of the project and grantee. This is important for the acceptability of the report's findings, conclusions, and recommendations.

The evaluator will be responsible for developing and carrying out a data analysis plan which is compatible with the evaluation's goals and audience. To a large extent, data will be descriptive in nature and may be presented in narrative and tabular format. However, comparisons of pre- and post-measures related to literacy and computational skills, vocational skills and knowledge, job wages, etc. may require more sophisticated techniques. These depend on the nature of the data.

The data will be analyzed to answer the evaluation questions specified in the evaluation plan. Thus, the analysis win allow the evaluator to:

• describe the program environment;

• describe the program participants;

• describe the program activities and services;

• describe the outcomes;

• examine and assess the extent to which the program plan was followed;

• examine and assess the extent to which the outcomes met the program goals and objectives; and

• examine how the program environment, participants, activities, and services affected the extent to which the outcomes were achieved, and how the program can be improved to achieve increased success.

• describe the accomplishments of the program, identifying those elements of the program that were the most effective;

• describe elements of the program that were ineffective and problematic as well as areas that need modifications in the future,

• describe the outcomes or the impact of program services on the participants; and

• document how the program environment, activities, and services contributed to the accomplishment or lack of accomplishment of the program goals and objectives.

The level of information and documentation suggested in this Guidebook will make the report quite useful for making decisions about improving program services, instructional strategies, etc., for the future. In other words, the evaluation report is a decision-making and planning tool for the project director and his/her staff. The report will support the broad management functions of decision-making, program improvement, accountability, and quality control.

It is important to keep the report's audience in mind. In most cases. this will be the project director and funding agency administrators. These individuals may not be experienced in the technical aspects of data analysis. Therefore, the report should present data in a straightforward manner, using tabular presentations to help the reader. Interpretations and conclusions drawn from the data and implications for future directions should be discussed in a clear narrative. An example of a report outline is provided in Exhibit 6 on the next page.

In addition to the written report, the evaluator should provide an oral briefing to the project director, project staff, and appropriate administrators. This should take place after those individuals have had a chance to read the report. The briefing should first include a short presentation on the most important findings, and on the conclusions and recommendations drawn by the evaluator. A question and answer period following that presentation will allow for discussion of those conclusions and recommendations that are most relevant to improving the program in order to yield greater benefits to participants, employers, and the community-at-large. The project director should then prepare and execute an action plan for implementing the appropriate recommendations. In sum, evaluation results should be used for program improvement. Learning from the past to improve future programming should be a primary management goal.

Exhibit 6
Organization of Evaluation Report

I. Executive Summary (3-4 pages)

• Purpose

• Procedures

• Major Findings

• Conclusions and Recommendations

II. Introduction

• Background of Program

• Objectives of the Evaluation

• Summary of Procedures

III. Description of Program Environment

IV. Program Components

• Descriptions of Program Activities and Services

• Deviations from Plan in Grant Application

V. Number and Characteristics of Participants

VI. Program Outcomes

• Program Completion

• Follow-Up

VIII. Conclusions and Recommendations for Program Improvement

-###-

VII. Program Outcomes

Appendix A. Sample Data Collection Forms