A r c h i v e d  I n f o r m a t i o n

I. Introduction

A. Purposes of This Evaluation Guide

The goal of this Guide is to help local adult and vocational education programs improve their approach to evaluation. At the local program level, there is a need to obtain more comprehensive data in order to assess effectiveness and improve planning and management. The Guide presents an overall framework which can be used by program staff and evaluators to help establish the scope of their evaluations and to give direction to meeting local evaluation needs. As resources become more scarce, programs will be under pressure to document their effectiveness in order to maintain continued funding and attract new funding sources. The framework presented in this guide can be used to provide this much needed documentation.

What the Guide Is Intended to Do

In the broadest sense, the Guide will provide guidance and assist local program staff and evaluators to design and carry out their local evaluations. Thus, the Guide is intended to:

• provide direction for planning a relevant evaluation which meets the unique needs of each local program;

• provide a compendium of evaluation questions around which a local evaluation can be structured;

• provide sample instruments for collecting data on program participants and for documenting program process and procedures; and

• provide specific guidance for carrying out data collection activities. such as follow-up data from former students or trainees, and from employers.

What the Guide Is Not Intended to Accomplish

The Guide is not intended to be a "textbook" about evaluation in general, nor about evaluation of vocational and adult education programs in particular.

Specifically, it is not intended to be:

• a "cookbook" for evaluating local programs with specific recipes and menus for each aspect of an evaluation;

• considered a lock-step methodology with a compendium of precise instruments for examining every aspect of a specific program nor;

• a limiting or restrictive directive on what should or should not be evaluated.

Since the features and needs of projects vary, specific measures and procedures for assessing vocational and literacy skills are not goals of this Guide. Program staff should feel free to adapt the materials in this Guide to their own needs.

B. Audiences

The primary audience for this Guide includes:

• local program directors;
• program staff; and
• evaluators.

Others who may profit from using the Guide are state program administrators and coordinators who wish to know more about the impact of local programs across their States and can use the Guide to help plan evaluations of state-wide programs.

In summary, this document is called a Guide for deliberate reasons. Local program staff and evaluators can use the framework, the evaluation questions, sample instruments, follow-up procedures, and other aspects of data collection to address the issues and match the needs of their local program. The evaluation questions and the sample instruments presented in this document are not intended to be inclusive by any means. They provide a starting point to facilitate the process of formulating evaluation questions and focusing the evaluation.

***

The next chapter of this Guide presents a six-step evaluation process. This process is general in nature and discusses how an evaluation should be planned and conducted. Chapter III, on the other hand, sets forth a framework for specifically evaluating local programs. This framework provides suggestions and guidance for what aspects of the program should be included in the evaluation, what questions should be addressed, and what data should be collected. The general process described in Chapter II should be applied to the specific framework presented in Chapter III in order to plan and conduct a comprehensive evaluation. Chapters IV - VII of the Guide provide guidance in planning for each component of the evaluation framework, while Chapter VIII discusses how the data and the evaluation fundings may be presented.

Foreword

II. Evaluation Process and Plans