A r c h i v e d  I n f o r m a t i o n

VI. Program Implementation

This chapter focuses on documenting how a vocational or adult education program is implemented and the services which are provided. This includes recruitment, intake and assessment activities; literacy, basic skills, and vocational instruction; job counseling and other support services; job development; and job placement. The data to be collected will focus on what has actually taken place, rather than what was originally planned. With proper documentation, the project staff can assess the changes needed for future program improvement.

The project director and the evaluator should work together to decide how the specific activities and services of the program are to be described or documented, and the level of detail desired. Once this has been determined, the project director and evaluator should develop a set of evaluation questions which focus the data collection requirements. Sample questions, shown in Exhibit 4, cover a wide range of program activities. The project director and local evaluator should review these and select the ones which are relevant to their program. In addition, modifications and additions to these questions should be developed to fit the needs of the individual program.

The evaluator will be responsible for developing the plans and instruments for collecting the necessary data. Appendix A2 contains an example of one such instrument, a sample Program Profile whose purpose is to collect and compile the data needed to describe the program's processes and services. Appendix A3 contains a sample Student Instructional Status and Attendance Form which can be used to collect data on instructional services received by individual students. These sample forms can be adapted to fit the needs of the specific evaluation questions which have been developed by the project director and evaluator.

Data for the Program Profile can be collected from project records or directly from the project director and his/her staff. Two strategies are available. The evaluator can review project records and interview the project director and staff to complete the form, or the project director and staff members can respond to the form in writing. The latter data collection strategy is probably more cost effective since it minimizes the time of the evaluator.

Exhibit 4
Program Implementation: Examples Of Evaluation
Questions, Variables, and Data Sources

	Evaluation Questions	Variables	Possible Data Sources
1.	What are the objectives of the program? Are these objectives clearly stated and measurable?	Program Objectives	Project Application; Management Plan
2.	In what adult education/vocational training areas does instruction take place?	Instructional Area	Project Application and Records
3.	How long is each training cycle?	Number of Weeks or Months	Project Schedule
4.	What is the total number of hours of instruction provided to each student?	Number of Training Hours	Project Schedule
5.	What are the instructor/student ratios?	Number of Students; Number of Instructors	Project Records
6.	To what extent is enrollment meeting the program goals?	Program Objectives; Number of Students	Project Application; Project Records
7.	How is recruitment conducted? What criteria are used for accepting individuals? Are criteria appropriate?	Recruitment Procedures; Enrollment Criteria	Project Application; Observation
8.	How is the project staffed? Is staffing appropriate and sufficient for project operation?	Number of Staff by Position	Project Records
9.	What are the qualifications of the staff? Do staff have necessary qualifications to meet the needs of the students?	Background and Experience of Staff	Employment Applications; Staff Resumes and Interviews
10.	What kind of staff development and training are provided to staff members? Are development and training appropriate and sufficient?	Staff Development and Training Activities	Project Records; Staff Interviews; Observations
11.	What specific vocational competencies are addressed by the training program?	Vocational Competencies	Project Application; Project Records
12.	What specific basic skill competencies are addressed by the program? Are competencies appropriate and sufficient?	Basic Skills Competencies	Project Application; Project Records
13.	What criteria are specified for successfully completing the program? Are criteria appropriate?	Completion Criteria	Project Application; Project Records
14.	Does the training curriculum as implemented follow the project plan? Is curriculum appropriate?	Description of Training	Project Application; Project Records; Staff Interviews
15.	What training methods and materials are used? Are the methods appropriate?	Description of Training; Methods and Materials	Project Application; Staff Interviews; Observation
16.	What job counseling services are provided to students? Are these services appropriate and sufficient?	Description of Job Counseling Services; Number of Counseling Sessions; Number and Percent of Students Counseled; Number of Counseling Hours Provided	Project Application; Project Records; Staff Interviews; Observation
17.	What job development activities are implemented by the staff? Are these activities appropriate and sufficient?	Description of Job Development Activities; Number of Employers Contacted	Project Application; Project Records; Staff Interviews; Observation
18.	What job placement services are provided to students? Are these services appropriate and sufficient?	Description of Job Placement Services; Number and Percent of Students Served	Project Application; Project Records; Staff and Participant Interviews; Observation
19.	How are employers and the general community involved in the training program? Is this involvement appropriate and sufficient?	Number of Group and Individual Meeting Held; Number of Phone and Written Communications	Project Application; Project Records; Staff and Employer Interviews
20.	What facilities and equipment are used by the program? Are the facilities and equipment appropriate and sufficient?	Description of Facilities and Equipment	Project Application; Staff Interviews; Observation
21.	What is cost of training per student?	Total Project Funding; Number of Students	Project Budget; Project Records

-###-

V. Program Participants

VII. Program Outcomes