Written by Dr. Susan Lehmann, susan.lehmann@ed.gov
FIPSE Education Research Analyst and Program Evaluator

These guidelines apply to applicants and grantees for the following FIPSE Programs:

• Comprehensive Program
• Special Focus Competitions
• EU-U.S. Atlantis Program
• North American Program for Mobility in Higher Education
• Underground Railroad Educational and Cultural Program
• U.S.-Brazil Program
• U.S.-Russia Program

Evaluation Information that Must Be in Your Grant Application:

1. The evaluation section of your application should briefly present two to five main project goals centering on educational outcomes for postsecondary students or professional development of those working in the field of postsecondary education ¹. Proposal reviewers will be looking for assessment that focuses on gains or changes in knowledge, skills, behaviors, and/or attitudes. The evaluation may also include assessment of other outcomes, particularly if the project aims to change organizational structures, create cost-efficiencies, or achieve other ends not specifically represented by learning. In these few paragraphs you should explain the data gathering procedures you plan to use to monitor and assess progress toward your educational aims. When describing the measurement instruments you plan to use (surveys, interviews, focus groups, assessments of e-portfolios or capstone projects, measures of class performance, scores on standardized tests, cost-benefit analyses, etc.), be sure to mention why they are appropriate to gauging the success of your endeavor ².

   Though this section of the applications is brief, it should make a convincing case to the proposal reviewers that at the conclusion of the grant you would be in a position to discuss the extent to which learning outcomes, professional development goals, and/or organizational outcomes have been achieved. The proposal reviewers will be looking for evidence that you have thought about how you will know what worked, what did not work as planned, and what adjustments could be made to enhance program outcomes in the future. Collection of this evidence is critical to achieving FIPSE’s goal of making data about educational improvements and innovation available to the postsecondary education community. An evaluation section that states that you will discuss program administration, describe your activities, and the present the results of a satisfaction survey will be considered a poor evaluation plan. Please consult "What a FIPSE Program Evaluation Is Not" for additional guidance.

2. Your budget must contain funds set aside to hire an outside project evaluator who will assist you throughout the life of your project. Your independent evaluator must be someone who does not stand to gain personally or professionally from the project results. For a discussion of the role of the outside evaluator, please refer to the "Role of the Independent Outside Evaluator." Unless otherwise specified in the Application Package for your particular grant program, a minimum of five percent of your total budget should be allocated to cover the cost of your evaluation activities and the salary of the outside evaluator.

   Unless otherwise specified in the Application Package, you do not need to name your outside evaluator in your application ³. You will need to locate an outside project evaluator and submit his/her name to FIPSE within three months of receiving your grant. At this time FIPSE cannot recommend certain evaluators. For general guidance as to where to look for a project evaluator, please refer to "Finding a FIPSE Project Evaluator."

Grantee Evaluation Requirements:

1. Resume and Contact Information for Your Outside Evaluator – due three months after the start of your grant ⁴.

2. Draft Evaluation Plan - due three months after the start of your project. For guidance on the type of information that should be contained in this evaluation plan, please see "FIPSE Evaluation Plan with Examples." Though this plan is in a spreadsheet format, you may submit a draft evaluation plan in the form of a spreadsheet, a document, or a graphic such as a logic model. Your plan should be short – three to five pages long.

   If you, like many FIPSE grantees, have student foreign language gain as a project objective, you may wish to consult "Language Testing Options" for ideas about how to measure language gain among novice or beginner language learners. This guidance was written primarily to address the concerns of project directors who are not foreign language specialists, but who wish to set improved student foreign language ability as a project goal.

3. Year 1 Evaluation Report - This report is submitted along with the Annual Performance Report and is due at the end of your first year. The Evaluation Report contains a revised evaluation plan, details of baseline data collection, and draft data collection instruments. This evaluation report will be written by your outside evaluator.

4. Year 2 Evaluation Report - This report is submitted along with the Annual Performance Report and is due at the end of your second year. The Evaluation Report contains your early evaluation findings and a discussion of the adjustments to project activities that you have made in order to increase the impact of your project. This evaluation report will be written by your outside evaluator. (If you have a two-year grant, you skip this report and submit a final evaluation report instead.)

5. Year 3 Evaluation Report – This report is submitted along with the Annual Performance Report and is due at the end of your third year. The Evaluation Report updates your evaluation findings and discusses any further adjustments to project activities that you have made in order to increase the impact of your project. This evaluation report will be written by your outside evaluator. (If you have a three-year grant, you skip this report and submit a final evaluation report instead.)

6. Final Evaluation Report - This report is submitted along with the Final Performance Report and is due no later than 90 days after the end of your grant. This Final Evaluation Report contains a summary of all your evaluation findings. It contains a discussion of who benefited/learned the most from your project. It discusses any unforeseen outcomes – desirable and not. It discusses recommended modifications that you think should be implemented. It should contain advice that you would give to colleagues who are considering adopting your model at their own institution or organization. This evaluation report will be written by your outside evaluator.

Evaluation Feedback to Project Directors and the Public

FIPSE believes that thoughtful and appropriate evaluation of projects is crucial if FIPSE is to be able to assist the postsecondary educational community by defining the cutting edge of educational innovation. Most members of the postsecondary educational community will never receive a FIPSE grant, but these people are an important constituency to the Department of Education. Our goal over the next few years is to start a continuous flow of information about the results of the projects that we fund. You can find some of our initial postings on the "FIPSE Grants in Action" page.

We have also increased our commitment to assisting grantees in designing program evaluations that go beyond a basic summary of activities and a satisfaction survey. Beginning with the 2009 Annual Evaluation Reports, FIPSE’s staff evaluator will review each Annual Evaluation Report and send feedback to project directors in a standardized format. We do believe that most project directors sincerely wish to know if their ideas really work, but many need some additional guidance with assessment. Our intention is to provide more service to grantees via this regular feedback which will flag aspects of their evaluation that need attention.

After reading the Final Evaluation Report submitted by each project director, Dr. Lehmann will make a final project evaluation assessment that will become the permanent record of the quality and basic content of the evaluation component of each project. This will provide FIPSE and, one day the field, with a database that could be searched by subject field, type of institution or organization, theme, project goals, type of evaluation data collected, and quality of assessment data. Because the overall evaluation assessment for each project will be public, along with the contact information for the outside evaluator for each project, it will be possible for potential applicants to use the new database to locate evaluators with a track record of high quality work assessing FIPSE grants. This meets a need that project directors have brought to our attention.

The findings from the evaluation database coupled with publically available project information located at www.fipseaed.org should give anyone working in postsecondary education the project information they need to consider before investing in time-consuming and potentially costly educational reforms. In pursuit of this goal FIPSE welcomes feedback on any of the evaluation materials referenced on this page. Please send comments, concerns, and suggestions to susan.lehmann@ed.gov.

¹ Applicants to the Underground Railroad Educational and Cultural Program may include the educational advancement of museum audiences and the professional development of museum personnel.
² Applicants to the Underground Railroad Educational and Cultural Program must specify why their measures would be deemed appropriate by museum standards.
³ Applicants to the Underground Railroad Educational and Cultural Program must submit the resume and contact information for the outside evaluator with their application package.
⁴ Applicants to the Underground Railroad Educational and Cultural Program must submit the resume and contact information for the outside evaluator with their application package.

Evaluation Resources | FIPSE Home