A r c h i v e d  I n f o r m a t i o n

Appendix C: Evaluation Criteria for Peer Review

For the fiscal year 1997 FIS competition, the broad evaluation criteria, the points assigned to each, and the associated specific factors are as follows:

1. National Significance (30 Points) Specific factors: (a) the importance of the problem or issue to be addressed; (b) the potential contribution of the project to increased knowledge or understanding of educational problems, issues, or effective strategies; and (c) the potential contribution of the project to the development and advancement of theory and knowledge in the field of study. 2. Quality of Project Design (30 points) Specific factors: (a) whether the goals, objectives, and outcomes to be achieved by the project are clearly specified and measurable; and (b) whether a specific research design has been proposed, and the quality and appropriateness of that design including the scientific rigor of the studies involved. 3. Quality and Potential contribution of Personnel (20 points) Specific factors: (a) the qualifications, including training and experience, of the project director or principal investigator; and (b) the qualifications, including training and experience, of key project personnel. 4. Adequacy of Resources (10 points) Specific factors: (a) whether the budget is adequate to support the project; and (b) whether the costs are reasonable in relation to the objectives, design, and potential significance of the project. 5. Quality of Management Plan (10 points) Specific factors: (a) the adequacy of the management plan to achieve the objectives of the project, including the specification of staff responsibility, timeliness, and benchmarks or accomplishing the project tasks; (b) whether time commitments of the project director or principal investigator and other key personnel are appropriate and adequate to meet project objectives; and (c) how the applicant will ensure that persons who are otherwise eligible to participate in the project are selected without regard to race, color, national origin, gender, age, or disability.

For the fiscal year 1996 FIS competition, the point structure was different, with 15 points each being assigned to quality of personnel and adequacy of resources.¹ For center competitions in both 1996 and 1997, the broad criteria were the same as for FIS, but the specific factors were somewhat different:

1. National Significance (30 Points) Specific factors: (a) the importance of the problem or issue to be addressed; (b) the potential contribution of the project to increased knowledge or understanding of educational problems, issues, or effective practice; and (c) the potential contribution of the project to the development and advancement of theory and knowledge in the field of study; and (d) the nature of the products (such as information, materials, processes, or techniques) likely to result from the project and the potential for their effective use in a variety of other settings. 2. Quality of Project Design (30 points) Specific factors: (a) whether there is a conceptual framework underlying the proposed activities and the quality of that framework; (b) whether the proposed activities constitute a coherent, sustained program of research and development in the field, including a substantial addition to an ongoing line of inquiry; (c) the extent to which the research design includes a thorough, high-quality review of the relevant literature, a high-quality plan for research activities, and the use of appropriate theoretical and methodological tools, including those of a variety of disciplines, whether appropriate; and (d) the quality of the plan for evaluating the function and impact of the project including the objectivity of the evaluation and extent of which the methods of evaluation are appropriate to the goals, objectives and outcomes of the project. 3. Quality and Potential contribution of Personnel (20 points) Specific factors: (a) the qualifications, including training and experience, of the project director or principal investigator; and (b) the qualifications, including training and experience, of key project personnel; and (c) whether the applicant has assembled a group of hi quality researchers sufficient to achieve the mission of the center. 4. Adequacy of Resources (10 points) Specific factors: (a) the adequacy of support from the lead applicant organization; (b) the relevance and commitment of each partner in the project to its implementation and success (c) whether the costs are reasonable in relation to the objectives, design, and potential significance of the project; (d) whether the proposed organizational structure and arrangements will facilitate achievement the mission of the center; and (e) whether the directors and support staff will devote a majority of their time to activities of the center. 5. Quality of Management Plan (10 points) Specific factors: (a) the adequacy of the management plan to achieve the objectives of the project, including the specification of staff responsibility, timeliness, and benchmarks for accomplishing the project tasks; (b) the adequacy of plans for ensuring high-quality products and services; (c) how the applicant will ensure that a diversity of perspectives are brought to bear in the operation of the project, including those of parents and teachers, where appropriate; (d) whether there is substantial staff commitment to the work of the center; (e) the contributions of primary researchers (other than researchers at the proposed center), the appropriateness of such researchers' experience and expertise in the occontextf the proposed center activities and the adequacy of such primary researchers' time and commitment of the achievement of the mission of the center; and (f) the manner in which the results of education research will be disseminated for further use, including how the center will work with the Office of Reform Assistance and Dissemination.

[ Technical Review Form ]