![[Table of Contents]](/icons/up.gif){kind=icon}
Before trying a new medicine, most Americans want assurance that it has met certain safety and efficacy standards and earned the approval of the U.S. Food and Drug Administration. But what assurances are there about the reliability of education programs intended to improve the learning and achievement of the nation's children? How can consumers judge their quality, and who will vouch for their effectiveness?
If the program in question is an activity funded by the U.S. Department of Education's Office of Educational Research and Improvement (OERI), educators and taxpayers can be assured that it has been subjected to a vigorous peer review process and that it meets the highest standards of professional excellence.
OERI is the branch of the U.S. Department of Education primarily responsible for funding research and demonstration projects, collecting education statistics, and sharing information about education programs that work.
OERI has always used a system of peer review, the use of recognized experts to evaluate proposed and funded research and program activities. Now, however, OERI has for the first time defined and published the professional qualifications individuals must possess in order to become peer reviewers and a consistent set of criteria reviewers must use to determine the success or potential for success of any program funded and disseminated by OERI.
The new standards govern how OERI conducts business. All OERI activities must now meet a specific set of published standards that reflect the highest levels of professional excellence. The standards not only offer OERI a yardstick by which to measure quality, but also ensure that everyone is measured by the same yardstick. The standards also reinforce OERI's confidence that the projects it funds will contribute to improving the education of all students.
The standards can help many OERI constituents. For researchers, they offer guidance about OERI's goals and criteria for awarding grants and contracts. For teachers and administrators, they offer reassurance that the programs OERI funds and the information it disseminates have met certain high requirements. For parents, other taxpayers, policymakers, and members of Congress, they offer a means of holding OERI accountable for its use of federal dollars.
The new standards were developed by the Assistant Secretary, his staff, and by members of the National Educational Research Policy and Priorities Board, which collaborates with the Assistant Secretary in the development of research priorities. The 15-member Board includes educational researchers, practitioners, parents, and other individuals who have expertise and broad perspectives in the education field.
This handbook provides a general overview of the standards and highlights some of their key characteristics. Precise legal explanations of the standards are available from the Federal Register, as indicated in the appendix to this document.
The new standards require OERI to use peer review for three major program activities: to review applications for funding; to evaluate all existing projects to determine how well they are performing; and to select promising and exemplary programs to share with educators across the country.
To help ensure that OERI funds only high quality projects, all applications for OERI grants and cooperative agreements and any contract proposal for more than $100,000 must undergo a peer review. Generally, applications and proposals will be reviewed by three to five expert reviewers.
In a major move to continuously improve the quality of funded projects, OERI now requires that these projects be evaluated at least twice after they are funded: at least once while the project is under way and again at its conclusion. The interim reviews will allow OERI to fine tune project activities and, if necessary, to correct any problems. In the past, projects were evaluated but not according to a published set of standards. All OERI research, development and dissemination activities, ranging from the smallest purchase orders and commissioned papers to the largest research projects and national research centers, will now be subjected to this peer review.
Another of OERI's functions is to examine education programs from across the country to determine whether they should be designated as "exemplary" or "promising" and disseminated to the field. The new standards require that each program submitted for consideration be assessed by an expert panel of peer reviewers and judged by a common set of standards. The panels will designate a program as "exemplary" if empirical data demonstrate that it is effective in improving student achievement, or "promising" if there is sufficient evidence that the program shows promise for improving student achievement. Programs may be submitted to OERI for review at any time by individuals and organizations, or they may be proposed by the Secretary of Education.
It depends on what program is being reviewed. Generally, all peer reviewers must be leaders in the field and must have proven expertise in the subject matter they are reviewing. They also must meet stringent conflict-of-interest regulations. The Secretary of Education selects each peer reviewer and may solicit nominations for peer reviewers from professional associations, nationally recognized experts, and other sources. Some of the more specific qualifications include:
The answer depends on which of the three activities is being reviewed. While the criteria are different for each of the three, all were designed with one goal in mind: to ensure that everything OERI does is oriented towards its customers, credible to researchers, and useful to practitioners. Here is a summary of the three sets of criteria.
Grants and cooperative agreements are selected for funding according to a set of evaluation criteria specifically designed to identify the best. Peer reviewers consider one or more of the following:
Answering these questions will help reviewers determine whether to recommend to OERI that the project be funded. For each competition, criteria are further defined to target the specific purposes and expected outcomes for funded activities.
Contract proposals will be judged by specific criteria, which may include but are not limited to:
To help OERI determine how well each of its funded projects is performing, peer reviewers will examine the projects to see if each is meeting a standard set of four criteria. The same criteria will be used for both interim and final reviews. The criteria will be especially useful during interim reviews because they will allow OERI to assess a project's progress and determine if technical assistance is necessary to ensure that the project achieves its objectives.
Peer reviewers will thoroughly review each project and look for evidence that the project:
OERI will ask reviewers to consider additional criteria for specific types of projects.
If they are to use them with confidence, educators must have assurance that the programs OERI deems "exemplary" or "promising" truly meet these high standards. OERI requires its peer review panels to use four criteria when examining programs that apply or are nominated for this honor. The standards help to assure that these programs are high quality, research-based programs that have provided evidence that they have improved teaching, learning, or both, or have demonstrated other worthy educational performance outcomes. To determine if a program should be recommended as "exemplary" or "promising," peer reviewers must evaluate:
OERI and the National Educational Research Policy and Priorities Board have invested much thought and energy in developing these new standards. They sought input from practitioners, policymakers, researchers, the public, and other federal agencies in an effort to improve the quality of the agency's programs and projects. The new standards will enhance OERI's ability to provide the broad field of education and the nation access to programs and activities that meet the highest level of professional excellence.
For further information, consult:
