A r c h i v e d  I n f o r m a t i o n

The Board's Findings

The Board has conducted its efforts with a view of education research, development, and dissemination as bringing opportunity and promise to learning for America's youth. As the Board has surveyed the field of education research, it has tried to identify the major issues in the existing system—the longstanding problems and the increasing need for better returns from the investments in the system. It has grouped these in terms of resources, balance and linkage across the system, and processes.

Resources

Longstanding problems in education research start with insufficient resources. Limited funds have been spread thinly over a large number of topics and problems, rather than concentrated on fewer issues. OERI's national research institutes, created in 1994 to re-focus education R&D on important educational topics and problems, are a prime example. Notwithstanding some bright spots, the institutes lack sufficient internal staff to mount credible programs consistent with their mandates for comprehensive and high-quality work and to provide national leadership on critical issues. This means that the "critical mass" found in other research institutions to be necessary for an effective, high-quality program is missing. The Board's concern about critical mass extends to the national educational research and development centers, which in many instances have too few resources for the work and leadership expected of them. The regional laboratories, as well, have immense formal missions, but only modest resources to achieve them. Some of these institutions have addressed the critical mass problem by aggressive efforts to obtain resources from other sources, but they all still face a mismatch between ambitious missions and limited resources to meet those expectations.

If the quality, utility, and resources for education research are to improve, more effort, focus, and resources will be needed to strengthen the supporting infrastructure in three respects:

1. The "demand" for research should be strengthened. Building effective demand will require a substantial effort to educate teachers about the value and use of education research. That effort will succeed, however, only if educators are participants in the planning and execution of research.

2. More resources can be used to attract new, highly qualified scholars from many disciplines to educational research and its issues. Lack of resources and prestige now inhibit such recruiting.

3. The institutions that undertake the critical work will need to be selected on merit, nurtured with sustaining resources and the demand for quality, and rigorously evaluated for performance.

The National Educational Research Policy and Priorities Board is not alone in its conclusion that education research is shamefully underfinanced. In 1997, the Panel on Educational Technology of the President's Committee of Advisors on Science and Technology (PCAST) made several recommendations to improve the quality of education in all subject areas. The report did not focus only on technology as a topic of study, but as a means to strengthen content and pedagogy in education, enhance professional development, and increase student learning. The Panel called for a large-scale program of rigorous, systematic research on education in general and educational technology in particular. It recommended an investment equal to at least 0.5 percent of our nation's expenditures for elementary and secondary education—about $1.5 billion annually—a five-fold increase over what the Panel identified as the current level. That figure was contrasted with the pharmaceutical industry's investment in research of an amount equal to 23 percent of all U.S. expenditures for prescription and non-prescription medications.

Balance and Linkage

The involvement of teachers and other education professionals in knowledge-building and implementation activities is stimulating new thinking about the design and conduct of research in education. It is increasingly clear that teacher acceptance of and success in revised practice is strengthened by understanding and involvement. This realization is leading toward efforts to seek active participation of teachers, schools, and districts in the research and development planning, and conduct and evaluation process. Some refer to this as creating "learning organizations." In this role, the education professional community becomes vested in the objectives of the innovation and reform, provides helpful input in fitting concept with operational reality, and contributes a continuing basis for accountability and mid-course correction.

The span of activities authorized for OERI is very broad, from fundamental research through large-scale demonstration and effective communication of knowledge and information to the practitioner community. But that very breadth carries consequences when the research investments that are needed to improve student learning are not congruent with task. The portfolio of the Department is aggregated in two areas: applied research and small-scale development; and communication activities. The Department conducts essentially no basic research, and is not deeply involved in large-scale development or demonstration, especially about comprehensive or standards-based reform. The cumulative science base supporting the applied agenda of the OERI and departmental R&D activities lacks clear visibility, which adversely affects its force and credibility. The OERI Institutes are contributing some of the important applied research aimed at comprehensive or standards-based reform, testing and assessment. The Department's participation in large-scale development has been generally modest. The pattern of foundation funding often mirrors the federal focus, although a few foundations are supporting large-scale comprehensive reform experimentation. Generally, foundations appear to give more focus to curriculum and teaching topics than the federal programs. It is the Board's impression that the fit and relationship between foundation and federal funding are more happenstance than intentional. The absence of substantial large-scale development activity aimed at critical problems with rigorous research and evaluation is noticeable. This concern is particularly strong in light of the continuing difficulties of scaling up small, promising developments that require systemic change for widespread success. Providing knowledge of effectiveness at a large scale is an important insulation against faddism and insufficiently tested ideas.

Turning to dissemination, OERI's work most frequently has followed traditional approaches, no longer believed adequate, which leave to those in need of exemplary practice and sound knowledge the burden of finding it. This is particularly true for those undertaking large-scale comprehensive reforms. Even with the Internet and other forms of user-friendly electronic access, the passive systems do not fully meet the needs of those with ambitious innovation agendas, and the volume of information can overwhelm the practitioners involved. The more intensive efforts appear to require a combination of traditional dissemination, technical assistance, and short-term applied research or problem solving. A new set of intermediaries and adjustments in existing organizations are emerging to meet these needs, and dissemination must be reconceptualized in this broader context.

The Board has learned that fundamental research, largely in the cognitive and neural sciences, is conducted in other federal agencies, most notably the National Institute of Child Health and Human Development (NICHD), the Office of Naval Research, and to a lesser degree the National Science Foundation (NSF). There is also modest foundation support for basic educational research. The location and level of fundamental research is of concern in several different ways. It is not desirable that basic research be sponsored or conducted in one organizational framework, but it is important that such work be linked to the applied research and ultimate practitioner communities that will exploit and make use of its findings. Sound linkage requires that staff in applied research organizations are sensitive to implications of the findings and are qualified to design the applied and related basic research needed to push sound findings toward utilization. Such linkage requires continuing identification of application problems and unmet needs from practitioner and applied research communities. Efforts to link across organizational performers are occurring more frequently, such as recent planning work concerning a new initiative among OERI, NICHD, and NSF.

The research planning processes in Education are newer and less well articulated than those for Defense and Health. Department plans set forth broad goals and objectives, and do a particularly commendable job of relating research and development efforts to mission objectives. But they are unlike the Defense and Health counterparts in two undesirable ways. First, they are much clearer about the mission and application goals than about the research goals and priorities. Second, there is a strong sense in the defense and health cases that the science base is firmly rooted, and that there is a clear sense of direction and cumulative learning. Moreover, in those cases, the growing knowledge base is a powerful determinant of both future research and operational actions. There is no such comprehensive sense for education research and development. These differences are in part attributable to the huge difference in resources among the three agencies, which inevitably affects the style that has been adopted. But more than just staff and dollar resources are at issue; there is also the question of whether the education research and development program is an endless series of small applied research projects unrelated to an evolving critical set of knowledge bases or a cohesive agenda of cumulative knowledge-building. By this criterion, these other federal agencies are better developed.

Processes

The Board's review of the current education research system included specific attention to important processes by which agendas are set, support is mobilized, resources are allocated, and progress is achieved, assessed, and made known. The experiences and models in other federal R&D programs provide insights for assessment of OERI's work.

• Balance of researcher initiatives with national focus: The first area is agenda setting, in which two important objectives need to be harmonized. On the one hand, long experience suggests that R&D is most productive if researchers are given substantial latitude in initiating work on their own ideas or perceived problems, generally known as field-initiated research. On the other hand, there is a strong and continuing need to give the R&D agenda a sense of national focus and priorities aimed at the most important gaps in our knowledge and the most promising research approaches. OERI has adopted several devices to create the desired harmony, yet the national focus part of the balance remains weaker than it should be. Education lacks a process similar to that employed in NIH, for example, that could ensure a continuing dialog among OERI, the Institutes, Centers and Regional Laboratories plus other important education research institutions and sponsors about critical research problems and opportunities.

• Respect for research as well as for policy decisionmaking: A second issue centers on mobilization of support. Given the importance of federal funds in education R&D, garnering support means the involvement of the federal political process. Respect for the appropriate domain and responsibilities of the research community and political policymakers is both necessary and is accomplished in other federal R&D programs. Where it works well, all parties are engaged in setting broad objectives and parameters of the work, while leaving specific design and execution of the projects to the sponsoring agency and the research and practitioner communities.

• Meeting high standards of quality: The third issue is development of appropriate research methodologies and the establishment of standards of evidence to be applied to knowledge-building activities. More rigor is clearly desired and needed in education research. Traditional rigorous methodologies such as randomized experiments may not routinely be feasible, affordable, or appropriate in education research, though control group methodology has a continuing and important role in the clarification of uncertainty about critical research issues. Members of the Board believe that further efforts are needed to define appropriate research designs. The choice is not between randomized techniques and nothing at all. There is an appropriate level of rigor associated with the stage and purpose of the research being undertaken.

[Introduction]

[Goals & Recommendations]