A r c h i v e d  I n f o r m a t i o n

Changing Federal Strategies for Supporting Educational Research, Development, and Statistics - September 1998

Appendix C

Dear Eve:

Thanks for sending me a copy of Maris Vinovskis’s paper, "Changing Federal Strategies for supporting Educational Research, Development, and Statistics." I asked several members of my staff to read the document and provide me with comments. In this letter, I summarize our discussions about the paper.

Overall the document makes for depressing reading. His portrayal of the history of federal funding of research is one of overblown expectations, underfunded projects, lack of focused direction, fragmented work, etc. Unfortunately, as one who has been involved as a researcher and developer on federally sponsored projects since the late 1950’s, I find the story he tells to be both incomplete and biased. It is incomplete in that he tells only the story about federal strategies in the U.S. Department of Education. NSF is mentioned but not in terms of its support of research and development, and no other agency’s work in educational research is mentioned. It is biased in that he tells the story strictly from the political policy perspective in relationship to the Department of Education. His facts about funding, shifts in political priorities, changing personnel, and so forth may be correct, but as a researcher he fails to picture the impact of such funding on the educational research enterprise in this country.

The picture could be more balanced by acknowledging four facts:

First, since the 1960’s, increased federal funding for educational research has provided the impetus for a proliferation of studies. The number of research journals, scholarly books, attendance at research conferences (e.g., attendance at AERA’s annual meeting was 300, and over 10,000 in 1997) has increased dramatically. We will not argue that all such research studies have been of high quality, but the questions investigated, methods used to investigate those questions, and what we know about schooling have expanded. Unfortunately, our gain in knowledge is not often apparent. For example, if one examines an issue of a first-rate scholarly research journal (e.g., American Educational Research Journal, Cognition and Instruction, Journal for Research in Mathematics Education, all started since 1960), one could question how the particular set of articles contributes to solving any educational problem. The problem is that an individual study contributes only a small piece of information, from a particular perspective, about some phenomenon. Similarly, the combined studies from a research center can contribute only several related studies. Resources are too limited to do more. Nevertheless, we would argue that we know a lot more about learning, current schooling practices, and so forth, than was known a quarter century ago. For example, our understanding of how children learn to add and subtract whole numbers is today quite different from what it was in the past. Federal funds from NIE, NSF, and OERI have supported work for over 20 years that has produced a consensus on a framework, instructional materials, staff development, comparative data with traditional approaches, and longitudinal data for both student development and teacher change. Similarly, syntheses of research on the way humans learn (see NRC’s Developments in the Science of Learning) have implications for the way scholars now.

Second, the political assumption that basic research findings can directly change educational practice needs to be challenged. Good research produces only reliable knowledge. Using that knowledge to produce new products or change practices involves problems of a different sort, engineering and design problems. Changing schooling practices is an engineering design problem, but design studies are hard to do and rarely funded. Also, much of what is now understood from basic research challenges the current design characteristics of contemporary schooling. For example, much of the research about the teaching and learning of mathematics clarifies problems that need to be addressed. Many middle school mathematics teachers have never been taught with understanding some of the mathematics they are expected to teach. Design studies should follow, but rarely do. Even when new and better instructional practices based on research are introduced, it is very difficult to implement them, because there is resistance to changing traditional modes of operation. Such was the case with team teaching in the elementary grades, or replacing high school algebra and geometry courses. The successful introduction of new materials or models presupposes a whole constellation of changes in teacher preparation and assessment strategies.

Third, the allocation of scare resources is out of balance. While copious funds are spent each year documenting the existence of problems (NAEP, TIMSS), other funds are spent on studies designed to demystify components of particular problems. We take exception to Professor Vinovskis’s contention that there is "little evidence that most of the new centers are engaged in the systematic, long-term development of educational materials and models," which he sees as a continued weakness of OERI in general. On the contrary, the development of educational materials and models has not been the focus of NIE or OERI efforts, although such work has been undertaken. In our center, we have been able to do this by coupling funds from NSF and the MacArthur and Spencer Foundations. No one source of funds is adequate to support the creation of a community of scholars committed to long-term research and development. Our position is that OERI research ought to be just that—research, with some development efforts that add value to work sponsored by other agencies. For example, for mathematics and science education let NSF pay for curriculum or technology development, and let OERI figure out classroom implications. Also, let other agencies worry about scale, and let OERI develop ideas worthy of scaling. Finally, at the present time there are political calls for more comparative evaluation studies. There is no question that such research is warranted, but only if new materials or models are worthy of such expensive and time-consuming investigations. NSF is currently funding us to conduct on a small scale one such study on materials developed with NSF funding. A significant complication for educational researchers, unlike those in the medical or agricultural fields, is that school administrators, teachers, and parents do not want their students to be subjects in experiments, on the one hand, but on the other, they do not want any of their students to be control subjects, when convinced that new material is worthy.

Fourth, those of us who have directed research centers over the years have learned to live with the vagaries of Washington politics and changing federal bureaucracies so that we can carry out systematic, long-term research. We are concerned about leadership and staffing of OERI, inasmuch as OERI, with a few exceptions, is not staffed with experienced scholars knowledgeable about research or familiar with past studies in an area of schooling.

We agree that NIE and OERI have tried to disseminate research information to the public and practitioners, but providing reliable knowledge is not sufficient. Vinovskis argues quite effectively that devoting large amounts of center funding to traditional forms of dissemination is a waste of money. It is important for all of us to focus on developing knowledge that can be spread before we commit so much of our funds to newsletters, interim reports, and so forth.

We support his contention that centers should conduct sustained, substantive programs of research, rather than fragmented, isolated, short-term projects. At present, we are attempting to involve in our agenda science education researchers, who previously have been individual entrepreneurs. We agree that having adequate funding for field-initiated studies is an important alternative, but to conduct important work in a sustained manner requires an established research environment where a community of scholars can interact over time. It takes substantial funding over a very long period to create such an environment.

Finally, as the reauthorization of OERI is being considered, we would be willing to assist. Schooling is a complex social process that does not lend itself easily to controlled, "scientific" investigative processes. Notwithstanding the ongoing problems resulting from meager funding and a lack of well-trained investigators, we think the federal investment in educational research has had a considerable beneficial impact. Continued and increased support of research is important for our nation’s future.

Sincerely,

Thomas A. Romberg
Director
National Research Center for Improving Student Learning
And Achievement in Mathematics and Science


-###-

[Appendix B]

[ Biographical Information About Maris A. Vinovskis]