![[Table of Contents]](/icons/up.gif){kind=icon}
Archived Information
Educational Technology Expert Panel: Exemplary and Promising Educational Technology Programs 2000Educational technology: A variety of electronic tools, media, and environments that can be used to enhance learning, foster creativity, stimulate communication, encourage collaboration, and engage in the continuous development and application of knowledge and skills.
Educational technology has now evolved to a point where its power to transform schools is evident. This report, focused on the potential of technology to affect preK-12 student learning outcomes, is expected to advance the dynamic field of educational technology by helping
By 2001, public schools had invested almost $5.5 billion dollars in technology (Market Data Retrieval's 2000-2001 Public School Technology Survey), and 98 percent of public schools had access to the Internet (Internet Access in U.S. Public Schools and Classrooms: 1994-2000, NCES). Currently, the critical questions about educational technology for practitioners, researchers, policymakers as well as parents are: How can digital technology best be put to use in the service of precollege education? How can this technology be used to accelerate conventional learning as well as transform what students are learning and understanding? How can technology contribute to the acquisition of information as well as to the ability to assess reliability of findings and to synthesize incomplete but relevant data across several disciplines? How can technology enable students to demonstrate high standards of performance as well as to become contributing citizens of our increasingly global society and economy?
The field of technology is in a state of constant change. Development has been uneven. Leaders in the field see pockets of excellence but also areas of genuine need (such as more universal individual classroom access to the Internet and home computers for more students), particularly on the part of underserved populations.
Many in the education community are aware of the need for more guidance and discourse on how to maximize any investment in technology. Many educators want to know how technology will make a difference to their students in the classroom. Trained to evaluate curricula and develop methods for teaching traditional disciplines, many educators feel unprepared to evaluate or use technology, yet they have an obligation to explaintheir technology expenditures to the tax-paying public by demonstrating how technology improves educational outcomes. The use of technology cuts across all academic areas, and it is changing rapidly, gaining greater capabilities seemingly every day. Most institutions--from businesses supported by resource-rich technology departments dedicated to mastering these powerful, expensive, and evolving electronic tools, to schools and school districts--are hard-pressed to keep current. Few general, informative, and stable guidelines exist anywhere. Thus, concrete examples of successful technology use and thoughtful criteria can be an important source of practical expertise and information for others.
In the field of education, the basic issues are: What is the appropriate role of the new technologies in education? How can the evolution of technology be directed efficiently and effectively toward improving student learning for all? How can what is known be shared and developed further? The work of the Educational Technology Expert Panel is a beginning attempt to address some of these questions. Much more work remains to be done.
THE EXPERT PANEL
In 1994, Congress directed the Office of Educational Research and Improvement (OERI) to establish "panels of appropriate qualified experts and practitioners" to evaluate educational programs and recommend to the Secretary of Education those programs that should be designated as exemplary or promising. The purpose of these panels is to provide teachers, administrators, policymakers, and parents with reliable information about the quality and effectiveness of programs and materials so that they can make better-informed decisions in their efforts to improve the quality of student learning. Other expert panels have been convened for Mathematics and Science Education; Gender Equity; and Safe, Disciplined, and Drug-Free Schools.
In 1998, OERI convened an Expert Panel on Educational Technology. The experts--drawn from universities, school-district administrations, preK-12 classrooms, research institutions, industry, government, philanthropic organizations--encompassed a wide variety of backgrounds in using technology for education.
After several months of in-depth discussions, and after an interactive process that solicited and used feedback from the field, the Expert Panel released its application and criteria for program submissions. The application and criteria are the result of a rare coming together of disparate, occasionally adversarial, points of view. Teachers insisted on the need for usable ideas from academe and practical support with technology in the classroom. Researchers insisted on the need for sound data, not anecdotes or hunches. Everyone agreed on the need for clarity--on the part of the Panel and on the part of programs for the purpose of dissemination.
The Educational Technology Expert Panel recognized the importance of precise definitions of terms and objectives very early in the process. The Panel began by defining information technology as a tool with multiple potential applications for education, and agreed that learning to use this tool is necessary but is only one component of sound educational practice. In the past, technology discussions tended to focus on the infrastructure and hardware, such as how to wire schools or whether to put the computers in the classroom or a separate laboratory. Today the field has matured sufficiently to focus on how to use the technology to improve teaching and learning. In fact, the objective of using educational technology should be to create an instructional environment that fosters significant learning for preK-12 students. Technology affords new ways of teaching established disciplines that are likely to be more effective for more students; equally important, technology can inspire teachers and students alike to move beyond conventional content and deepen the process of learning in various disciplines. Technology can be used to advance administrative and business functions in a school or district (e.g., to improve purchasing power, record-keeping, data-driven decision-making, communications with parents or between principal and faculty), which are important precursors to advancing student achievement. The Panel, however, decided that programs designated by the Department of Education should focus primarily on student learning.
The Panel viewed significant learning as being multifaceted and active. It is a combination of skills, knowledge, and the ability to apply and communicate understanding as well as to learn further.
What examples of technology programs working toward this kind of learning would be most useful as models to the field? The Expert Panel observed that with any endeavor involving technology, what is innovative today will be standard practice tomorrow. If the Panel had been assembled 2 years previous, the picture would have been very different. Then, the Internet was still a promising but exotic novelty; a technology plan meant dealing with equipment and was rarely linked to a plan for education.
One lasting indicator of technological innovation, however, might be student learning that cannot be achieved without technology. Today, for example, schools are using the Internet to connect disparate communities that otherwise would have no opportunity to meet or interact. Another powerful use of technology is modeling, an aid to experimentation that seeks to understand the physical world.
The Expert Panel further recognized that an exhaustive review of the many programs currently active would be impossible. Compared to math and science, disciplines in which the content is fairly stable but approaches change over time--particularly in the early grades--educational technology is a fast-changing, dynamic tool that responds to emerging innovations and allows for many different approaches to many disciplines, often several at the same time. As the capacity of technology grows, so does the power of new programs. Thus, the Panel concluded that the most lasting, far-reaching guidance to the field would come from establishing useful, thoughtful criteria and dissemination strategies rather than from any particular programs it designated.
The Expert Panel developed the six evaluation criteria and program indicators based on the four categories of criteria--(1) quality of program, (2) educational significance, (3) evidence of effectiveness, and (4) usefulness to others--established by federal regulations for all panels. The Panel's six evaluation criteria were carefully field-tested and resulted in the final review criteria presented on pages 9-13.
The Panel determined that an exemplary program
The Panel also determined that a promising program also possesses these qualities, but may not yet have rigorous and sufficient evidence for the program's achievements.
The Expert Panel was not seeking programs that teach basic technology skills (how to use e-mail or the Internet) or that automate current classroom activities ("drill and practice"). Nor was the Panel interested in exciting programs developed by teachers in single classrooms because historically, successful scale-up of approaches idiosyncratic to an individual instructor have proved frustratingly elusive. Rather, the Panel focused on significant learning for preK-12 students and sought innovative, thoughtful programs that reflect systemic improvement within and across disciplinary boundaries with various student populations.
From the Panel's research review and familiarity with educational outcomes in the classroom, it was well aware that schools are complex organizations. Among the essential concomitants of increased student learning are organizational change (e.g., scheduling of common planning time, new job titles such as technology coordinator, and prompt on-site technical support) and structural alignment (e.g., coordinating teacher professional development, curricula for inquiry-based learning, and assessment and curriculum standards) as well as an active commitment to equity. A truly powerful program benefits from contributions from different points of view and is successful with many kinds of learners, not just a narrow segment of students.
In addition, the Panel had observed that the most effective programs are the most coherent; that is, they have a clear, purposeful linkage between actions and outcomes. The Panel attributed coherence to a clear articulation of the program's objectives and conditions for success, combined with a constant, critical monitoring of evidence that supports or contravenes the program's assumptions. The Panel further noted that useful evidence might consist of simple pre- and posttests, or the tracking of changes in student behavior (e.g., attendance and graduation rates) and is not limited to formal quantitative studies.
The Panel believes that successful outcomes are the cumulative result of effort in multiple areas and has structured its criteria to reflect that belief.
The Panel's definition of program goes well beyond that of a textbook or curriculum for a single discipline:
Program: The implementation of a set of interrelated strategies and activities directed toward increased learning. A program includes assessment that demonstrates its impact on participants and on a wider organizational structure (i.e., a school or cluster of schools, a district, community organization, partnership, or other distributed system). Eligible programs depend on technology to achieve their goals.
The Educational Technology Expert Panel believes that no one electronic device can fulfill its definition of program without a sustaining and complex structure of support. A program cannot be dissected and only certain components extracted and used--a program functions as an integral, interlocking whole.
The Designated Programs
The selection of exemplary and promising programs reflects the potential of educational technology for significant learning. Equally important, the programs serve to affirm and encourage the continued efforts of many others across the country who share a commitment to maximizing educational technology for the benefit of improved learning.
Given that new projects are constantly being developed and that the Panel was dependent on the set of self-selected applications received, there is no assurance that the list of selected programs is fully representative of the variety of programs being implemented across the country. Rather, the list is illustrative; the programs exemplify excellence regarding certain features the Panel considered critical, including their effectiveness in exploiting the unique capabilities of technology and their contribution to professional development, equity, and assessment. The designated programs stand as models of the kind of thinking that leads to increased and sustained student learning.
The process of identifying excellence is ongoing. The exemplary programs and many of the promising programs can serve as models to all schools. The learning goals, methods, and design have universal application. Virtually any school with leadership and imagination can adapt these programs to their own populations, strengths, and weaknesses. The winners will be the students, teachers, administrators, and the community.
How to Use the Criteria
The Panel's criteria reflect how experts in the field are currently thinking about technology and learning. The criteria can be used as guidelines when creating and improving programs that use educational technology. They provide a rigorous, analytical model for districts and schools undergoing planning and development. They are useful for policymakers deciding on how to benefit fully from connecting their buildings to the Internet, what objectives they want to achieve and with whom, what sorts of technology programs they want to create or adapt, what sorts of implementation issues must be considered, and what sort of assessment will be effective and persuasive.
The criteria can also be used to explain to school boards, parents, and others why educators should bring technology into schools. They can also be used as an assessment guide to help schools scrutinize their current use of technology: How is technology currently being used for student learning? Is equity addressed consciously and sufficiently? Are systemic supports in place, and are more supports needed?
The criteria reflect the experience of experts in the field, and, as such, can stimulate creativity and help avoid pitfalls.
The Review Process
The submission process was open, widely publicized, and designed to encourage applications from a variety of organizations. One hundred thirty-four programs were submitted for review. The review encompassed three stages. First, every submission was examined by a team drawn from 30 reviewers, each of whom had expertise in technology in addition to a content specialty such as math or an area of expertise such as school reform, research, or evaluation. This was the Quality Review Panel (QRP). Reviewer training included a series of online discussions with Expert Panel members over a period of 3 weeks, and then an onsite orientation conducted by a team of 4 Panel members. These QRP teams determined eligibility and verified that the application was complete; scored the quality of the submission in accord with the Panel's published criteria; and determined whether the submission merited being forwarded to the next group of experts--the Impact Review Panel (IRP). In the second stage, the IRP, consisting of national experts in evaluation design and analysis, assessed the persuasiveness of the evaluation data and the claims made by the program. Third, members from the Expert Panel reviewed all of the programs, along with the ratings and comments of the QRP and IRP review teams, to determine which programs to recommend to the Secretary of Education as exemplary or promising.
The criteria used to judge the programs and the program descriptions for the two exemplary and five promising programs designated by the Secretary of Education follow. The descriptions are based on information provided by the developers. Contact information for each program is included.
Next Steps
The main goal of the Educational Technology Expert Panel was to develop and promulgate a shared vision, a common framework, and specific definitions and criteria to raise the level of conversation and agreement within the field about educational technology in order to improve teaching and learning. Educational technology is still viewed by some as a matter of installing hardware and software, with insufficient attention being devoted to quality teacher training and evaluation. We hope that the Panel's work will help policymakers, school administrators, educators, and parents gain a more comprehensive view of the integral role that technology can play in curriculum development, organizational change, and school reform.
[Evaluation Criteria]
This page was last updated March 15, 2002 (jer)