R&D: Innovate and Scale

The model for learning presented in this plan assumes that we will develop, adopt, and ensure equitable access to a technology-based education system that provides effective learning experiences, assessments, and teaching and a comprehensive infrastructure for learning to support both formal education and all other aspects of learning. It also assumes we will incorporate many of the practices other sectors regularly use to improve productivity and manage costs and will leverage technology to enable or enhance them.

In the past, we have relied on public education entities and private companies to develop technology resources and tools for learning. In both these sectors, however, incentives are provided for developing discrete products and services without regard for how they work as parts of a system or for research on their effectiveness. Public education entities can mandate use of their products and services. Commercial enterprises gain market share through compelling value propositions, effective marketing, and broad distribution channels. But research on the effectiveness of learning technology typically comes after products and services have been deployed—when it is too late to result in major improvements—if it comes at all.

If we are to achieve our goal of leading the world in education, we must be leaders in the design and implementation of a more effective education system. To that end, this plan calls for a new approach to R&D for education that focuses on four areas:

• Continuing to provide competitive grants for scaling up innovative and evidence-based practices through the Department of Education's Investing in Innovation Fund (i3).

The i3 program provides funding for grants that are awarded to schools and nonprofit organizations for scaling innovations that improve K–12 education. There is a particular focus on the identification of evidence and increasing the level of understanding of what strategies and innovations work for what students under what circumstances.

• Transferring existing and emerging technology innovations from such sectors as consumer, business, and entertainment into education.

The Department of Education will promote the inclusion of innovative thinkers in consumer, business, and entertainment technology in federally funded convenings and collaborations with educational technology specialists.

• Supporting and sustaining the education R&D that is currently happening at the National Science Foundation, especially through its cyberlearning initiatives.

In June 2008, the NSF Task Force on Cyberlearning published Fostering Learning in the Networked World: The Cyberlearning Opportunity and Challenge, a comprehensive report on the role technology can and should play in STEM learning. For 2011, the NSF has established a new multidisciplinary research program to fully capture the transformative potential of advanced learning technologies across the education system. The Cyberlearning Transforming Education (CTE) program's investments will focus on anytime, anywhere learning, personalized learning, and using technology to advance our fundamental knowledge of how technology and learning sciences can come together in learning and assessment.

• Creating a new organization with the mission of serving the public good through R&D at the intersection of learning sciences, technology, and education (Pea and Lazowska 2003).

The Higher Education Opportunity Act (P.L. 110-315), passed in August 2008, authorizes establishment of the National Center for Research in Advanced Information and Digital Technologies (also called the Digital Promise). The center is authorized as a 501(c)3 that would be able to accept contributions from the public and private sectors to support the R&D needed to transform learning in America.

The National Center for Research in Advanced Information and Digital Technologies would support research at scale, facilitating the participation of educators, schools, and districts as partners in design and research. It would also promote transparency and collaboration, encouraging multiple researchers to work with the same data and interoperable software components and services. Its unique charter is to identify the key research and development challenges in the education field and coordinate the best combination of expertise for addressing them. These characteristics, along with an emphasis on public-private collaboration, distinguish the National Center for Research in Advanced Information and Digital Technologies from existing centers that help state and local education entities identify and implement established best practices in learning technology. The center's work would also be distinct from field-initiated research on the effectiveness of technology-based interventions.

The Defense Advanced Research Projects Agency (DARPA) offers an example of how such a research agency can promote work that builds basic understanding and addresses practical problems. DARPA sponsors high-risk/high-gain research on behalf of Department of Defense agencies, but it is managed and staffed by individuals from both industry and academia who are experts in the relevant research areas. DARPA program officers are given considerable discretion, both in defining the research agenda and making decisions about the funding and structuring of research.

In a similar manner, the National Center for Research in Advanced Information and Digital Technologies should identify key emerging trends and priorities and recruit and bring together the best minds and organizations to collaborate on high-risk/high-gain education R&D projects. It should aim for radical, orders-of-magnitude improvements by envisioning the impact of innovations and then working backward to identify the fundamental breakthroughs required to make them possible.

Through the funding of rapid and iterative cycles of design and trial implementation in educational settings, the national center can demonstrate the feasibility and early-stage potential of innovative tools, content, and pedagogies that leverage knowledge, information, and technology advances at the cutting edge.

The center should also ensure that teams working on each individual project share developments, progress, best practices, and outcomes with each other to take advantage of key findings and economies of scale and to ensure integration and interoperability between projects when desirable. The national center will need to work closely with representatives of private industry to develop clear memoranda of understanding concerning the terms for precompetitive fundamental research.

The national research center can focus on grand challenge problems in education research and development. "Grand challenge problems" are important problems that require establishing a community of scientists and researchers to work toward their solution.

American computer science was advanced by a grand challenge problems strategy when its research community articulated a set of science and social problems whose solutions required a thousand-fold increase in the power and speed of supercomputers and their supporting networks, storage systems, and software. Since that time, grand challenge problems have been used to catalyze advances in genetics (the Human Genome Project), environmental science, and world health.

To qualify as grand challenge problems suitable for this organization, research problems should be

• Understandable and significant, with a clearly stated compelling case for contributing to long-term benefits for society

• Challenging, timely, and achievable with concerted, coordinated efforts

• Clearly useful in terms of impact and scale, if solved, with long-term benefits for many people and international in scope

• Measurable and incremental, with interim milestones that produce useful benefits as they are reached.

This kind of grand challenge problem strategy has driven innovation and knowledge building in science, engineering, and mathematics. The time is right to undertake it to improve our education system (Pea 2007).

The following grand challenge problems illustrate the kinds of ambitious R&D efforts this organization could lead. Notably, although each of these problems is a grand challenge in its own right, they all combine to form the ultimate grand challenge problem in education: establishing an integrated, end-to-end real-time system for managing learning outcomes and costs across our entire education system at all levels.

1.0: Design and validate an integrated system that provides real-time access to learning experiences tuned to the levels of difficulty and assistance that optimize learning for all learners and that incorporates self-improving features that enable it to become increasingly effective through interaction with learners.

Today, we have examples of systems that can recommend learning resources a person might like, learning materials with embedded tutoring functions, software that can provide UDL supports for any technology-based learning materials, and learning management systems that move individuals through sets of learning materials and keep track of their progress and activity. What we do not have is an integrated system that can perform all these functions dynamically while optimizing engagement and learning for all learners. Such an integrated system is essential for implementing the individualized, differentiated, and personalized learning called for in this plan. Specifically, the integrated system should be able to

• Discover appropriate learning resources;

• Configure the resources with forms of representation and expression that are appropriate for the learner's age, language, reading ability, and prior knowledge; and

• Select appropriate paths and scaffolds for moving the learner through the learning resources with the ideal level of challenge and support.

As part of the validation of this system, we need to examine how much leverage is gained by giving learners control over the pace of their learning and whether certain knowledge domains or competencies require educators to retain that control. We also need to better understand where and when we can substitute learner judgment, online peer interactivity and coaching, and technological advances, such as smart tutors and avatars for the educator-led classroom model.

2.0: Design and validate an integrated system for designing and implementing valid, reliable, and cost-effective assessments of complex aspects of 21st-century expertise and competencies across academic disciplines.

The multiple-choice tests used in nearly all large-scale assessment programs fail to meet the challenge of capturing some of the most important aspects of 21st-century expertise and competencies. Past attempts to measure these areas have been expensive and of limited reliability. Technology offers new options for addressing the multiple components of this challenge. For example, technology can support

• Systematic analysis of the claims about student competence (including competence with respect to complex aspects of inquiry, reasoning, design, and communication) intended by academic standards and the kinds of evidence needed to judge whether or not a student has each of those aspects of competence;

• Specifying assessment tasks and situations that would provide the desired evidence;

• Administering complex assessment tasks capable of capturing complex aspects of 21st-century expertise through the use of technology; and

• Developing and applying rules and statistical models for generating reliable inferences about the learner's competencies based on performance on the assessment tasks.

Promising R&D applying technology to each of these components of the grand challenge is ongoing, but the pieces have yet to be integrated into a single system that is applicable across content domains and that is cost-effective to implement. In addition to system development, solving this grand challenge problem will require studies to demonstrate the validity of the new assessments and their usefulness for both making formative instructional decisions to improve learning and summative evaluative decisions for purposes of establishing competency and accountability.

3.0: Design and validate an integrated approach for capturing, aggregating, mining, and sharing content, student-learning, and financial data cost-effectively for multiple purposes across many learning platforms and data systems in near real time.

To meet the education and productivity goals articulated in this plan, learners and their parents, educators, school and district leaders, and state and federal policymakers must use timely information about student-learning and financial data to inform their decisions. Today, these data are maintained in a variety of digital formats in multiple systems at local and state levels. As the processes of learning, assessment, and financial management and accounting move into the digital realm, education data systems and education research have become exceedingly complex in terms of scale, heterogeneity, and requirements for privacy. Still, we must create systems that capture, curate, maintain, and analyze education and financial data in all scales and shapes, in near real time, from all areas where learning occurs: school, home, and community. This must be done in a manner fully consistent with privacy regulations.

Although underlying technologies for exchanging data sets exist, education does not yet have the kind of integrated Web-enabled data-sharing system that has been developed for the health-care, telecommunications, and financial sectors. Such a system must be capable of dealing with both fine-grained data derived from specific interactions with a learning system and global measures built up from that data, and it must be able to collect, back up, archive, and secure data coming from many different systems throughout a state. It must also be capable of integrating the financial data essential for managing costs. Addressing this challenge will require:

• A data format to represent learning and financial data;

• A service to discover and exchange data;

• A data security standard for the service;

• A specification, test suite, and reference implementation of the service to ensure vendor compliance; and

• Best practices to guide the deployment of such services.

4.0: Identify and validate design principles for efficient and effective online learning systems and combined online and offline learning systems that produce content expertise and competencies equal to or better than those produced by the best conventional instruction in half the time at half the cost.

Research labs and commercial entities are hard at work developing online learning systems and combined online and offline learning systems that support the development of expertise within and across academic disciplines. Although we have isolated examples of systems producing improved learning outcomes in half the time, we have yet to see this kind of outcome achieved within the K–12 system and particularly in those schools where students need help the most. In addition, in both K–12 and higher education, we have yet to see highly effective systems that can be brought to scale.

We have evidence that learning can be accelerated through online tutoring, restructuring curricula, and by providing guiding feedback for improvement throughout the learning process. Further, we know that the current "packages" of learning that define semester and yearlong courses are generally arbitrary and a result of long-standing tradition rather than of careful studies. Achieving twice the content expertise and competencies in half the time at half the cost through online learning systems seems very possible, but it will require careful design, development, and testing.