U.S. DEPARTMENT OF EDUCATION
PLANNING AND EVALUATION SERVICE
EDUCATIONAL TECHNOLOGY EVALUATION ACTIVITIES OVERVIEW
(January 2002)
In a modern classroom, educational technology improves teaching and learning while also equipping students with the skills needed for tomorrow's workforce. Evaluation activities at the U.S. Department of Education address what can be learned from the federal investment in educational technology, the effectiveness of federal programs involving educational technology, and how they are contributing to improving teaching and learning. Key evaluation activities in educational technology are outlined below.
EDUCATIONAL TECHNOLOGY EVALUATION ACTIVITIES
Technology Literacy Challenge Fund and E-rate
Integrated Studies of Educational Technology (ISET)
The Department of Education is coordinating three studies examining various areas related to educational technology so that the same nationally representative sample of teachers, schools and districts can be used to answer questions about all issue areas. Collectively called ISET, these three studies are:
This study is reviewing and synthesizing research on the connection between home and school regarding the use of educational technology for learning. In addition, case studies will be conducted to identify model practices for technology enhanced home-school interactions. The final report, planned for Spring 2002, will provide an overview of the existing research, identify trends, note where additional information is needed, and be used to inform the design of future evaluation work.
Assessing the Technology Proficiencies of Students and Educators
This study is reviewing and synthesizing research on efforts to measure, report and use data on student, teacher, and administrator proficiencies in the use of educational technology. Lessons learned will be used to inform Department of Education activities, including work on program performance indicators and in program evaluation. A final report is planned for Spring 2002.
Evaluation of Discrete Educational Software for Knowledge-building (E-DESK)
This study is reviewing and synthesizing research on the effectiveness of discrete educational software. The report developed for this task will provide an overview of existing research, note areas where additional information is needed, identify promising models, and inform design of future evaluation work. The final report is planned for Spring 2002.
The Department of Education is coordinating three studies examining various areas related to educational technology so that the same nationally representative sample of teachers, schools and districts can be used to answer questions about all issue areas. Collectively called ISET, these three studies are:
- Supplemental Study of the Technology Literacy Challenge Fund
This study is collecting and analyzing information about the implementation and outcomes of the Technology Literacy Challenge Fund at the state and local levels. Drawing upon sources such as the annual state TLCF performance reports, local technology plans, and survey work, this study will produce a national representative picture of TLCF's contributions to the availability and use of technology in schools and provide information on targeting, flexibility and other key aspects of the program. The final report for this study is expected in Spring 2002. - Professional Development for the 21st Century Classroom
This study is considering what represents best practices in professional development for teachers with respect to educational technology. It draws heavily on professional development literature, review the current status of professional development in educational technology, and study the circumstances under which professional development in the instructional use of technology results in changes in teaching and learning. The study includes the ISET surveys and longitudinal case studies in a subsample of districts surveyed. The final report for this study is expected in Spring 2002. - Formative Evaluation of the E-rate
This study is analyzing the extent to which the E-rate is equalizing access to educational technology and the extent to which educational technology is integrated with systemic reform efforts to improve teaching and learning in schools and districts utilizing the E-rate. The study includes an analysis of the existing data on the E-rate, a review of E-rate applicant technology plans, and the ISET surveys. This study's interim report is based on administrative data for the program and is available at http://www.ed.gov/offices/PES/eval/elem.html#technology. A final report including ISET data is planned for Spring 2002.
This study is reviewing and synthesizing research on the connection between home and school regarding the use of educational technology for learning. In addition, case studies will be conducted to identify model practices for technology enhanced home-school interactions. The final report, planned for Spring 2002, will provide an overview of the existing research, identify trends, note where additional information is needed, and be used to inform the design of future evaluation work.
Assessing the Technology Proficiencies of Students and Educators
This study is reviewing and synthesizing research on efforts to measure, report and use data on student, teacher, and administrator proficiencies in the use of educational technology. Lessons learned will be used to inform Department of Education activities, including work on program performance indicators and in program evaluation. A final report is planned for Spring 2002.
Evaluation of Discrete Educational Software for Knowledge-building (E-DESK)
This study is reviewing and synthesizing research on the effectiveness of discrete educational software. The report developed for this task will provide an overview of existing research, note areas where additional information is needed, identify promising models, and inform design of future evaluation work. The final report is planned for Spring 2002.
Other Educational Technology Programs
First Formative Evaluation of Preparing Tomorrow's Teachers to Use Technology (PT3)
This formative study evaluates the activities, strategies and trends used by PT3 grantees to prepare future teachers to effectively integrate technology into classroom practice. The evaluation will: 1) examine the goals, strategies and processes of reform among program grantees through case studies and analyses; 2) design and analyze program performance report data; 3) compile a literature review of quality research, evaluation, and "best practices" in the field of pre-service training in technology; and 4) report on current assessment instruments available to evaluate teachers' technology proficiency, and the methodology and design issues ED should consider in a future evaluation of the technology-proficiency of teacher program students and graduates. Final products include a grant review, a case study report and an analysis of program performance reports. All products are available at http://www.ed.gov/offices/OUS/PES/higher.html#pt3.
Formative Evaluation of the Learning Anytime Anywhere Partnerships (LAAP)
This formative evaluation focuses on the 29 projects funded in LAAP's FY99 grant competition. LAAP's goal is to improve the use of emerging technologies in distance delivery of postsecondary education and training through broadening access to underserved learners, improving quality of instruction, and demonstrating economies of scale. The evaluation will test instruments for collecting data on indicators of these and related objectives; review, validate, and analyze first-year Annual Reports; establish baseline data; and design a multi-year evaluation plan for assessing LAAP's impact on the practice of distance education and "best practices" in the field. Preliminary information on early obstacles and characteristics of success in large-scale, national partnerships will also be developed. "A new Executive Summary of the report will be on the LAAP website in Spring 2002."
Formative Evaluation of the Technology Innovation Challenge Grant Program
This formative evaluation examines how and to what extent the TICG Program is supporting the development, implementation, and expansion of innovative uses of technology to improve teaching and learning. A case study report will be released in Spring 2002.
The Star Schools Program: Reaching Students and Teachers through Distance Education Technology
Based on surveys of teachers, site visits, and discussions with program staff, project staff, project partners, and teachers, this formative evaluation examines the design and administration of the Star Schools Program in providing access of under-served student populations to education through distance learning. A final report will be released in Summer 2002.
This formative study evaluates the activities, strategies and trends used by PT3 grantees to prepare future teachers to effectively integrate technology into classroom practice. The evaluation will: 1) examine the goals, strategies and processes of reform among program grantees through case studies and analyses; 2) design and analyze program performance report data; 3) compile a literature review of quality research, evaluation, and "best practices" in the field of pre-service training in technology; and 4) report on current assessment instruments available to evaluate teachers' technology proficiency, and the methodology and design issues ED should consider in a future evaluation of the technology-proficiency of teacher program students and graduates. Final products include a grant review, a case study report and an analysis of program performance reports. All products are available at http://www.ed.gov/offices/OUS/PES/higher.html#pt3.
Formative Evaluation of the Learning Anytime Anywhere Partnerships (LAAP)
This formative evaluation focuses on the 29 projects funded in LAAP's FY99 grant competition. LAAP's goal is to improve the use of emerging technologies in distance delivery of postsecondary education and training through broadening access to underserved learners, improving quality of instruction, and demonstrating economies of scale. The evaluation will test instruments for collecting data on indicators of these and related objectives; review, validate, and analyze first-year Annual Reports; establish baseline data; and design a multi-year evaluation plan for assessing LAAP's impact on the practice of distance education and "best practices" in the field. Preliminary information on early obstacles and characteristics of success in large-scale, national partnerships will also be developed. "A new Executive Summary of the report will be on the LAAP website in Spring 2002."
Formative Evaluation of the Technology Innovation Challenge Grant Program
This formative evaluation examines how and to what extent the TICG Program is supporting the development, implementation, and expansion of innovative uses of technology to improve teaching and learning. A case study report will be released in Spring 2002.
The Star Schools Program: Reaching Students and Teachers through Distance Education Technology
Based on surveys of teachers, site visits, and discussions with program staff, project staff, project partners, and teachers, this formative evaluation examines the design and administration of the Star Schools Program in providing access of under-served student populations to education through distance learning. A final report will be released in Summer 2002.
