![[Table of Contents]](/icons/up.gif){kind=icon}
Although comparisons give an indispensable global dimension to performance information, they are by no means the only, or even the most important, contribution of TIMSS. If state and local educators focus only on the "horse race" aspect of the TIMSS data, they will miss its true value for benchmarking--namely, what states and districts do with the information. Countries like Japan that rank near the top of the TIMSS nations use international data to find areas for further improvement, rather than basking in their accomplishments. The U.S. educational system does not have this same concept of continuous improvement, partly because we lack a sense of what the gradations of quality are and partly because we do not have incentives and support structures that encourage students, teachers, and schools to become better and better.
One basic use of TIMSS at the state and local level is as a benchmark. States and school districts can use TIMSS to help determine how well their own schools and students perform by international standards in mathematics and science and where they need to improve.
Some states and school districts have gone the full route of administering actual TIMSS tests to representative samples of their students, at their own expense and by arrangement with the U.S. Department of Education. The state of Missouri, for example, has administered the TIMSS tests to a sample of 3,000 students. The First in the World Consortium, a group of 20 high-achieving school districts in Illinois, has undertaken an ambitious benchmarking effort by giving the TIMSS tests to a statistically valid sample of Consortium student (Paul Kimmelman, West Northfield School District #31, Illinois; David Kroeze, Northbrook School District #27, Illinois, TIMSS Policy Forum).
After receiving the preliminary achievement data, the First in the World Consortium decided to expand the project by contracting for curriculum and videotape analyses patterned after the TIMSS model. The districts hope to use these data to identify areas where they need to do better and to engage educators, parents, students, and community in a process of continuous improvement. The process designed by the Consortium has four basic components: (1) developing a common, coherent vision; (2) using data as a vehicle for inquiry and self-reflection; (3) encouraging professional mastery and sustained professional development; and (4) collaborating with other school districts, universities, research centers, and additional partners.
A district does not have to go through the full route of testing and analysis in order to use TIMSS as a catalyst for local introspection. The Department of Education has developed a five-part TIMSS "toolkit" to help districts through the benchmarking and improvement process (OERI, 1997). The "Guide" to the kit outlines its contents and how they might be used. The "Education" module of the kit is intended to provide educators, parents, policymakers, and concerned citizens with an overview of the TIMSS findings. It includes summaries of the main TIMSS reports, an introductory video, and a guide for local discussion. It is meant to serve as a kind of "medical alert" that will wake up complacent communities who think their schools are good enough just because they're as good as the neighboring district's (Lois Peak, NCES; and Kroeze, TIMSS Policy Forum).
The "Achievement" module of the toolkit takes users through a "diagnosis" phase. It is intended to help educators use TIMSS items, in conjunction with their own assessment results, to reflect on how their students measure up by international standards (Peak, TIMSS Policy Forum). This module includes in-depth student achievement data from the TIMSS nations, as well as actual items from the TIMSS tests. For communities and parents that are accustomed to seeing their children score above average on norm-referenced tests, this type of analysis can spur them to think in terms of what their children do and do not know, and what they can and cannot do.
The final two modules of the TIMSS toolkit help educators research "treatment options" by making them aware of instructional and curriculum practices in other countries. The "Teaching" module includes videotaped excerpts of lessons from the TIMSS video study; transcripts, notes, and discussion guides to accompany the taped lessons; and excerpts from mathematics standards produced by the National Council of Teachers of Mathematics (NCTM) in the United States and the Japanese Ministry of Education. The "Curricula" module contains a guidebook to help people analyze their own curriculum and compare it with other international and national approaches, so they can decide directions for curricular and instructional reform.
Some states and districts are already engaged in data-driven benchmarking and introspection, similar to the process laid out in the TIMSS toolkit. When educators deeply analyze student achievement data, they may come across surprises, both positive and negative. Vermont, for example, was compelled to ask hard questions about the impact of its new portfolio assessment program when data showed that less than one-fourth of the participating students had met the state standards for problem solving (Marjorie Petit, Vermont Department of Education, TIMSS Policy Forum). In some schools with fully implemented portfolios, no student met the standard. These results were particularly disconcerting because strengthening students' problem-solving was a primary goal of the state reforms. When state officials delved into samples of student work from the portfolios they found that many teachers did not fully understand what a substantive problem was, and therefore were not adequately teaching problem solving or higher math. Many Vermont schools and districts have responded by providing teachers with professional development to build a deeper understanding of problem-solving in math. Thus, although the Vermont data did not show positive results, something positive came out of it: state and local educators identified a problem with teachers’ knowledge and took steps to address it.
TIMSS can be an effective tool for using achievement data to benchmark progress and drive school improvement. Several large urban systems are using a model of a data-driven improvement process for urban schools which includes the following five components: (1) student outcome data (such as disaggregated student test scores and graduation rates) and staff data (such as staff attendance, involvement in professional development, and preparation in various content areas); (2) information on parents' involvement in activities that support education; (3) information about special programs currently operating at the school level; (4) school-community partnerships; and (5) information on grant writing (Sharon Johnson-Lewis, Council of the Great City Schools, TIMSS Policy Forum). The TIMSS findings can provide an international benchmark for these schools.
[What TIMSS Says About Equity Issues]