TIMSS 12th-Grade Report
Questions and Answers

The Context of Student Achievement

U.S. performance has improved over time on the National Assessment of Educational Progress (NAEP). Since the 1980s, the mathematics and science performance of our nation's students has improved for 9-, 13-, and 17-year-olds. Moreover, there has been significant improvement in students scoring at the proficient or above levels in mathematics at 4th, 8th, and 12th grades since 1990. For example, on the 12th-grade mathematics NAEP, 12 percent of students in 1990, 15 percent of students in 1992, and 16 percent of students in 1996 scored at or above the proficient level. In addition, the average score of 12th-grade students at the 90th percentile on NAEP has increased significantly since 1990. There has also been significant improvement on the Scholastic Aptitude Test (SAT) over time. Between 1982 and 1996, SAT mathematics scores increased by 15 points. [Mathematics NAEP 1990, 1992, 1996; NAEP 1996 Trends in Academic Progress; The College Board, Scholastic Aptitude Test Data, 1982-1996]

However, TIMSS and other international assessments show that the bar is being raised and we are not keeping up. While international comparisons over time are difficult, it appears that our relative standing to other nations has not improved despite our students' increasing achievement in mathematics and science as measured by NAEP. The first international studies of mathematics and science achievement were conducted in the 1960s, and there have been three assessments since that time. However, each assessment has been done differently (nations participating, content area topics, types of students sampled in each nation, the improvement of assessment, the addition of specialist exams, and even the changing borders of some countries). Despite these factors, the pattern of U.S. performance has been relatively consistent. Our relative ranking at the 12th grade was below the international average in the First and Second International Mathematics and Science Studies conducted in the 1960s and 1980s. Improving our relative standing requires us to improve more rapidly over time than other nations.

2. How rigorous are U.S. state assessment standards in mathematics?

Many state assessments are much less rigorous than nationally set standards like those used in NAEP. A recent Southern Regional Education Board study revealed that in some states, more than 80 percent of the students are proficient on state 8th grade mathematics assessments, but only 20 percent or less were proficient based on the NAEP 8th grade mathematics standards. (See accompanying chart with state-by-state comparisons of 8th grade NAEP mathematics scores with states' own assessment scores.) Similarly, the recent Quality Counts study by Education Week rated the rigor of state standards in mathematics at 8th grade and gave only one state a grade of "A."

3. To what extent do our students take high level mathematics and science?

Despite some progress, many students are still getting limited exposure to challenging mathematics and science content in middle and high school. Many high school students do not make it from algebra and geometry to advanced mathematics and science courses. Fully 90 percent of all U.S. high school students stop taking rigorous mathematics before getting to calculus. Even among college bound seniors, 52 percent have not taken physics, 48 percent have not taken trigonometry, and 77 percent have not taken calculus. Unlike most other countries where algebra and geometry are introduced in the middle grades, in the U.S. only 25 percent of students take algebra before high school. The most common mathematics course taking path for high school students is 9th grade algebra I, geometry, and algebra II, and then no further mathematics. [Mathtech, Analysis of NELS: 88 data, September 1997; U.S. Department of Education, National Center for Education Statistics, The 1994 High School Transcript Study Tabulations: Comparative Data on Credits Earned and Demographics for 1994, 1990, 1987, and 1982 High School Graduates, 1996]

State requirements for and numbers of students enrolled in challenging mathematics and science courses has increased in the last decade, but there is still a great need for improvement. In the 1980s, over 40 states raised the number of credits in mathematics and science necessary for graduation. In 1996, 43 states required at least two years of mathematics and science compared to 9 nine states in 1980. As a result, since 1990, all states show an increase in the proportion of high school students taking higher level mathematics and most states show an increase in higher level science. Among 1994 high school graduates, 9 percent had taken calculus and 24 percent had taken physics, as compared to 5 percent and 14 percent respectively in 1982. Additionally, AP participation has increased dramatically since 1982, while the percentage of examinations with passing scores and the mean score have not declined significantly. The percent of students taking the AP calculus examination more than tripled, and the percent of students taking and passing the AP physics exam also grew dramatically. [Council of Chief State School Officers, State Indicators of Science and Mathematics Education 1997, State-by-State Trends and New Indicators from the 1995-96 School Year; The College Board, Advanced Placement Data, 1982-1996; U.S. Department of Education, National Center for Education Statistics, The 1994 High School Transcript Study Tabulations: Comparative Data on Credits Earned and Demographics for 1994, 1990, 1987, and 1982 High School Graduates, 1996]

4. How do the 12th-grade TIMSS scores relate to the 4th and 8th grade scores? How do U.S. students perform in mathematics and science over time?

Although U.S. students' achievement increases as they progress through school, our relative standing compared to other TIMSS countries drops from 4th to 8th to 12th grade. One reason is that the rate of increase in U.S. student achievement slows in high school. Data from NAEP show that there is less change in average scores in mathematics between 8th and 12th grades than between 4th and 8th grades. Part of the explanation for this may be that many U.S. high school students do not take 4 years of mathematics. For example, even among college bound high school seniors (those taking the SAT), 31 percent did not take 4 years of mathematics. [Mathematics Course-Taking and Gains in Mathematics Achievement,@ 1995. NCES 95-714]

5. How well prepared are our high school mathematics and science teachers?

An extremely high percentage of high school mathematics and science teachers lack adequate preparation in the content area they teach. Twenty-eight percent of U.S. public high school (grades 9-12) mathematics teachers were teaching out-of-field (without a major or minor) in 1993-94. In that same year, 55 percent of public high school physical science teachers (physics, chemistry, or geology) were teaching out-of-field. These percentages increase with poverty status of the schools, and for schools with high percentages of minority students. [Schools and Staffing Surveys, 1993-94]

Dispelling Common Myths About U.S. Student Performance

No. A similar proportion of students complete secondary schooling in most other TIMSS countries and in the U.S. and TIMSS measured the performance of all students completing secondary school regardless of the type of secondary program they completed. While in the past a much higher proportion of students completed secondary schooling in the U.S. than in many European countries, today enrollment rates in secondary education (as a percentage of the appropriate age cohort) for most of the countries participating in TIMSS are as high as or higher than the U.S. Moreover, the design of the study required countries to include students in the last year of secondary education in all "tracks," so countries did not include only those students in the highest academic tracks. Thus, countries with highly differentiated systems with some programs extending through grade 13 and 14 and others, usually vocationally-oriented programs, ending as early as grade 10, tested students in the last year of all of those programs.

7. Isn't it true that the range of scores for U.S. students is greater than in many countries and we have a concentration of very poor performing students who drag down the overall average of our students?

No. The range of scores in the U.S. is not greater than in most of the other countries on the general knowledge assessments. The difference in scores between the 5th and 95th percentile is very similar in most of the countries (about 300 points), including the U.S. The lower "tail" of the distribution is not larger or longer in the U.S. than in the other countries. The difference between the scores of students at the 5th percentile and the median score (the 50th percentile) is also similar in the U.S. to the other countries. What is true is that the entire U.S. distribution of scores, throughout the distribution, is shifted downward compared to the higher achieving countries -- the scores of our students are generally lower than those in the other countries at the 95th, 75th, 50th, 25th and 5th percentiles.

8. Don't private school students perform better than our public school students? In general, the courses students take in high school determine their achievement more than the type of school they attend. When course-taking patterns are accounted for, the mathematics and science achievement of public and private school students is very similar. For example, a recent analysis shows that public and private school students who took the same mathematics courses were almost equally likely to score at the highest level on the 12th-grade mathematics achievement test of the National Educational Longitudinal Study. (See accompanying chart on mathematics achievement.) [U.S. Department of Education, Mathematics Equals Opportunity; Madigan, Science Proficiency and Course Taking in High School, 1997, NCES 97-838].

9. Are our top students as good as those in other countries?

No. This is true neither in the general knowledge assessments nor in the advanced assessments. In the general knowledge assessments, the students scoring at the 95th percentile in the U.S. consistently have scores that are below those at the 95th percentile in most of the other countries. Furthermore, in the mathematics general knowledge assessment, the scores of U.S. students at the 95th percentile are comparable to students at the 75th percentile in some of the high scoring countries. In other words, the 5 percent of our students score about as well as the best 25 percent of students in those other countries.

In the physics and advanced mathematics assessments, a comparison of the fourteen percent of U.S. students taking advanced courses to comparable proportions of advanced students in the other countries shows that our students perform poorly. Our most advanced students, defined as the 5 percent taking AP calculus and the 1 percent taking AP physics, score about as well as 10-20 percent of the most advanced students in the other countries.

10. Isn't there a "gender gap" in mathematics and science achievement?

On our measures of all students in the U.S., there was not a significant gender gap in mathematics in TIMSS at 4th, 8th or 12th grade. This contrasts with most other nations participating in TIMSS which demonstrated a significant gender gap in mathematics at 12th grade (but not at the earlier grades). While there was a gender gap in science in the U.S. at the 12th grade as in all the other countries except one, the U.S. gender gap was one of the smallest. However, on the advanced mathematics and on the physics assessments, males outperformed females in the U.S. and most of the other TIMSS countries even though females take these courses at about the same rate as males.

11. Did U.S. students do poorly because there were items on the assessment that involved the metric system, with which they were unfamiliar?

U.S. students did no more poorly on items involving the metric system than they did on other types of items. Many of the items in which the metric system was an element did not require students to know anything about the metric system (other than it is a system of measurement) in order to answer the item. For example, in a geometry item where the students were asked to calculate the area of a figure, a student would calculate the area the same way whether the dimensions of the figure are given in centimeters or inches.

Background on TIMSS and Its Design

The Third International Mathematics and Science Study (TIMSS) is the largest, most comprehensive international study of mathematics and science achievement ever. More than 500,000 students from 41 countries participated in the assessments, administered in 30 languages to pupils at three grade levels.

13. Who sponsored TIMSS?

The tests were coordinated by the International Association for the Evaluation of Educational Achievement (IEA), an independent international cooperative of research centers and departments of education in more than 50 countries. Dr. Albert E. Beaton is directing the international activities of the study with his staff at the International Study Center at Boston College.

The international costs of TIMSS were provided by the National Center for Education Statistics (NCES) of the U.S. Department of Education, the National Science Foundation, and the Canadian Government. NCES had oversight of the collection, analysis and reporting of the U.S. data with direction from Eugene Owen. Dr. William Schmidt of Michigan State University is the TIMSS national research coordinator.

14. How does TIMSS assess student performance?

TIMSS is using data collection methodologies that go beyond those used in two previous international studies of mathematics and science achievement. Student achievement was measured through written tests that included multiple-choice questions and a substantial number of open-ended questions. In many of the countries, samples of students also were selected to engage in performance assessments (i.e., to design experiments, test hypotheses and record findings through a variety of hands-on activities).

In addition, students, teachers and administrators filled out questionnaires that solicited information on 1,500 topics such as student background, teacher instructional methods and a country's commitment of staff and materials to science and mathematics instruction. TIMSS also analyzed the curricula in participating countries through a review of textbooks, curriculum guides and other curricular materials.

Beyond these approaches, NCES designed two new methodologies that were carried out in the U.S., Germany and Japan. In the classroom videotape study, teachers in grade eight mathematics classes were taped to study classroom interactions. The tapes offer insights into the organization of lessons and instructional methods. In the ethnographic case study, researchers conducted in-depth interviews and observations with teachers, students, administrators and parents in these countries. The case studies focus on the implementation of national standards, teacher training and teachers' lives, the learning environment for adolescents, and methods for dealing with student ability differences.

15. Which students were included in the TIMSS assessment?

Schools were encouraged to have all students take the assessment. Students with limited English proficiency generally could be excluded if they had not been in an English speaking classroom for at least two years. Attempts were made to accommodate all learning disabled students depending on specific test taking provisions in their Individual Education Plans. The sample assessed in the United States was representative of all ethnic groups and all residence categories.

16. How many U.S. students took the TIMSS assessment?

Approximately 5,400 U.S. 12th-grade students in 210 public and private schools nationwide took the general knowledge assessments in mathematics and science. Approximately 2,500 students took the advanced mathematics assessment and another 2,500 students took the physics assessment.

17. When will TIMSS be offered again?

A TIMSS follow-up assessment will be given to 8th graders in the U.S. and other countries in 1999. The study will examine the performance of the same cohort of students who participated in TIMSS at the 4th grade level in 1995 four years later when they are in 8th grade. Trends in performance between 4th and 8th grade will be analyzed, and the 1999 scores will be compared to the performance of 8th graders in 1995. More than 20 countries (including Japan, Korea, and Singapore) have tentatively agreed to participate in the study, and preliminary discussions for another videotape study of teaching are underway.

18. What do the parentheses enclosing country names in the 12th grade Pursuing Excellence report mean?

Parentheses indicate that the country named had difficulty meeting the very high standards for conducting TIMSS. The nature of such difficulties are documented in the appendix of the report. The U.S. is in parentheses because its combined school and student participation rate was 64 percent, below the standard of 75 percent. These kinds of problems are typical for most assessments at this level in the U.S. It is most likely that as a group, schools and students who were selected for TIMSS but did not participate in the assessments in the U.S. would have had below average scores, thus lowering the U.S. average. This was probably the case in other countries as well. A full documentation of the data collection methodologies and statistical analyses used in all the participating countries is available in technical and quality control reports published by the TIMSS International Study Center at Boston College.

19. What general reports analyzing TIMSS data have been released to date?

There are over 30 different publications, videos, and CD-ROMS that have been released as part of the TIMSS study thus far. For a complete listing, see the web site at http://nces.ed.gov/timss. The following provide a good starting point for research into the TIMSS project, and can all be downloaded from the NCES web site:

• Pursuing Excellence: A Study of U.S. Fourth-Grade Mathematics and Science Achievement in International Context

• Pursuing Excellence: A Study of U.S. Eighth-Grade Mathematics and Science Teaching, Learning, Curriculum, and Achievement in International Context

• Pursuing Excellence: A Study of U.S. Twelfth-Grade Mathematics and Science Achievement in International Context

If you desire a comprehensive look at TIMSS and the tools to make its data meaningful in local reform efforts, you may be interested in the:

• Attaining Excellence: A TIMSS Resource Kit -- a package of 12 publications and 2 videos that helps educators, practitioners, policymakers, and concerned citizens reflect upon their own local practices in light of TIMSS findings.

20. Where can I get additional information on TIMSS?

Look at the web site at http://nces.ed.gov/timss or contact the Customer Service Line: Telephone: (202) 219-1333; E-mail: timss@ed.gov

-###-

[ Return to 12th-Grade Results page ]