Designing Effective Development: Lessons from the Eisenhower Program - December 1999
The primary goal of the Eisenhower Professional Development Program is to support professional development experiences for teachers that will enhance classroom teaching and, ultimately, improve student learning. Because improved teaching is critical to improved learning for students, it is a cornerstone of the standards movement. Therefore, this report begins by focusing on classroom teaching practice.
The purpose of this chapter is to lay a foundation for our evaluation of the Eisenhower program, by reviewing the current literature on teaching and learning in mathematics and science and by describing classroom teaching practices in the 30 in-depth case study schools. By drawing on the literature and data on the 30 case schools, we will characterize the strengths and weaknesses of current teaching practices in mathematics and science and identify areas in which further professional development should focus.
The data on classroom practice that we will report come from two sources: the baseline year of our three-year longitudinal survey of teacher change and classroom observations. In a subsequent report, we will use the second and third waves of the longitudinal survey, along with the data reported here, to examine the effects of participation in professional development on changes over time in teaching practice. The data we report in this chapter are thus the first step in our examination of the effect of Eisenhower-assisted activities on teaching practice. Exhibit 2.0 shows how the material covered in this chapter fits into the entire framework of the evaluation.
This chapter draws on two sources of data collected for the Longitudinal Study of Teacher Change: 1) a baseline survey of teachers in 30 schools and 2) observations of classes for a sample of teachers (two per school) who completed the surveys. 1 In addition, we conducted a content analysis of the National Assessment of Educational Progress, in order to compare the content of instruction that teachers reported in the survey to a national standards for content. In the paragraphs below, we describe each of these sources in turn.
To gather data on teachers? classroom teaching practices, we surveyed all teachers who teach mathematics or science in the 30 schools-one elementary, one middle, and one high school in each of the 10 in-depth case study districts. The baseline wave of the survey, which was conducted in the fall of 1997, asked teachers to describe their teaching during the 1996-97 year.2 In the survey, we asked teachers to select a year-long mathematics or science course to describe. We asked them to choose, if possible, a course they had taught in 1996-97, were continuing to teach in 1997-98, and expected to teach in 1998-99.
The survey contained two main sections concerning teaching practices in the selected course, one on the content taught and one on pedagogy. We discuss these two sections further when we present our results, in subsequent sections of the chapter.
Of the 575 teachers surveyed in the 30 in-depth case schools, 436 teachers (76 percent) responded.3 Some teachers who responded did not teach mathematics or science during the 1996-97 school year, either because they were not employed as teachers in 1996-97 or because they taught other subjects, and thus they are not included in the analyses of classroom teaching. In addition, we excluded some teachers from particular analyses because they did not complete all of the required items. The sample is 74 percent female and 18 percent minority. Six percent of the teachers in the sample are novice teachers, or teachers who have taught the surveyed subject for two or fewer years. (See Appendix C for a more complete description of the sample and response rates.)
Several features of the sample should be considered in interpreting our results. First, by design, the sample of 30 schools is disproportionately high poverty (50 percent of the sample schools are high poverty, compared to the national average of 25 percent). This feature of the sample is useful in an evaluation of the Eisenhower program, because the program targets teachers in high-poverty schools. Throughout the analyses, we tested whether differences between teachers in high- and low-poverty schools are statistically significant (at the .05 level); we note these findings only when they are significant. Second, we selected the districts and schools in the sample because they had adopted diverse approaches to professional development, in addition to traditional workshops and conferences. If such professional development is more effective than traditional approaches, then the teachers? instruction in the sample schools might be better than that of the average teacher. Finally, the Longitudinal Study of Teacher Change focuses on mathematics and science teachers because they are the primary participants in Eisenhower-assisted activities. For all of these reasons, the sample is not nationally representative, but neither is it extremely unusual.
As part of our site visits to the 30 in-depth case study schools, we conducted one-time classroom observations of two teachers in each school-usually one mathematics teacher and one science teacher. In conjunction with the observations, we conducted a brief pre-observation interview and a somewhat longer post-observation interview with each of the 60 teachers we observed. The teachers we observed were selected by the principals of the schools we visited, in part based on their availability at the time of our visit and willingness to be observed; participation in professional development was not a factor in selecting teachers to observe. Thus, the teachers we observed are not necessarily representative of all teachers in the study schools.
We conducted the observations using a structured protocol, designed in part to parallel our teacher survey instrument. Prior to conducting the observations, we conducted a training session in which our site visitors observed videotaped lessons and coded them using our observation protocol. This allowed the site visitors to develop a common understanding of the protocol and to check inter-rater reliability.
To report on the consistency of the content taught with high standards, we needed to identify an appropriate measure of high standards. The National Council of Teachers of Mathematics (NCTM) and National Research Council (NRC) standards set a framework for important mathematics and science concepts that should be taught in the classroom. However, these standards are at a level of generality that makes quantitative content analysis difficult; therefore, we look to the National Assessment of Educational Progress (NAEP) to make explicit the content focus of the standards. The NAEP provides items that reflect this framework and permit content analyses items to determine relative emphases for mathematics and science content. In order to develop a test that would be perceived as national, the National Center for Education Statistics has modeled the NAEP on the professional associations' standards (Reese et al., 1997). For example, 30 percent of the science assessment involves hands-on performance exercises and 50 percent involves open-ended questions (NAGB, 1997); these also are areas of emphasis for the standards. The high standards set by the test are evident in the scores reported for the 1996 science assessment; only three percent of students tested at the advanced level and 21 to 29 percent tested at or above the proficient level (Raizen, 1998). As "the nation's report card," the NAEP represents an appropriate standard, although admittedly not the only possible standard. Because the NAEP focuses on content and performance goals consistent with standards developed by national professional associations, and because the
NAEP establishes high expectations for achievement, it is reasonable to use the items on the NAEP tests as a measure of high instructional standards. 4 ,5
The rest of the chapter is organized in four sections. The first section reviews current standards and literature on effective content and pedagogy. The second section describes the content of instruction in the 30 in-depth schools, drawing on data from the baseline wave of the longitudinal survey and the classroom observations. The third section focuses on pedagogy in the 30 schools. Finally, in a brief concluding section, we draw together the implications of our analysis of teaching in the 30 schools.
2 The remaining two waves ask teachers to describe their teaching in the 1997-98 and 1998-99 years.
3 The response rate of high school teachers was higher than those of elementary and middle school teachers, in part because principals and department chairs in high school were more involved in administering the survey.
4 However, some performance goals for students--such as carrying out sustained work--cannot be adequately measured by a timed, paper-and-pencil test such as the NAEP.
5 Mathematics and science generally are tested in every other NAEP administration, or every four years. The data used for these analyses were the 1996 mathematics and science NAEP tests. See Appendix D for a description of the NAEP content analysis.
 [Overview of This Report] |
![[Table of Contents]](/icons/up.gif){kind=icon} |
 [Effective Content and Pedagogy] |