A r c h i v e d  I n f o r m a t i o n

Designing Effective Development: Lessons from the Eisenhower Program - December 1999

Chapter 3

Teacher Outcomes

Section Findings

• Many teachers report that participation in Eisenhower-assisted activities enhanced their knowledge and skills. For example, almost half of the teachers participating in district Eisenhower activities report that participation enhanced their knowledge of mathematics and science; nearly 70 percent of teachers participating in IHE/NPO activities report enhanced knowledge of mathematics and science.

• Many teachers also report improvements in their classroom instruction as a consequence of participation in Eisenhower-assisted activities. For example, 55 percent of the teachers participating in district Eisenhower activities and 66 percent of teachers participating in IHE/NPO activities report increasing the cognitive challenge of classroom instruction.

The main goal of the Eisenhower Professional Development Program is to increase teachers? knowledge and skills in order to improve their teaching practice, which in turn will increase student learning and achievement. The law requires that states develop professional development plans designed to give teachers "the knowledge and skills necessary to provide all students the opportunity to meet challenging State content standards and challenging State student performance standards" (Section 2205(B)(2)(b)). Also it requires that LEAs use Eisenhower funds to "give teachers and administrators the knowledge and skills to provide students with the opportunity to meet challenging State or local content standards and student performance standards" (Section 2210(b)(1)).

In the following paragraphs, we examine the extent to which teachers participating in Eisenhower-assisted activities report outcomes consistent with these legislative purposes, focusing first on the extent to which teachers? knowledge and skills are enhanced as a results of participation, and then on improvements in teaching practice.

Teacher Knowledge and Skills

To assess the effects of participation on teachers? knowledge and skills, we asked each teacher in our national sample to indicate the degree to which his or her knowledge and skills were enhanced as a result of participation in the specific Eisenhower-assisted activity that drew the teacher into the sample. We asked each teacher to indicate the extent to which knowledge and skills had been enhanced in each of the following areas:

• Curriculum (e.g., units, texts, standards)

• Instructional methods

• Approaches to assessment

• Use of technology in instruction (e.g., computers, graphing calculators)

• Strategies for teaching diverse student populations (e.g., students with disabilities, from underrepresented populations, economically disadvantaged, limited English proficient, range of abilities)

• Deepening knowledge of mathematics

Teachers reported their responses using a five-point scale, where 1=not at all and 5=to a great extent. Our data, shown in Exhibit 3.15, indicate that teachers who participated in SAHE-grantee activities tended to report more change in their knowledge and skill than teachers who participated in district activities.²⁴ For example, 68 percent of teachers participating in SAHE-grantee activities reported that their participation led to improvements in their content knowledge in mathematics and science (responding with a four or five on the five-point scale) compared to 48 percent of teachers in district activities. Similarly, 64 percent of teachers in SAHE-grantee activities reported enhanced knowledge and skills in curriculum, and 79 percent in instructional methods, compared with 56 and 63 percent, respectively, for district activities. In addition, teachers in SAHE-grantee activities are substantially more likely to report enhanced knowledge in technology than teachers in district activities (50 percent versus 24 percent), and somewhat more likely to report enhanced knowledge in approaches to diversity (35 percent versus 26 percent).

This pattern differs somewhat for approaches to assessment: 48 percent of teachers in SAHE-grantee activities report enhanced knowledge and skills, as do 46 percent for teachers in district activities, a negligible difference. We are not sure how to explain the lack of difference between districts and SAHE grantees in this domain.

EXHIBIT 3.15
Percent of Teachers Reporting Enhanced Knowledge and Skills Due to Participation in Eisenhower-assisted Professional Development Activities (District n=731 to 750, SAHE Grantee n=233 to 240)

We averaged each teacher?s responses on these six items to create a composite scale measuring enhanced knowledge and skills. The results are displayed in Exhibit 3.16. One fact immediately visible in the exhibit is the substantial variation among teachers in reported enhancement in knowledge and skills. A few teachers have composite scores of one, meaning no change in knowledge and skills, while a few have composite scores of five, indicating knowledge and skills were enhanced to a great extent in all six domains measured. The average composite score for teachers participating in district activities is 3.2, while the average for teachers in SAHE-grantee activities is 3.6. Our results also indicate that the average enhancement of knowledge and skills is somewhat higher for teachers participating in reform activities than for teachers in traditional activities.

EXHIBIT 3.16
Extent to Which Participation in Eisenhower-assisted Professional Development Activities Enhanced Knowledge and Skills, as Reported by Teachers (District n=750, SAHE Grantee n=240)

Source: Mail Survey of Teachers Participating in Eisenhower-assisted Professional Development Activities, 1998. How to read this exhibit: The first distribution shows that, on average, those teachers who participated in district activities reported that the extent to which their knowledge and skills have been enhanced in six different areas was 3.2, where zero indicates no enhancement and five indicates great enhancement. Compared to traditional types of activities, reform types were significantly higher in terms of enhancement of knowledge and skills. Each dot represents one teacher. As the number of teachers at one data value increases, the dots form a horizontal line that increases in length. Each distribution represents the distribution of teachers for that particular category. The number to the right of the distribution is the mean. Note: "Districts" refers to teachers who participated in Eisenhower-assisted activities provided through the district component of the program. "SAHE Grantees" refers to teachers who participated in Eisenhower-assisted activities supported through the SAHE component of the program.

Change in Classroom Teaching Practice

We conducted a similar analysis of changes in teaching practice. We asked the teachers in our national sample to what extent they made changes in their teaching practices in each of the following domains, as a result of the professional development activity:²⁵

• The mathematics curriculum content

• The cognitive challenge of math classroom activities

• The instructional methods employed

• The types or mix of assessments used to evaluate students

• The ways technology (calculator or computer) is used in instruction

• The approaches taken to student diversity.

Teachers were asked to report responses on a scale from 0 to 3, where 0=no change, 1=minor change, 2=moderate change, and 3=significant change.

Exhibit 3.17 shows that the results are generally parallel to those for knowledge and skills. The exhibit displays the percent of teachers responding that they made moderate or significant changes in their teaching. In particular, in most domains, teachers in SAHE-grantee activities are more likely to report change in teaching practice than teachers in district activities. However, in one domain?approaches to assessment?teachers in district activities are more likely to report change in practice than teachers in SAHE-grantee activities.

EXHIBIT 3.17
Percent of Teachers Reporting Improvement in Classroom Teaching Practice Due to Participation in Eisenhower-assisted Professional Development Activities (District n=731 to 750, SAHE Grantee n=233 to 244)

We averaged each teacher?s responses to these six items to create a composite scale measuring change in teaching practice. The results appear in Exhibit 3.18. One notable pattern is that some teachers (about 17 percent in district activities and 10 percent in SAHE-grantee activities) report no change in teaching practice in any of the domains we measured. At the same time, some teachers report very high levels of change, with composite scores of 3 (the maximum value) on our scale.

EXHIBIT 3.18
Degree of Improvement in Classroom Teaching Practice Due to Participation in Eisenhower-assisted Professional Development Activities, as Reported by Teachers (District n=767, SAHE Grantee n=244)

Source: Mail Survey of Teachers Participating in Eisenhower-assisted Professional Development Activities, 1998. How to read this exhibit: The first distribution shows that, on average, those teachers who participated in district activities reported that the extent of their improvement in classroom teaching practice was 1.3, where zero indicates no change and three indicates the highest degree of change. Each dot represents one teacher. As the number of teachers at one data value increases, the dots form a horizontal line that increases in length. Each distribution represents the distribution of teachers for that particular category. The number to the right of the distribution is the mean. Note: "Districts" refers to teachers who participated in Eisenhower-assisted activities provided through the district component of the program. "SAHE Grantees" refers to teachers who participated in Eisenhower-assisted activities supported through the SAHE component of the program.

The average composite value for teachers in district activities is 1.3, indicating some change in teaching practice, while the average value for teachers in SAHE-grantee activities is 1.4. Although this difference of about 0.1 point on the composite scale appears small in numeric terms, it is statistically significant. (The standard deviation among district teachers for our change in teaching composite is about .8. This means that the difference between teachers in SAHE-grantee activities and teachers in district activities is about .125 standard deviations.) The difference between traditional and reform activities is somewhat greater. The average composite score is 1.4 for teachers in district reform activities and 1.2 for district traditional activities, a difference of 0.2 point. The parallel results for teachers in SAHE-grantee activities are 1.6 and 1.4, also a difference of 0.2 point.

Summary: Teacher Outcomes

Overall, our results indicate that a substantial portion of teachers in SAHE-grantee activities, and a smaller but still notable proportion of teachers in district activities, report that participation enhanced their knowledge and skills. For example, 68 percent of teachers participating in SAHE-grantee activities reported enhanced knowledge in mathematics and science, compared with 48 percent of teachers in district activities.

A large proportion of teachers in SAHE-grantee activities and a somewhat smaller proportion in district activities also report changes in teaching practice, as a result of participation. For example, 66 percent of teachers in SAHE-grantee activities and 55 percent of teachers in district activities report making changes in the cognitive challenge of classroom activities as a result of participation.

Our data indicate that teachers participating in reform types of activities, such as mentoring, coaching, and study groups, are somewhat more likely than teachers in traditional forms of professional development, such as workshops, to report enhanced knowledge and skills. They are also more likely to report change in teaching practice.

With respect to knowledge and skills, the effect of program component (SAHE grantee versus district) is larger than the effects of activity type (reform versus traditional). But, with respect to change in teaching practice, the effect of program component (SAHE grantee versus district) is smaller than the effect of activity type (reform versus traditional). This may be a consequence of the fact that SAHE-grantee activities are especially likely to focus on subject matter content, and thus they have a strong effect on teachers? reported change in knowledge and skills. Reform activities are especially likely to encourage coherent professional development, including alignment and professional communication among teachers, and these elements may be important supports for change. (This issue is discussed further in the following section.)

Indicator 1.1 Teachers' Skills and Classroom Instruction. By 1998, over 50 percent of a sample of teachers will show evidence that participation in Eisenhower assisted professional development has resulted in an improvement in their knowledge and skills, and by 2000, over 60 percent will show such evidence. By 1999, over 50 percent of a sample of teachers in selected sites will show evidence that participation in Eisenhower-assisted professional development has resulted in improved classroom instruction.

Our data can be used to assess Eisenhower professional development with respect to the Department of Education?s performance indicator 1.1 (see box), pertaining to change in teacher knowledge and skills. The indicator sets a standard of 50 percent of teachers reporting enhanced knowledge and skills as a result of participation. Our results indicate that district activities meet this standard for two domains of knowledge and skills (curriculum and instruction), but not the other four (in-depth knowledge in mathematics and science, approaches to assessment, uses of technology, and approaches to student diversity). SAHE-grantee activities meet the standard in four of the six domains (in-depth knowledge, curriculum, instruction, and uses of technology).

One way to assess the magnitude of the reported enhancement of knowledge and skills is to compare the results we obtained with the results for professional development activities that have been identified as exemplary. Data on enhancement of knowledge and skills somewhat parallel to ours are available for teachers participating in 34 exemplary summer institutes in mathematics and science, supported by the National Science Foundation, the Department of Education, and other agencies (Carey & Frechtling, 1997). A comparison of our data for SAHE-grantee activities with the results obtained for the 34 exemplary activities indicates that teachers participating in SAHE-grantee activities report enhancement of knowledge and skills in mathematics and science content roughly comparable to the results for the 34 exemplary activities. ²⁶ Teachers participating in district Eisenhower activities show somewhat weaker results than do teachers in the 34 exemplary activities.

²⁶ Carey and Frethling (1997) indicate that 44 percent of participants in outstanding teacher development programs reported that the programs enhanced their knowledge and understanding of science content to "a great extent" (value only 5 on their 5-point scale). If we isolate the percentage of participants in SAHE-grantee activities who reported that the activity enhanced their mathematics or science knowledge "to a great extent" (value of 5 on the 5-point scale), the percentage is 41 percent. The comparable percent for district activities is 24 percent.

-###-

[Core Features]

[Participation of Teachers from High-Poverty Schools]