A r c h i v e d  I n f o r m a t i o n

Designing Effective Development: Lessons from the Eisenhower Program - December 1999

Chapter 3

Core Features

Section Findings

• Many teachers participating in Eisenhower-assisted activities, especially SAHE-grantee activities, report that the activities place a major emphasis on deepening their content knowledge in mathematics and science.

• Most Eisenhower-assisted activities incorporate some form of active learning-- for example, preparation of lesson plans and discussion of classroom implementation. Fewer activities involve other aspects of active learning--for example, observation of teaching and examination of student work.

• Eisenhower-assisted activities generally encourage a coherent process of teacher learning. Most teachers participating in Eisenhower-assisted activities report that the activities are consistent with their goals and aligned with state and district standards and assessments.

In the previous section, we focused on three features of the structure of professional development activities that establish the setting in which professional development takes place: the type of activity, its duration, and opportunities for collective participation. In this section, we turn to the substance of the core experiences provided as the professional development activity unfolds.

The reauthorizing legislation for the Eisenhower program emphasizes the intent to support activities that are of "high quality," but it does not spell out the particular characteristics that high-quality professional development should have. However, some elements of quality are enumerated in the rationale and supporting statements accompanying the legislation. For example, in referring to the literature on professional development, Section 2001(4)(A) states that:

professional development must be focused on teaching and learning in order to improve the opportunities of all students to achieve higher standards.

And Section 2001(4)(B), drawing on the literature, states that:

effective professional development focuses on discipline-based knowledge and effective subject-specific pedagogical skills, involves teams of teachers, and, where appropriate, administrators and pupil services personnel, in a school and, through professional networks of teachers, and, where appropriate, teacher educators, administrators, pupil services personnel, and parents, is interactive and collaborative, motivates by its intrinsic content and relationship to practice, builds on experience and learning-by-doing, and becomes incorporated into the everyday life of the school.

In addition, the explicitly stated purpose of the law is to ensure that teachers have access to professional development that "reflects recent research on teaching and learning" and "includes strong academic content and pedagogical components" (Section 2002(2)(B)(C)).

In addition to the law, we have drawn on the literature to specify the dimensions of quality. In the discussion that follows, we examine three features in particular that the literature suggests may be related to improvements in teacher knowledge and skills, teaching practices, and student achievement. The first dimension concerns the extent to which Eisenhower-assisted activities focus on strong academic content; the second concerns the extent to which activities include opportunities for teachers to become actively engaged in learning new skills; and the third concerns the extent to which Eisenhower-assisted activities form part of a coherent, meaningful program of professional development that is consistent with teachers? goals and related to teachers? work.

While these three dimensions are not specifically identified in the legislation, they are broadly consistent with the purposes of the law. In particular, the Congressional findings and supporting statements for the law clearly encourage professional development that focuses on content ("discipline-based knowledge and effective subject-specific pedagogy"), that encourages active learning ("builds on experience and learning by doing"), and that encourages a coherent program of professional development ("becomes incorporated in the everyday life of the school") (Section 2001(4)(B)).

In the following sections, we consider each of these three core features in turn, drawing on our survey and case-study data to assess the extent to which Eisenhower-assisted activities reflect these central features of quality professional development.

Focusing on Content

Although there is a large literature on professional development, surprisingly little attention has been given to the composition and characteristics of professional development activities. In particular, little research has been conducted on the relative efficacy of professional development activities that focus on different types of knowledge, skills, and teaching practices. ²⁰

The available descriptive research suggests that the content covered during professional development activities varies along at least four dimensions. First, activities vary in the relative emphasis they give to the content teachers are expected to teach and the teaching methods teachers are expected to employ. Some activities are intended primarily to improve teachers? knowledge of subject-matter content; some are designed to improve general pedagogy or teaching practices, such as classroom management, lesson planning, or grouping methods; and some are intended to improve what Shulman (1987) has termed "pedagogical content knowledge" - teaching practices in specific content domains, such as the teaching of multi-digit addition in elementary mathematics or forces and motion in physics.

Activities also vary in the specificity of the changes in teaching practice that are encouraged. Some activities focus on helping teachers use particular curriculum materials (e.g., new textbooks, science kits, or curriculum replacement units) or prescribed teaching strategies (e.g., specific student questioning strategies). Others focus on general principles, giving less attention to specific curricula or strategies. (See Kennedy, 1998, for a discussion of this distinction.)

In addition, activities vary in the goals for student learning emphasized. Some activities emphasize helping teachers improve student performance in the basic skills--for example, memorizing facts and mastering procedural skills, such as long division or solving linear equations in one unknown. Other activities focus on helping teachers improve students? conceptual understanding-for example, the ability to explain the reasons behind a solution strategy.²¹

Finally, activities vary in the emphasis they give to the ways students learn particular subject matter. Some activities give considerable emphasis to improving teachers? understanding of how children learn, by focusing, for example, on common student preconceptions, misconceptions, and solution strategies in specific subject domains. Other activities focus primarily on new curricula or teaching methods, while giving little attention to the ways students learn.

Although there is little evidence on the relative effectiveness of professional development activities that focus on different types of knowledge, skills, and teaching practices, a small literature has begun to emerge focusing on these issues. In particular, an emerging body of work suggests that professional development that focuses on subject-matter content and how children learn it may be an especially important element in changing teaching practice (e.g., Ball & Cohen, in press; Corcoran, 1995). In part, researchers base this argument on the fact that many teachers lack strong content-specific teaching skills. Reynolds, for example, in a review of the knowledge base for elementary school teachers, concluded that "beginning teachers have surprisingly few content-specific pedagogical understandings" (1995, p. 214). And Rhine (1998), in a discussion of the role of research in teaching, pointed out that "[r]eform-minded teachers are hungry for continuing education that provides novel ways to address content" (p. 27).

A number of authors argue that professional development requires a dual focus on both knowledge of subject matter content and an understanding of how children learn specific content. Hiebert et al. (1996), for example, argue that teaching for understanding in mathematics requires two forms of knowledge:

knowledge of the subject to select tasks that encourage students to wrestle with key ideas and knowledge of students? thinking to select tasks that link with students? experience and for which students can see the relevance of the ideas and skills they already possess (p. 16).

This point of view is bolstered by several recent studies of the effects of professional development on student achievement. Cohen and Hill (1998) conducted a study of mathematics teaching in California, based on data on teachers? professional development experiences and school-level data on student performance on a mathematics test administered statewide. They found that, controlling for the characteristics of students enrolled, average mathematics achievement was higher in schools in which teachers had participated in extensive professional development focusing on the teaching of specific mathematics content than in schools in which teachers had not. Participation in professional development focusing on general pedagogy, however, was not related to student achievement.

Kennedy (1998) found similar results in a review of well-designed experimental studies of the relationship between professional development and student achievement in mathematics and science, commissioned as part of this evaluation. The review found that, compared to more general professional development, professional development that focuses on specific content and how students learn has larger positive effects on student achievement outcomes, especially achievement in conceptual understanding.

Based on this emerging evidence, we view the degree of content focus as a central dimension of high-quality professional development. To examine the content focus of Eisenhower-assisted activities, we asked each teacher in our national sample to indicate the degree of emphasis the activity in which the teacher participated gave to deepening content knowledge in mathematics and science, using a three-point scale (no emphasis=zero, minor emphasis=one, major emphasis=two).²²

The results are reported in Exhibit 3.5. As can be seen, many teachers in both district and SAHE-grantee activities report a major emphasis on content in the activities in which they participated, although teachers in district activities report somewhat less focus on content than teachers in SAHE-grantee activities. Overall, 51 percent of teachers in district activities report a major emphasis on content and 35 percent report a minor emphasis. For SAHE-grantee activities, 68 percent of teachers report a major emphasis and 28 percent report a minor emphasis. There is no significant difference in content focus between traditional and reform activities.

EXHIBIT 3.5
Percent of Teachers Reporting a Major Emphasis on Mathematics and Science Content Knowledge in Eisenhower-assisted Professional Development Activities (District n=754, SAHE Grantee n=243)

Source: Mail Survey of Teachers Participating in Eisenhower-assisted Professional Development Activities, 1998. How to read this exhibit: The first distribution shows that 51 percent of the teachers who participated in district activities report that a major emphasis was put on content knowledge in their activity. Each dot represents one teacher. As the number of teachers at one data point (or value) increases, the dots form a horizontal line that increases in length. Each distribution represents the distribution of teachers for that particular category. The number on the top of the distribution is the percent of teachers responding to the category. Note: "Districts" refers to teachers who participated in Eisenhower-assisted activities provided through the district component of the program. "SAHE Grantees" refers to teachers who participated in Eisenhower-assisted activities supported through the SAHE component of the program.

Although our survey data do not tell us whether the content emphasized is of high quality, our data do indicate that a substantial proportion of Eisenhower-assisted activities are tied to subject-matter content. Given the growing literature and recent research emphasizing the potential relationship between content focus and student achievement, these results are encouraging.

Promoting Active Learning

A second core feature of professional development concerns the opportunities provided for teachers to become actively engaged in meaningful discussion, planning, and practice as part of the professional development activity. (See, for example, Lieberman, 1996; and Loucks-Horsley, Hewson, Love, & Stiles, 1998.) Opportunities for active learning can take a number of forms, including the opportunity to observe expert teachers and to be observed teaching; the opportunity to plan how new curriculum materials and new teaching methods will be used in the classroom; the opportunity to review student work in the topic areas being covered; and the opportunity to lead discussions and engage in written work.

Lieberman (1996) argues that, for teachers to change their teaching, they must have the opportunity to "talk, think, try out, and hone new practices" (p. 189). Darling-Hammond (1997b) has characterized professional development with characteristics such as "rich professional development." According to Darling-Hammond, rich professional development:

is centered around the critical activities of teaching and learning—planning lessons, evaluating student work, developing curriculum—rather than around abstractions and generalities; it grows from investigations of practice through cases, questions, analysis, and criticism; and it is built on substantial professional discourse that fosters analysis and communication about practice and values in ways that build collegiality and standards of practice [emphasis in original] .

The importance of engaging teachers in concrete tasks as a part of professional development is given some support in a recent study of 34 exemplary professional development activities in mathematics and science, supported by the National Science Foundation, the Department of Education, and other federal agencies (Carey & Frechtling, 1997). As part of an evaluation of these activities, teachers were surveyed about the degree to which the activities included the opportunity to plan how the concepts and materials would be used in the classroom, as well as the opportunity to engage in hands-on activities, challenging problem solving, and the development of curriculum units. Carey and Frechtling indicate that teachers who participated in activities that offered these opportunities reported more change in knowledge, skills, and teaching practices than teachers in activities that did not afford these opportunities.

In the following paragraphs, we examine the degree to which opportunities of this kind are provided in our national sample of Eisenhower-assisted activities. Although we do not have information on the extent to which a particular strategy was used in an activity nor whether it was used in conjunction with conceptual, in-depth learning, we do have information on the prevalence of several types of learning strategies. We focus in particular on four dimensions of active learning: observing and being observed teaching; planning for classroom implementation; reviewing student work; and presenting, leading, and writing.

Observing and Being Observed

One element of active learning is the opportunity for teachers to observe expert teachers and be observed teaching in their own classroom and obtain feedback. These opportunities can take a variety of forms. Videotaped lessons, for example, can offer participants an opportunity to observe lesson content, teaching strategies, and student learning. Teachers can visit each others? classrooms to observe lessons. Activity leaders, lead teachers, mentors, and coaches can observe classroom teachers and engage in reflective discussions about the goals of a lesson, the tasks employed, teaching strategies, and student learning.

Eisenhower funds were used to support in-depth opportunities for observation in West City, Connecticut, one of our exploratory case sites. As part of the portfolio of activities offered with Eisenhower support in West City, two skilled in-house facilitators provide coaching and support for elementary teachers, including demonstration lessons and help with planning. Teachers who receive intensive coaching generally spend one to two hours per week in pre-lesson discussion, demonstration lessons, and post-lesson feedback.

To determine the frequency with which opportunities of this kind are provided as part of Eisenhower-assisted activities, we asked each teacher in our national sample how the activity helped the teacher use new skills in the classroom. In particular, we asked each teacher whether the teacher received coaching or mentoring in the classroom as part of the Eisenhower-assisted activity; whether the teacher?s teaching was observed by the activity leader(s) and feedback was provided; and whether the teacher?s teaching was observed by other participants and feedback was provided. We also asked whether the activity was evaluated in part based on an observation of the teacher?s classroom.

The results, shown in Exhibit 3.6, indicate that relatively few teachers report opportunities to observe and be observed as part of Eisenhower-assisted activities, but, overall, SAHE-grantee activities provide more opportunities for classroom observation than do district activities. For example, 22 percent of teachers in SAHE-grantee activities report being coached in the classroom as part of the activities in which they participated, compared with 10 percent of teachers in district activities. Similar differences appear in the percentage of teachers who report being observed by the leader (16 versus five percent), who report being observed by other teachers (15 versus 10 percent), and who report that the activity was evaluated in part based on observation (22 versus five percent).

EXHIBIT 3.6
Percent of Teachers Reporting That They Had Opportunities to Observe or Be Observed Teaching in Eisenhower-assisted Professional Development Activities
(District n=783, SAHE Grantee n=244)

Planning Classroom Implementation

A second element of active learning involves the opportunity to link the ideas introduced during professional development experiences to the teaching context in which teachers work. The introduction of new approaches may have different implications depending on the curriculum in place in a teacher?s school, the specific textbooks adopted in the teachers? classrooms, and the required assessments in the teachers? districts, as well as the characteristics of the students enrolled in the teachers? classrooms, including the material covered in previous grades and students? expectations for classroom instruction.

To assess the extent to which Eisenhower activities provided teachers with opportunities to plan classroom implementation, we asked each teacher in our national sample whether, as part of the activity in which the teacher participated, the teacher practiced under simulated conditions, with feedback; met formally with other activity participants to discuss classroom implementation; communicated with the leader(s) of the activity concerning classroom implementation; met informally with other participants to discuss classroom implementation; and developed curricula or lesson plans that other participants or the activity leader reviewed.

Exhibit 3.7 shows that opportunities to plan classroom implementation are a more common part of Eisenhower-assisted activities than opportunities to observe and be observed teaching. But, as in our results for observation, SAHE-grantee activities are more likely to provide opportunities for planning than are district activities. For example, 32 percent of teachers participating in district Eisenhower-assisted activities report that the activities in which they participated provided the opportunity to discuss classroom implementation, as do 51 percent of teachers in SAHE-grantee activities. Similarly, 36 percent of teachers in district activities report that the activity provided an opportunity to communicate with the leader about classroom implementation, as do 54 percent of teachers in IHE/NPO activities.

EXHIBIT 3.7
Percent of Teachers Reporting Opportunities to Plan Classroom Implementation in Eisenhower-assisted Professional Development Activities
(District n=783, SAHE Grantee n=244)

Reviewing Student Work

Another element of active learning is the opportunity to examine and review student work. By examining students? written responses to problems, for example, teachers may gain an understanding of students? preconceptions, misconceptions, and solution strategies (Schifter, 1996; Carpenter et al., 1989). And, by observing students? responses to questions posed by expert teachers, teachers may gain insight into students? assumptions and reasoning. Well-presented examples of student work can provide insight into the types of difficulties students are likely to have with complex subject matter and the materials and teaching strategies that are likely to be effective. In particular, examples of student work may help teachers develop skills in diagnosing student problems and designing lessons at an appropriate level of difficulty. Finally, by discussing examples of students? work, teachers may develop shared interpretations of the kinds of problems that are easy and difficult and the standards of performance against which students should be held.

To determine the extent to which opportunities to review student work are provided as part of Eisenhower-assisted activities, we asked each teacher in our national sample whether the teacher reviewed student work or scored assessments as part of the activity; we asked whether work completed by students in the teacher?s classroom were reviewed by other activity participants or the activity leader; and we asked whether student outcomes were examined as part of an evaluation of the activity.

The results, shown in Exhibit 3.8, indicate that opportunities to review student work are relatively rare, although they are somewhat more common in SAHE grantee than district activities. For example, 19 percent of teachers in district Eisenhower activities reported that the activities provided an opportunity to review student work, as did 26 percent of teachers in SAHE-grantee activities; and nine percent of teachers in district activities scored assessments, as did 13 percent of teachers in IHE/NPO activities.

Presenting, Leading, and Writing

Apart from opportunities to observe teaching, plan classroom implementation, and review student work, professional development activities may also offer teachers the opportunity to give presentations, lead discussions, and produce written work. Active participation of this kind may improve outcomes by permitting teachers to delve more deeply into the substantive issues introduced.

To examine the degree to which these forms of active participation are provided as part of Eisenhower-assisted activities, we asked each teacher in our national sample whether, as part of the activity, the teacher gave a lecture or presentation; conducted a demonstration of a lesson, unit, or skill; led a whole-group discussion; led a small group discussion; or wrote a paper, report, or plan.

The results, shown in Exhibit 3.9, indicate that three of these forms of active learning-giving a lecture or presentation, conducting a demonstration, and writing a report-are relatively common, at least among SAHE-grantee activities. Overall, 42 percent of teachers in SAHE-grantee activities report giving a lecture or presentation as part of the activity in which they participated, 43 percent conducted a demonstration, and 44 percent wrote a paper or report. Fewer teachers who participated in district activities report these kinds of opportunities: 18 percent of teachers in district Eisenhower-assisted activities report giving a lecture or presentation, 24 report conducting a demonstration, and 15 percent report writing a paper.

Relatively few teachers in either district or SAHE-grantee activities led whole-group or small-group discussions, although discussion is often emphasized as a strategy to be encouraged in the teaching of mathematics and science.

EXHIBIT 3.8
Percent of Teachers Reporting Opportunities to Examine Student Work in Eisenhower-assisted Professional Development Activities (District n=783, SAHE Grantee n=244)

EXHIBIT 3.9
Percent of Teachers Reporting Opportunities to Present, Lead, and Write in Eisenhower-assisted Professional Development Activities
(District n=783, SAHE Grantee n=244)

Overall Index of Active Learning

In the previous sections, we have focused on the extent to which Eisenhower-assisted activities provide specific types of opportunities—for example, the opportunity to be observed teaching by the activity leader, score student assessments, develop a lesson plan, or write a report. The description we have provided of the frequency with which each of these types of opportunities is offered provides one way of assessing the degree to which Eisenhower-assisted activities support active learning.

A complementary approach to examining active learning is to ask how many different types of active learning opportunities Eisenhower-assisted activities generally provide. While it is not clear that a particular number or combination of active learning strategies is necessarily more effective, activities that support multiple forms of active learning may be able to integrate these opportunities to provide strong supports for teacher change.

An Eisenhower-assisted activity provided by a SAHE-sponsored IHE in Texas, one of our in-depth case states, illustrates the ways a varied set of approaches for active learning can build upon one another to facilitate teacher development. Nineteen teachers participated in the 6^th-Grade Physical Science Enhancement and Math Integration program offered by the IHE. The program was designed to create a support system to help teachers implement teaching practices consistent with the national standards in mathematics and science and the Texas Essential Knowledge and Skills. During a six-week summer institute, teachers kept journals, model classrooms were used to help teachers learn new strategies, and instructors videotaped teachers in the classroom and provided feedback. In addition, the project encouraged participants to form a network to provide support during the school year. Finally, the activity leaders spent about 200 hours in the teachers? classrooms during the following school year, observing and providing follow-up and guidance.

To examine the number of types of active learning opportunities of this kind provided in the activities in which our national sample of teachers participated, we created a composite index, summing all of the types shown in Exhibits 3.6, 3.7, 3.8, and 3.9. Because our survey included four items to measure opportunities for observation, five for planning, four for reviewing student work, and five for presenting/writing, simply summing the 18 types of opportunities included would give more weight to planning and presenting/writing than to observing and reviewing student work. Thus, in computing the index, we weighted each of the four items pertaining to observation and the four items pertaining to student work by 1.25. This produces an index that runs from zero (no opportunities were provided for active learning) to 20 (all types of active learning were provided).

The results, shown in Exhibit 3.10, indicate that activities vary widely in the number of opportunities they provide for active learning, but, overall, SAHE-grantee activities provide more opportunities for active learning than district activities. On the average, teachers report that district activities provide 3.6 types of active learning (of the maximum of 20 possible on the scale), but the range is enormous. About 21 percent of teachers are in district activities that provide no opportunities for active learning, while some are in activities providing more than eight types. On the average, teachers in SAHE-grantee activities report 5.8 types of active learning, although, as for districts, there is a wide range. About nine percent of teachers in SAHE-grantee activities report no opportunities for active learning.

As anticipated, reform activities tend to offer more opportunities for active learning than traditional activities, for activities supported through the district component of the program. On average, teachers participating in district reform activities report 4.4 types of active learning, compared with 3.4 types for traditional activities. There is little difference in opportunities for active learning between traditional and reform activities supported through the SAHE component of the program.

Our data on active learning opportunities suggest that some Eisenhower-assisted activities-particularly those supported in the SAHE component of the program--provide a diverse set of active learning activities. But many Eisenhower-assisted activities--especially traditional activities supported under the district component of the program-do not yet incorporate the kinds of active learning opportunities the literature suggests are required to support ongoing improvement in teacher knowledge, skills, and teaching practice. Thus, this is an area in which further program improvement efforts might focus.

EXHIBIT 3.10
Number of Opportunities for Active Learning in Eisenhower-assisted Professional Development Activities, as Reported by Teachers
(District n=767, SAHE Grantee n=244)

Nurturing Coherence

The third core feature of professional development concerns the extent to which professional development activities are perceived by teachers to be a part of a coherent program of teacher learning. Up until this point, in examining the quality of professional development activities supported by the Eisenhower program, we have focused on experiences that occur as part of the activity itself-the content covered and the opportunities for active learning provided. To some extent, however, the quality of a professional development experience is also a function of the relationship of the experience itself to other aspects of a teacher?s work (Darling-Hammond & McLaughlin, 1996). Developing coherent knowledge-based visions are important conditions for learning (McCarthey & Peterson, 1993). Professional development for teachers is frequently criticized on the ground that the activities are disconnected from one-another-in other words, individual activities do not form part of a coherent program of teacher learning and development. A professional development activity is more likely to be effective in improving teachers? knowledge and skills if it forms a coherent part of a wider set of opportunities for teacher learning and development.

The coherence of a teacher?s professional development experience can be assessed in at least three ways. First, coherence can be judged in terms of the degree to which an individual professional development activity builds on what teachers have already learned and is followed up with other, more advanced work. The coherence of an activity can also be judged in terms of the extent to which the activity emphasizes content and pedagogy aligned with national, state, and local standards, frameworks, and assessments. Finally, the coherence of an activity can be judged on the basis of the extent to which the activity supports teachers in developing sustained, ongoing professional communication with other teachers who are trying to change their teaching in similar ways.

In the following sections, we consider each of these aspects of coherence in turn.

Connections with Goals and Other Activities

One way to assess whether a professional development activity is part of a coherent program of teacher learning is to ask whether the activity builds on earlier activities and is followed up with later, more advanced work. Are the individual professional development opportunities provided for teachers linked so they build upon one another? Do professional development activities make it possible for teachers to advance from more basic to more advanced topics and skills, covered in greater depth? Or, do professional development opportunities tend to repeat content from earlier activities, at the same depth?²³

Furthermore, are the opportunities provided linked to teachers? goals for development? Do teachers perceive that the available activities are central to the issues of teaching and learning that they believe need more attention?

To address these issues, we asked each teacher in our national sample to report the extent to which the activity the teacher attended was consistent with the teacher?s goals for professional development; based explicitly on what the teacher had learned in earlier professional development experiences; and followed up with activities that built upon what was learned in this professional development activity. Teachers responded on a scale from one to five, where one=not at all and five=to a great extent.

The results, shown in Exhibit 3.11, indicate that almost all teachers believe the Eisenhower-assisted activities in which they have participated are consistent with their goals: 79 percent of teachers in district activities report that the activities in which they participated were quite consistent with their goals (that is, they reported a value of four or five on the five-point response scale); and 88 percent of teachers in SAHE-grantee activities reported that the activities were consistent with their goals.

Many fewer teachers report that the Eisenhower-assisted activities build upon earlier activities: 35 percent of teachers in district activities and 39 percent of teachers in SAHE-grantee activities report that the activity built on earlier activities.

Somewhat more teachers report that the Eisenhower-assisted activities in which they participated were followed up with additional activities than reported that Eisenhower-assisted activities built upon earlier work. About 53 percent of teachers in district activities report that they were followed up with more advance work, as did 70 percent of teachers in SAHE-grantee activities.

EXHIBIT 3.11
Percent of Teachers Reporting That Eisenhower-assisted Professional Development Activities are Related to Their Other Professional Development Experiences (District n=748 to 760, SAHE Grantee n=238 to 239)

Alignment with State and District Standards and Assessments

A second aspect of coherence concerns the alignment of the content and pedagogy emphasized in the activities with national, state, and local frameworks, standards, and assessments. Teachers receive guidance about what to teach and how to teach it from multiple sources, including the material covered in formal professional development, as well as in their pre-service education, textbooks, national standards, state and local policies and assessments, and the professional literature (Cohen & Spillane, 1992). To the extent that these sources provide a coherent set of goals for teaching and learning, they may facilitate teachers? efforts to improve their teaching practices. To the extent that they conflict, however, the sources of guidance may create tensions that impede teachers? efforts to develop their teaching in a consistent direction (Grant, Peterson, & Shojgreen-Downer, 1996).

Efforts to align professional development with state and district frameworks, standards, and assessments offer one approach to increasing the coherence of the instructional guidance teachers receive. The process of aligning professional development with state and district standards and other policies can take a number of forms. For example, professional development activities can be chosen to reflect the topics emphasized in state and district standards. Or, professional development activities can focus on the goals for student learning emphasized in state assessments or the pedagogical methods emphasized in state curriculum frameworks (Webb, 1998).

A professional development activity in Riverside, Washington, one of our case-study districts, illustrates the use of Eisenhower funds to encourage professional development that is aligned with standards. The Riverside Elementary Science Kit project is an ongoing process that started in 1990, when teachers began attending a series of extended workshops on science concepts. After several years of related workshops, building from simple to more advanced topics, between 30 and 50 teachers were prepared and ready to begin creating kits They began creating three to four kits in each grade level: one concept kit, and two or three content kits per grade.

Thirty-one kits have now been completed, focusing on earth, life, and physical science. Currently, Eisenhower-assisted workshops are being used to develop standards for the scoring of the student work included in the kits. These workshops are designed to help match state and national learning standards to the scoring methods used for the kits. After completing the workshops, the teachers are expected to know which state and national standards the science kits satisfy, and they should be able to understand the goals for student learning embodied in each kit.

To obtain a measure of the alignment of Eisenhower-supported professional development with state and district standards, we asked each teacher in our national sample to indicate the extent to which the activity in which the teacher participated was aligned with state or district standards and curriculum frameworks, and with state and district assessments. Teachers were asked to respond using a five-point scale, from 1=not aligned at all to 5=aligned to a great extent.

The results, displayed in Exhibit 3.12 show that most teachers report that the activities they participated in were aligned with state and district standards, frameworks, and assessments, although teachers report somewhat more alignment between professional development with standards and frameworks than assessments. Overall, 80 percent of teachers participating in Eisenhower-assisted activities report that the activities were well aligned with state and district standards (reporting a four or five on the five-point scale), while 68 percent report that the activities were well aligned with state and district assessments. The results for SAHE-grantee activities are nearly identical. Although our data do not permit us to judge the depth of the alignment reported in our surveys, our results do indicate that teachers perceive that the activities in which they participate are quite consistent with state and district standards and frameworks, and moderately consistent with state and district assessments.

EXHIBIT 3.12
Percent of Teachers Reporting That Their Eisenhower-assisted Activities are Aligned with State and District Standards, Frameworks, and Assessments (District n=748 to 753, SAHE Grantee n=238)

Communication with Others

The third dimension of coherence concerns the ways in which professional development activities encourage professional communication among teachers who are engaged in efforts to reform their teaching in similar ways. Efforts to change teaching practices generally engender problems, challenges, and dilemmas as teachers attempt to carry out in the classroom ideas that were introduced in professional development activities. An ongoing discussion among teachers who confront similar issues can facilitate change by encouraging the sharing of solutions to problems, as well as by reinforcing the sense that, with time, improvement is possible. There is some evidence, for example, that networks of teachers involved in change can help sustain motivation (Lieberman & McLaughlin, 1992). In addition, by sharing methods, discussing written work, and reflecting on problems and solutions, teachers may foster a better understanding of the goals for student learning that proposed changes in teaching imply. Communication with other teachers is another dimension of coherence.

To measure the extent to which teachers in our national sample were encouraged to establish professional communication as part of the Eisenhower-assisted activities in which they participated, we asked the teachers whether they had discussed what they learned with other teachers in their school or department who did not attend the activity; whether they had discussed or shared what they learned with administrators (e.g., principal or department chair); and whether they had communicated, outside of formal meetings held as part of the activity, with participants in the activity who teach in other schools.

The results are shown in Exhibit 3.13. The data indicate that most teachers report discussing what they learned with other teachers in their school who did not attend the professional development activity, and most also discuss what they learned with school administrators. Somewhat fewer teachers, however, report communicating with teachers in other schools.

EXHIBIT 3.13
Percent of Teachers Reporting That They Communicate with Other Teachers About Their Experiences in Eisenhower-assisted Professional Development Activities (District n=783, SAHE Grantee n=244)

Overall Index of Coherence

To provide a composite measure of the overall extent to which Eisenhower-assisted activities are a part of a coherent program of professional development, we combined the items that comprise our three specific dimensions of coherence. The composite sums the items shown in Exhibits 3.11, concerning connections to teachers? goals and other professional development experiences; Exhibit 3.12, concerning alignment; and Exhibit 3.13, concerning professional communication. Because three items are available for the first and third of these dimensions, while only two items are available for the second, we weighted the items for the second dimension by 1.5. This produces a scale that runs from zero (the activity did not include any of the types of coherence we measured) to nine (the activity provided all of the forms we measured).

The results, displayed in Exhibit 3.14, indicate that both district and SAHE-grantee activities incorporate many of the features of coherence we measured, although the variation across activities is substantial. The average number of types of coherence reported for district activities is 6.0 (on our 0-9 scale), and the average for SAHE-grantee activities is 6.4. Reform activities incorporate somewhat more forms of coherence than traditional activities (6.3 versus 5.9 for districts, and 7.0 versus 6.2 for SAHE grantees).

EXHIBIT 3.14
Degree to Which Eisenhower-assisted Professional Development Activities Foster Coherence, as Reported by Teachers (District n=747, SAHE Grantee n=235)

These results indicate that most teachers perceive Eisenhower-assisted activities to be integrated in a variety of ways into their professional lives. This clearly suggests that progress has been made since the previous Eisenhower evaluation, when Eisenhower-assisted activities were described as "one-shot" (Knapp et al., 1991). Given the survey data we have available, we cannot be sure how deeply integrated Eisenhower-assisted activities are into the fabric of teachers? professional work, but we can conclude that many activities are fostering important linkages with teachers? other professional development, state and district standards, and other teachers.

In addition, as we had anticipated, reform activities are more likely to foster coherence than traditional activities, such as workshops and institutes. Because reform types of activities, such as mentoring, coaching, and study groups, frequently take place during the regular school day, it is undoubtedly easier for such activities to foster communication among teachers. The fact that reform activities are tied to the daily life of the school may also help such activities foster other aspects of coherence-particularly alignment with standards.

Summary: Core Features

Our results on the core features of Eisenhower-assisted professional development activities provide a mixed picture. We examined three core dimensions of the professional development activity: content focus, active learning, and coherence. With respect to the first, most teachers participating in Eisenhower-assisted activities report that the activities in which they participated placed a major emphasis on deepening their content knowledge in mathematics and science. This is quite encouraging, given the conclusions in recent literature on the potential benefits of professional development focused on specific subject-matter content (Kennedy, 1998; Cohen & Hill, 1998).

With respect to opportunities for active learning, the results appear less positive. While most teachers report that the Eisenhower-assisted activities in which they participated provide an opportunity to plan classroom implementation and to give presentations or demonstrations, relatively few provide an opportunity for teachers to observe or be observed or to examine student work. There is a growing body of research to suggest that these forms of active learning are critical components of effective professional development (Schifter, 1996; Carpenter et al., 1989). If so, Eisenhower-assisted activities do not completely meet standards of high quality in this respect.

Finally, with respect to coherence, our results are reasonably positive. Most teachers report that the activities in which they participated were connected to their goals for professional development and other professional learning opportunities, aligned with state and district standards and assessments, and led to professional communication with other teachers.

Our results also support two other conclusions. First, teachers participating in activities supported through the SAHE component of the program are more likely to report that the activities emphasized mathematics and science content, have opportunities for active learning, and more aspects of coherence than teachers participating in district activities. As we will discuss later in this chapter, these differences can be explained in part by the longer duration of SAHE-grantee activities.

In addition, our results indicate that reform activities are more likely than traditional activities to provide active learning opportunities and more likely to encourage coherence. The differences between reform and traditional types of professional activities in opportunities for active learning are not as large as the differences between SAHE-grantee and district activities, however, and, in relation to the overall variation among activities, are only moderate in size. The reform-traditional differences in coherence are larger, however, suggesting that reform activities are especially valuable in encouraging integration of professional development with other aspects of teachers? work.

Indicator 2.1 District-level Professional Development By 1998, over 50 percent of teachers participating in district-level or higher education Eisenhower-assisted professional development will participate in activities that are aligned with high standards. By 2000, over 75 percent will. Indicator 3.1 High Quality By 1998, over 50 percent of teachers participating in district-level, Eisenhower-assisted professional development activities will participate in activities reflecting best practices, including a focus on continuous improvement. By 2000, over 75 percent will.

Our results provide evidence that can be used to assess the Eisenhower program with respect to two of the Department of Education?s Performance Indicators for the program (see box). The first indicator, pertaining to alignment, sets as a standard that over 50 percent of teachers participating in Eisenhower-assisted activities will participate in activities that are aligned with high standards. As shown in Exhibit 3.12, 80 percent of teachers participating in Eisenhower-assisted activities report that the activities were designed to support state and district standards—far more than the standard of 50 percent. We do not know the depth of alignment this represents, nor whether the state and district standards with which Eisenhower activities are aligned are truly "high." But these results are encouraging.

The second indicator, pertaining to quality, sets as a standard that over 50 percent of teachers participating in Eisenhower-assisted professional development will participate in activities that are of high quality. Here, we have multiple measures that can be used. Our data on content focus, the first core feature, indicate that 51 percent of teachers participating in Eisenhower-assisted activities participate in activities that place a major emphasis on content. By this measure, Eisenhower activities meet the standard set in the indicator. Our data on the active learning, the second core feature, are more mixed; fewer than 50 percent of teachers are in activities providing some of the key dimensions of active learning, such as the opportunity to observe and be observed, or the opportunity to review student work (see Exhibits 3.7 and 3.8). While more teachers report opportunities to plan classroom implementation and to conduct presentations and demonstrations, less than half of teachers in district activities report most of these experiences. Finally, our data on coherence, the third core feature, are reasonably positive. More than 50 percent of teachers participating in district activities report that the activities in which they participated are consistent with their goals were followed up with later experiences, were aligned, and nurtured professional communication.

In the next section, we turn our attention from the core features of Eisenhower-assisted activities to their reported effects on teacher knowledge, skills, and teaching practices.

²³ To some extent, the issues here are parallel to those posed in assessing the learning opportunitites provided for students.

-###-

[Structural Features]

[Teacher Outcomes]