A r c h i v e d  I n f o r m a t i o n

Designing Effective Development: Lessons from the Eisenhower Program - December 1999

Chapter 6

Building a Vision for Professional Development: Alignment with Standards and Assessments, and Coordination with Other Programs

Section Findings

• Most teachers in SAHE-grantee projects are in projects whose directors say that state standards play a role in the design of their Eisenhower professional development; however, state assessments and district standards and assessments are much less likely to play a role in project design. Projects in nonresearch universities are better aligned with state and district standards and assessments than projects in research universities.

• Few participating teachers are in projects that have ongoing feedback mechanisms with districts, work with the district Eisenhower coordinators, or co-fund with federal programs. However, most participating teachers are in projects that work with districts in other ways. These include working closely with staff from other federal programs, communicating with district staff, and relying on district needs assessments to plan the professional development project.

• IHE projects housed in education departments engage in significantly more types of coordination with districts than do projects in mathematics/science departments.

State and district standards and assessments provide a vehicle for unifying reforms and professional development. Thus one method of designing and developing a program of professional development is to base the activities, pedagogy and curriculum on standards or assessments adopted by the state or district, and to work with other programs in the state and district to develop a coherent reform strategy

In the law, Congress intended that SAHE grantees be part of system-wide reform efforts. The law requires SEAs to develop a state professional development plan to improve teaching and learning and to develop the plan in conjunction with SAHEs (Section 2205 (b)(2)(A)). Similarly, SAHE grantees must follow the law?s general provisions for alignment with state standards. Specifically, SAHE-sponsored professional development should help to ensure that professional development is linked to state content and performance standards (Section 2002(2)(A)).

In addition to general provisions about alignment, the Eisenhower program specifically describes particular coordination requirements for IHE/NPO projects. While the SAHE component of the program operates separately from the LEA component of the program, IHE/NPOs that receive Eisenhower grants are expected to work in conjunction with LEAs. Specifically, the legislation provides that

?the State agency for higher education, working in conjunction with the State educational agency (if such agencies are separate), shall make grants to, or enter into contracts or cooperative agreements with, institutions of higher education and nonprofit organizations of demonstrated effectiveness, including museums and educational partnership organizations, which must work in conjunction with a local educational agency, consortium of local educational agencies, or schools?(Section 2211(a)(1)).

In fact, the Eisenhower legislation requires that SAHE grantees establish a formal relationship with one or more school districts. It states that

No institution of higher education may receive assistance under (a)(1) of this subsection unless the institution enters into an agreement with a local educational agency, or consortium of such agencies, to provide sustained, high-quality professional development for the elementary and secondary school teachers in the schools of each such agency (Section 2211(a)(3)).

The importance of alignment and coordination also is emphasized in the literature. Linking professional development with other programs and reforms helps to provide a coherent vision of professional development (Elmore & Burney, 1996; Guskey, 1997), and teachers recognize when there are inconsistencies among these factors (Grant, Peterson, & Shojgreen-Downer, 1996). Researchers have emphasized the important role that standards and assessments can play in designing and implementing professional development and how coordination, in the form of co-funding, can bring coherence to a professional development plan (Elmore & Burney, 1996). While this literature focuses on school districts, SAHE grantees also can be a part of the overall vision if they are aligned and coordinated with the districts.

Although SAHEs are not in the same position as districts are to build professional development as part of a systemic reform strategy, they can establish their own vision for integrating their grantee activities into district and state reform strategies, and in this way can be part of the district and state?s vision for reform. The extent to which SAHE-grantee projects link their professional development design, implementation and management with district and state reforms can be measured by the projects? 1) alignment with state and district standards and assessments, and 2) extent of coordination with the district and with other federal programs.

Alignment of Eisenhower-assisted Activities with State and District Standards and Assessments

To test the extent to which SAHE-grantee activities are aligned with state and district standards and assessments, we asked SAHE-grantee project directors if "state standards or frameworks" and "state assessments" played a role in designing their Eisenhower project; we asked similar questions about district-level standards and assessments. Responses, shown in Exhibit 6.14a, indicate that standards are more likely to play a role in project design than assessments. Almost all teachers participating in SAHE-grantee projects (92 percent) are in projects that report that state standards play a role in project design, while only 62 percent of participating teachers are in projects in which district standards play a role in project design. Fifty-seven percent are in projects in which state assessments play a role in project design, compared to 40 percent in projects in which district assessments play a role.

EXHIBIT 6.14a
Percent of Teachers Participating in SAHE-grantee Projects in Which State and District Standards and Assessments Play a Role in Project Design (n=92)

For SAHE grantees, as with districts, standards are much more likely to play a role in Eisenhower activities than are assessments. This may be because more states have standards than assessments, and even states with assessments may not have their assessments well-aligned with their standards. Further, state standards are more likely than district standards to play a role in SAHE-grantee project design. Perhaps because SAHE grantees submit their proposals to the state, not to districts, there may be an automatic feedback and accountability loop that encourages grantees to be responsive to state standards. Several IHE project directors offer evidence of the importance of this link with the state. One project director from Ohio explains that her state has outcome requirements that are mandated, and the Eisenhower project is adapted to meet those state outcomes and meet the standards set by the state. A New York IHE project director also says that the implementation of the Eisenhower program was a response to statewide reforms in the mathematics and science curriculum. Similarly, an IHE project director in Kentucky says that the Eisenhower program in her state gives preference to projects that relate to the state?s learning goals and academic expectations for mathematics and science. Preference is given to projects aligned with the state assessments? core content, the Kentucky Instructional Results Information System (KIRIS).

Further, district standards may be more difficult to incorporate into SAHE-grantee professional development activities because grantees often work with multiple districts. Also, in the proposal that IHE/NPOs submit to the SAHE, they have to detail the Eisenhower professional development activities that they have planned. In some cases, identification of a district may occur after the grant is awarded, so district standards would have little or no influence on the design and development of the activity. If this is the case, requiring IHE/NPOs, in the proposal process, to identify and connect with the district from which they will draw participants for their Eisenhower project would help foster responsiveness to district goals and standards. As it now stands, it seems that the requirement for IHE/NPOs to "enter[s] into an agreement" with the LEA with which they plan to work may not be as visible in some state competitions as it could be. It could also be the case that some existing agreements have more form than substance. Our measures of alignment are limited, however, because they do not capture the extent to which informal, on-going communication and collaboration between IHE/NPOs and districts might affect alignment.

To examine whether the extent of alignment varies by institution type and departmental affiliation, we created a composite measure of alignment. We added the responses to the four alignment questions; the scale ranges from 0 to 4, where 0 means no state or district standards or assessments play a role in project design, and 4 means standards and assessments from both the state and districts play a role in project design. Results, shown in Exhibit 6.14b, indicate that there is variation across IHE/NPOs in the extent to which projects are aligned.

Exhibit 6.14b
Degree of Alignment between SAHE-grantees? Eisenhower Project and State and District Standards and Assessments, Overall and By Institution Type and Departmental Affiliation (n=92)

Projects do not differ significantly in alignment according to departmental affiliation, but nonresearch/doctoral-granting universities report more alignment with state and district standards and assessments than do research/doctoral-granting universities. This finding may reflect the fact that "other" colleges are more likely than research universities to have large teacher training programs, and thus are more likely to work with states and districts. Also, it may be that project directors at research/doctoral-granting universities are more likely to have their own research agendas that they use to shape their Eisenhower projects. In contrast, project directors at nonresearch/doctoral-granting universities are less likely to be engaged in ongoing research, and thus may be more inclined to shape their project around the state?s and district?s goals and interests.

Coordination with Other Programs

To be a part of building an effective professional development strategy, SAHE-grantee projects need to be coordinated with other programs in the district in which they are working. We asked several questions to measure the extent to which SAHE grantees coordinate their efforts with local districts.

The legislation requires each SAHE grantee to establish a cooperative agreement with the LEA and/or districts with which it works (Section 2211(3)). Of those teachers participating in SAHE-grantee projects that work with only one district, 58 percent are in projects whose directors have formal cooperative agreements with the district. Of those in projects that work with multiple districts, 71 percent are in projects whose directors report having formal agreements with one or more of these districts. Since these data are project director self-reports, the lack of formal agreements is probably at least as great as the data show, and these results suggest that many SAHE grantees are not complying with the legislation?s directive to secure an agreement with the district.

In addition to asking about formal agreements, we asked a series of questions about how SAHE grantees interact with district staff. We identified five separate dimensions of coordination: 1) feedback mechanisms; 2) support and extension of district activities; 3) ways of working with districts; 4) district involvement in planning, implementing, or monitoring; and 5) co-funding with other federal programs.

Feedback Mechanisms

To examine the required feedback mechanisms between SAHE grantees and districts, we asked each project director if either of the following feedback mechanisms takes place between the project and the district: regular required reports and required evaluations of professional development activities. Exhibit 6.15 shows that 18 percent of participating teachers are in projects that are required to provide districts with an evaluation of professional development activities, and 14 percent are in projects that are required to complete reports for the district. We conclude that IHE/NPO projects generally do not feel responsible to districts; however, they do demonstrate a responsibility toward states, possibly because, as previously discussed, it is through states that they receive their Eisenhower funds.

Exhibit 6.15
Percent of Teachers Participating in SAHE-grantee Projects That have Ongoing Feedback Mechanisms with Districts (n=92)

Support and Extension of District Activities

Another dimension of coordination is the extent to which SAHE-grantee projects are designed to support, extend, and coordinate with district activities. We asked each SAHE-grantee project director if the Eisenhower project was designed to support or extend professional development activities in the district, and whether the Eisenhower project was coordinated with district programs or reforms. Responses, shown in Exhibit 6.16, show that almost three-quarters (74 percent) of participating teachers are in projects whose directors report that their activities are designed to support or extend district professional development activities. However, less than one-quarter (24 percent) of participating teachers are in projects whose directors report coordinating with district programs and reforms. These results, again, probably reflect the SAHE-grantee project application process. Districts and IHE/NPOs may choose to work with each other when the IHE/NPO project fits with district professional development activities, but IHE/NPO projects are less likely to be coordinated with district reforms and programs than with state standards, because IHE/NPOs must be responsive to state priorities in their grant proposals.

Exhibit 6.16
Percent of Teachers Participating in SAHE-grantee Projects That Support and Coordinate with District Professional Development Activities and Programs (n=92)

Working with Districts

We asked each SAHE-grantee project director if he or she worked with districts in any of the following ways: 1) participate in district planning, 2) communicate periodically with staff, and 3) rely on district needs assessments to plan their project. We also asked if districts conducted regular visits and observations of professional development activities provided by the SAHE grantees and whether the grantee worked with districts in other ways.

As indicated in Exhibit 6.17, many of the participating teachers are in projects whose directors report working with districts in each of these five ways. Ninety percent of participating teachers are in projects whose directors communicate periodically with district staff, 71 percent are in projects that rely on district needs assessments, 53 percent are in projects that participate in district planning, 38 percent are in projects whose directors report that districts visit and observe regularly, and 64 percent are in projects that work with districts in other ways.

Exhibit 6.17
Percent of Teachers Participating in SAHE-grantee Projects That Work with Districts in Different Ways (n=92)

District Involvement in Planning, Implementing, or Monitoring

To measure the fourth component of coordination—involvement of district staff—we asked each project director if the district Eisenhower coordinator(s) and/or other district administrators are involved in planning, implementing, or monitoring the SAHE-grantee project. Responses, illustrated in Exhibit 6.18, indicate that 53 percent of participating teachers are in projects whose directors report working with non-Eisenhower district administrators in planning, implementing or monitoring the IHE/NPOs? Eisenhower project. Surprisingly, only 24 percent of participating teachers are in projects whose directors say they work with the district Eisenhower coordinator in these ways. However, more than half of participating teachers are in projects whose directors work with other district administrators. Working with district staff would seem to afford SAHE-grantee project directors the opportunity to become knowledgeable about and integrate district programs, standards, and assessments into the project; nevertheless, many do not seem to be doing this.

Exhibit 6.18
Percent of Teachers Participating in SAHE-grantee Projects in Which District Staff Are Involved in Planning, Implementing, or Monitoring Eisenhower-assisted Activities (n=92)

Source: Telephone Survey of SAHE-grantee Project Directors, Spring 1998. How to read this exhibit: The first bar shows that 24 percent of teachers participating in SAHE-grantee projects are in projects whose directors report that the district Eisenhower coordinator is involved in planning, implementing, or monitoring the project. Each bar and the number on top of it represent the percent of participating teachers for each category.

Co-funding

A final dimension of coordination is co-funding, or the extent to which SAHE-grantee projects coordinate funding with other programs operating in the districts with which they work. For a number of National Science Foundation and ED department programs (i.e., NSF?s State Systemic Initiative (SSI), Urban Systemic Initiative (USI), Rural Systemic Initiative (RSI), and Local Systemic Initiative (LSI), and ED?s Title I, Part A program), we asked each SAHE grantee if the program operated in the districts with which the IHE/NPO worked during the last year, and if the program supports professional development. We also asked whether the program co-funded professional development last year with the SAHE-grantee?s project, and whether the program staff worked closely with the grantee?s staff in the last year. Results in Exhibit 6.19 show that SAHE grantees do little co-funding with federal programs, although they report high levels of working with other programs. (Note: results reported for working closely with districts are contingent upon the program existing in the district in which the IHE/NPO is working; and co-funding results are contingent upon both the program?s existence in the district and the program?s inclusion of professional development.

Exhibit 6.19
Percent of Teachers Participating in SAHE-grantee Projects That Co-fund and/or Work Closely with Other Federal Programs (When the Program Operates in the District) (n varies)

Source: Telephone Survey of SAHE-grantee Project Directors, Spring 1998. How to read this exhibit: The first bar shows that for those teachers participating in SAHE-grantee projects in projects that work with districts in which the SSI program is operating, 46 percent of participations are in projects that report working closely with the SSI program. Each bar and the number on top of it represent the percent of participating teachers for each category.

Eighty-one percent of participating teachers are in projects whose directors say they work closely with the RSI program staff. To a lesser extent, SAHE grantees work with staff from the LSI (76 percent of participating are in projects whose directors work with LSI staff), the USI (62 percent), the SSI (46 percent), and Title I, Part A (22 percent). However, while SAHE grantees work with staff of other programs, grantees often do not co-fund with them. No SAHE grantees co-fund with the Rural Systemic Initiative, and few co-fund with the USI (15 percent of participating teachers in projects located in states with USIs, where the USIs support professional development, are in projects that co-fund with the USI). Sixty-one percent of participating teachers in projects in districts with LSIs that support professional development are in projects that co-fund with the LSI. To a lesser extent, participating teachers are in projects that co-fund with Title A (37 percent) and the SSI (27 percent).

These results support the notion that many IHE/NPOs work with districts only after the IHE/NPO activity has been designed. In order for co-funding to occur, there would have to be more coordination in the design phase of the activity, and our evidence suggests that SAHE grantees design their activities independently of the district. Although many project directors report working with district administrators, this may occur primarily in the implementation phase, and thus not have an effect on the design or structure of the activities.

Overall Coordination with Districts

We combined all five of these scales to create a coordination composite measuring the extent of coordination that each SAHE grantee has with districts. There are 12 possible types of coordination, so the scale ranges from 0 to 12.¹⁴ As Exhibit 6.20 indicates, projects in "other" departments rarely have more than six coordination strategies; projects in education departments have between 3 and 11; and projects in mathematics/science departments have from 1 to 9 types of coordination. Exhibit 6.20 also shows that the level of coordination does not vary significantly by institution type. However, SAHE-supported IHE projects in education departments have significantly more coordination than IHE projects in mathematics or science departments (6.5 types compared to 4.8). These findings offer more support for the notion that education departments are more closely tied to districts than are mathematics or science departments.

Exhibit 6.20
Number of Types of Coordination of the SAHE-grantee Project with Districts, Overall and by Institution Type and Departmental Affiliation (n=92)

Departmental Affiliation

Significant Pairwise Contrasts Mathematics/Science vs. Education

Source: Telephone Survey of SAHE-grantee Project Directors, Spring 1998. How to read this exhibit: The first distribution shows that on average, teachers in SAHE-grantee projects are in projects that report having 5.5 of a possible 12 types of coordination with districts. Each dot represents one IHE/NPO project. As the number of IHE/NPO projects at one data point (or value) increases, the dots form a horizontal line that increases in length. Each distribution represents the distribution for that particular category. The number to the right of the distribution is the mean.

Overall, our survey data indicate that districts and SAHE grantees work together in some ways, but not others, and that district goals and standards are not always integrated into SAHE-grantee activities. This conclusion is supported by data from our interviews with a number of SAHE-grantee project directors that we conducted in conjunction with our in-depth case studies. For example, in Ohio, a school district and partnership schools in the city are involved in the SAHE-sponsored IHE professional development activity, which focuses on providing science materials and equipment to science teachers. The university provides direction, coordination, and evaluation of the project; facilitates the assembly, organization, and storage of teaching kits, hardware, and software; and provides technical assistance. The university also is involved in planning, guiding teachers? academic progress, preparing and grading exams, and lending and hauling specialized equipment for demonstrations. The local school districts provide the use of their library and classrooms, audiovisual equipment, utilities, and janitorial service for several classroom sessions. All cooperating schools also agreed to provide $150 per teacher-participant for instructional materials to be used with their students in local classrooms. In addition, one of the teachers from the district is involved in every planning session and reports personally to the superintendent. The local county Board of Education provides courier service and some clerical help and time for communications, recruitment, and supervision of teacher selection. Despite this multi-level collaboration, the IHE reports that their project does not relate to the professional development goals of the district, and in fact, the IHE is unfamiliar with the district?s professional development goals.

Similarly, in Texas, a SAHE-sponsored IHE project director collaborates with district and federal programs, but reports that district standards and assessments have little effect on the project?s professional development activities. The project?s primary goal is to create a support system to increase implementation of the national standards in science and math. The superintendent or assistant superintendent of the district must sign off on the project and agree that their teachers will participate. The project is coordinated with the USIs in the districts that the project serves, and there is collaborative planning with the districts? science coordinators, three USI directors from two different districts, a grant coordinator from one of the districts, and professors from the college. The Eisenhower and USI directors volunteer time to work with each other. Mentor teachers also give input into the project development. The IHE project director reports that she works very closely with the district. She spends approximately 200 hours in classrooms each year doing follow-up. Despite this level of coordination and collaboration, the director reports that district assessments, standards, or indicators do not affect the design or implementation of the program.

Thus, if the goal of requiring SAHE grantees to develop cooperative agreements with districts and to work closely with districts in implementing their projects is to foster collaboration in the design of professional development activities, alignment with district standards and assessments, and integration with district programs and reforms, IHEs and districts do not seem to be meeting the goal. While IHE/NPOs report working closely with districts on several activities, they report not working closely on other key dimensions, such as co-funding and working with the district Eisenhower coordinators. While the independence of IHE/NPOs may serve an important function in allowing them to provide new knowledge and innovative approaches to teaching, coordination with districts might be encouraged in specific areas where it may benefit both IHE/NPOs and districts, such as in co-funding and targeting.

---

¹⁴ The extent to which IHE/NPOs work closely with federal programs is not part of our overall coordination composite because the intent of the composite is to measure overall coordination efforts with districts, not federal- or state-level coordination.

-###-

---

[Targeting and Recruitment of Teachers]

[Implementing the Vision: IHE/NPO Procedures for the Continuous Improvement of Professional Development Activities]