A r c h i v e d  I n f o r m a t i o n

Assessment of Student Performance April 1997

CHAPTER 3

Part 4

The College Board's Pacesetter Mathematics Program:
Sommerville High School

Site Visit Dates: May 10, 1995

Developed jointly by the College Board and the Educational Testing Service, Pacesetter programs are high-school level "integrated programs of standards, teaching, and assessment." The College Board views Pacesetter programs as one element within a strategy designed to encourage all students, particularly students who are disadvantaged or members of a minority group, to take rigorous academic courses while in secondary school. As the description of the Pacesetter programs suggests, Pacesetter assessment is integrated with the program's curriculum and instructional approach. The focus of this summary is the College Board's Pacesetter mathematics program and its implementation at one high school.

The Pacesetter Programs

The Pacesetter programs were developed to complement the College Board's existing Equity 2000 program. Together, the two programs form the core of what the College Board calls its "push-pull" reform strategy: Equity 2000 is designed to "push" students, particularly minority and disadvantaged students, into more demanding academic preparation by requiring them to take advanced math courses; Pacesetter, in turn, is designed to "pull" students toward the goal of high standards of achievement for all students before graduating from high school.⁶

All Pacesetter programs (which are currently offered in mathematics, English, and Spanish, and in the future will include science) incorporate three central components:

• Statement of Standards,
• Teacher Preparation, and
• Assessments.

Pacesetter Mathematics

Pacesetter math was the first Pacesetter program to be developed. It was designed to be a "capstone" course in math, taken primarily by students in their senior year of high school. The College Board designed the curriculum as a pre-calculus course, with an emphasis on modeling..

The program comprises six units, each of which focuses on a particular type of mathematical function. The units include a variety of integrated instructional and assessment activities — in particular, lengthy "task sets" — that students work on guided by their teacher. Pacesetter also includes an end-of-year on-demand assessment, taken by all students across the country enrolled in Pacesetter, that assesses students' achievement in terms of "mathematical knowledge," "applied problem solving," and "communication in the language of mathematics."

Piloted in the 1993-94 school year at 15 volunteer schools in 10 school districts across the country, the pilot of the Pacesetter mathematics program provided the College Board with feedback about the content and implementation. Specific lessons learned from the pilot included:

• On the end-of-year assessment, students experienced the most difficulty with tasks that required them to "communicate in the language of mathematics;" and
• The end-of-year assessment was too long and needed to be more closely aligned with Pacesetter curriculum tasks.

For the 1994-95 school year, the College Board addressed these problems and, according to the College Board, most Pacesetter teachers agreed that the assessments were more fair.

Pacesetter Participant: Sommerville High School

Located in the suburbs of Washington, D.C., Sommerville High School serves primarily African-American students (89 percent), with the remainder of students being white (8 percent), Asian-American (2 percent), and Hispanic (1 percent). In addition to its regular program, Sommerville High offers programs for students who are gifted in the visual and performing arts, gifted students who are not enrolled in one of the district's two "flagship" schools for the gifted, and students in vocational education.

The school was selected by its district's mathematics supervisor to participate in Pacesetter because of its ongoing participation in the College Board's Equity 2000 program and because the school's mathematics department chair was enthusiastic about experimenting with the program. The Pacesetter course is taught by only three of the school's 21 math teachers and reaches only a small fraction of the school's students.

Financial Resources

The pilot implementation of Pacesetter math in 1993-94 was funded by the College Board and the Educational Testing Service. However, beginning in 1994-95, schools using the Pacesetter mathematics program had to pay for the curriculum and the support provided by the College Board and ETS, paying $30 per student enrolled in the Pacesetter math program.

The district provided the necessary financial support for Sommerville to participate in Pacesetter during the 1994-95 school year and agreed to fund the program through 1995-96. However, the math supervisor says that the program is too expensive for the district to be able to support its expansion and introduction into other high schools. "It is time to start talking with the College Board about the next steps with Pacesetter because it is too expensive the way it is now," commented the math supervisor.

Impact of Pacesetter Mathematics

The three teachers involved with Pacesetter at Sommerville High School all expressed positive comments about Pacesetter. They said that the Pacesetter approach to teaching mathematics and student assessment, which includes performance-based tasks, reflective journals, short answer quizzes, essays, and the integrated use of technology, was already compatible with their teaching styles and philosophies.

However, they also acknowledged a real shift in their teaching strategies resulting from their participation in Pacesetter. They have honed their skills at writing performance assessment tasks for use with their Pacesetter students. Furthermore, they are applying many Pacesetter strategies in their other math classes. For example, teachers say they are using more technology, encouraging group work, and constructing more performance-based assessment tasks.

Sommerville's Pacesetter teachers also expressed some reservations about the Pacesetter curriculum. Teachers' concerns included the following:

• "Math skills" are not adequately incorporated into the curriculum;
• The curriculum is too long for a year-long class; and
• Although the College Board presents the program as a pre-calculus course, Sommerville teachers found that the course is not adequate preparation for calculus.

However, Sommerville teachers emphasize that these concerns about Pacesetter math are minor in light of the benefits they believe the program offers their students.

All 12 students interviewed for this study were enthusiastic about Pacesetter. More than any other comment, students suggested that, for the first time, they understand, through their work on task sets, that math applications are valuable in "real life." As one student said, "You would never think it would take a function to get an interest rate. I want to go into business, and that is relevant to me." Another student said that the course had been a "confidence booster" for her; she said that she liked and understood math for the first time.

The two parents interviewed were both well informed about Pacesetter and were pleased with the impact the program had upon their children. Both parents said Pacesetter had turned around their children's understanding of mathematical concepts, their attitudes toward math, and their self-esteem with respect to learning math.

One of the parents, a biology teacher at Sommerville, said she had noticed that the Pacesetter students who came to her class already knew the mathematical applications that she had always had to teach her biology students. She said, "As a parent, I give Pacesetter a 10 out of 10. As a teacher, I give it a 12 out of 10."

Future Plans

Sommerville teachers, students, and parents all expressed their belief that mathematics learning has improved because of their involvement with Pacesetter. Hence, they would like to continue their participation into the future. However, given the expense of using Pacesetter, expanding — and even maintaining — Sommerville's participation in the program may, itself, prove problematic.

The New Standards Project:
Ann Chester Elementary School

Site Visit Dates: May 25-26, 1994

Ann Chester Elementary School in Fort Worth, Texas, opened its doors in the fall of 1992 with a mission to establish a "distinctive learning environment" for its students. Since then, the school has been a center of reform activity and participates in both the New Standards Project (NSP) and the district's Applied Learning Program (ALP).

Ann Chester's teachers and students are a self-selected group, as both teachers and students must apply to and be accepted by a school committee. During 1993-94, the school enrolled 380 students in grades K through 6. About half the students were white, 27 percent were African American, 20 percent were Hispanic, and 3 percent were Asian American.

NSP and ALP Partnership

Since the Fort Worth Independent School District joined the NSP in 1991, teachers at Ann Chester (and at other district schools) have been involved in developing assessment tasks and in piloting and adopting NSP assessment guidelines. In addition, Ann Chester teachers are involved in the Applied Learning Program, which encourages the use of applied, project-based instruction and performance assessments.

The goals of the two programs are compatible at Ann Chester. The goal of the NSP is to revitalize the education, using assessment as a tool for transforming teaching and learning, while the ALP philosophy emphasizes explicit connections between the classroom and the world of work as a central component of classroom instruction.

Since becoming involved with the NSP, Ann Chester has modeled its 4th-grade standards in reading and writing on the NSP literacy dimensions. The school also plans to adopt the NSP mathematics standards in the near future. Three of its teachers currently are developing and piloting mathematics tasks and scoring rubrics for the NSP. In addition, the school plans to develop performance standards in English language arts, social studies, mathematics, and science for all grade levels.

School-based Performance Assessments

Because of the school's philosophy and participation in reform activities, teachers at Ann Chester enjoy the freedom to use a variety of creative, extended-response assessments on an ongoing basis. Teachers use performance assessments to inform their instructional strategies and to provide information to students and parents about students' educational progress. They make no clear distinctions between assessments and instruction, using project-based assignments for instructional as well as for evaluative purposes.

Students are assigned hands-on, inquiry-based project work that is designed to emulate the world of work. For example, for one project, students formed a "news crew" that broadcast news events every day. For another project, a team of students designed the school admission application forms and procedures. All such projects explicitly juxtapose academic learning and social problem-solving to reinforce the connections between classroom and "real-life" experiences.

All students are required to keep a portfolio of their accomplishments in reading, writing, and other projects that illustrate how they are progressing as learners. In concrete terms, this means that each student's portfolio contains samples of work from all content areas. Students routinely review these portfolios to assess their progress vis-a-vis goals they set for themselves at the beginning of the year. In addition, the school invites parents to a yearly Assessment Night to examine their children's portfolios.

To enable teachers' use of project-based instruction and performance assessments, Ann Chester applied for and received a waiver from the district to replace the traditional letter-based report card with a narrative report card. Initially, teachers had described students' dispositions of character and habits of mind in the narrative report cards. However, because of requests from parents, teachers now describe students' progress in academic areas and include a statement about the student's rank in the classroom.

Impact on Teachers, Students, and Community

No formal evaluations of the effects of assessment reform at Ann Chester have been conducted. Yet, teachers believe that the reforms have contributed to the success of their students; they are impressed with their students' vocabulary, ability to think, and ability to work cooperatively in groups.

However, despite their commitment to the system and strong support from their peers, teachers want more time for professional activities such as learning how to write scoring rubrics. Furthermore, although much instruction is tailored to the student, many teachers believe that it could be individualized further.

Students, too, had positive reactions to project-based work and assessments. One student described the projects she had to complete as "fun activities." Students also like keeping portfolios, as it allows them to reflect on and take ownership of their work. "My portfolio has my best work and some of my favorite work," said one 5th grader. In addition, students enjoy constructing their own scoring rubrics to evaluate their progress.

Parents interviewed for this study indicated that they are generally satisfied with the school and with their children's overall progress. They like the active-learning pedagogical approach and the narrative report cards. However, they also expressed the concern that their children might not be getting enough exposure to "basic skills." This concern was echoed by a School Board member: "When you do applied projects, you can't sacrifice content," he said. Parents want their children to learn multiplication tables, spelling, and grammar and to perform well on multiple-choice tests.

In response to these concerns, one teacher allows parents to take their children out of her class to drill them in multiplication tables and spelling; and the school as a whole organizes forums to inform parents about how basic skills are being taught through project-based instruction.

Future Plans

Teachers at Ann Chester plan to continue to focus on developing performance standards for each grade-level in each subject area. In addition, they will collect student work that illustrates "how good is good enough."

The New Standards Project:
Noakes Elementary School

Site Visit Dates: February 21-22, 1995

As part of its attempt to establish an educational system that, according to the Superintendent, "makes learning real," the Anton School District in Iowa has been developing alternative forms of student assessment for over a decade. It was, in part, its long exposure to performance assessment that convinced the district to join the New Standards Project (NSP) in 1992. Iowa supported the decision and financially supports Anton's membership in the NSP. (The state supports several other districts' NSP participation as well.) The district also applied for and received a five-year waiver for reporting yearly progress of student achievement to the state.

Jointly run by the National Center on Education and the Economy and the Learning Research and Development Center at the University of Pittsburgh, the NSP is working toward developing and adopting both a set of academic standards and new ways to measure student learning and achievement.

This case-study summary focuses on Anton's participation in the NSP and the development and implementation of NSP-influenced performance assessments at Noakes Elementary School.

Anton School District's Involvement in NSP

As part of their involvement with the NSP program, teachers throughout the Anton School District have piloted numerous NSP English language arts and mathematics assessment tasks and English language arts portfolios.

An Example of a 4th-Grade NSP Pilot Task. For completing the NSP mathematics tasks, students are required to solve a mathematical problem, explain the method they used to solve the problem, and explain why they chose to use that method. For example, for one 4th-grade mathematics task, students are told they have $25 to buy fish for their 30 gallon classroom aquarium. Students are provided information about sizes of the fish, how much they cost, and their special needs. Then, in a letter to the principal explaining which fish they chose, students are asked to (a) tell how many of each kind of fish they bought, (b) give their reasons for choosing those fish, and (c) show that they are not overspending and that the fish will not be too crowded in the aquarium. Booklets for such tasks contain a general rubric and a rubric specific to the task for scoring students' performance.

English Language Arts Portfolios. In addition to piloting assessment tasks such as the one described above, several teachers also have committed to using the NSP guidelines for English language arts (ELA) portfolios. Teachers are expected to randomly select some of the portfolios and send them to an NSP benchmarking conference.

Each ELA portfolio must contain certain types of student work. For example, each elementary ELA portfolio must contain (a) a table of contents; (b) a reflection piece; (c) a response to literature; (d) some best pieces that, among other things, tell a story, demonstrate the collection and reporting of information, and show the student's best efforts in reading; and (e) four free choices.

Each portfolio entry is scored using a NSP-developed scoring rubric.

Several Anton teachers have attended the NSP conferences focused on benchmarking, scoring, and developing such assessment tasks and portfolios. The district pays for teachers to attend the conferences, and the participant teachers are expected to share the information from the conferences with their peers. Teachers trained in the use of the NSP scoring rubrics said they are comfortable with scoring the NSP assessments. However, one teacher said that a clear set of anchor papers is essential to good scoring.

Noakes Elementary School

Noakes, one of the six elementary schools in the district, spans grades K through 6, and in 1994-95 enrolled about 565 students, 98 percent of whom were white.

Noakes is an enthusiastic participant in the district's assessment reform efforts. One of Noakes' 1994-95 school year goals was to ensure that each child complete a portfolio in at least one academic area. In order to fulfill this goal, all Noakes teachers received some training in how to develop and implement performance assessments, and two 4th-grade teachers received extensive training through the NSP in the development, implementation, and scoring of the NSP portfolios.

Noakes, however, does not use a standard, school-wide portfolio system, and teachers are not required to use the NSP guidelines. Thus, teachers exercise individual control over how they develop and use portfolios in their classes.

All teachers at Noakes are committed to the use of performance assessments — albeit in different ways. Some teachers use portfolios in only one subject area, while others use them in all areas. In addition, some teachers do not score the portfolios, while others incorporate the scores they assign to the portfolios into students' final grades.

Overall, teachers — both those directly involved with the NSP and those devising and using their own performance assessments — had positive opinions about the pedagogical value of performance assessments. For example, a 6th-grade science teacher who administered a two-hour performance task on electrical circuits said that, although the task took more time to complete than a traditional quiz covering the same topic, ". . . it was [a] worthwhile, valuable learning experience that the students enjoyed."

One of the teachers directly involved with the NSP said that it had opened new doorways for professional growth by showing her how new units could be added to the curriculum. Other teachers pointed out that the portfolios help students demonstrate what they know, reflect on their progress, and take ownership of their work; performance-based assignments help students to gain a deeper understanding of the subject matter and to retain information longer than do other types of assignments.

Teachers also pointed out that performance tasks allow students who traditionally experience academic difficulties to perform well. On the other hand, one teacher noted that academically challenged students sometimes are unable to complete all the steps in an assessment task.

Teachers expressed concerns, however, about the lack of time and an inadequate amount of training to develop and implement performance assessments in a manner satisfactory to them. As one teacher said, "It puts stress on our free time, after school time, and weekends."

The 4th-grade students interviewed had very definite opinions about the performance task they were working on and about performance assessments in general. They found the performance task they were currently working on to be very interesting and engaging, and they enjoyed being able to write about their opinions. Students described other performance assessments they had done as learning experiences.

The district is trying to provide information about its participation in the NSP to parents and community members, most of whom are unaware of the NSP. However, according to a parent of a 4th grader, those parents who are aware of the changes in assessments are supportive of them. This parent said that she understands the NSP system and has seen its positive effects on her child. She has been able to see her child's progress by looking at his portfolio and is impressed by his ability to communicate his feelings about his work.

Future Plans

The Anton School District does not plan to mandate the use of portfolios or other performance assessments. Instead, the teachers and administrators currently involved in the assessment reform process want the adoption of a performance assessment system to be a "bottom-up" decision. As a result, teachers who are using performance assessments are involving more teachers by providing in-service training and help to those who are interested. In addition, the Anton Executive Director of Instruction hopes soon to establish a professional center that will house information on student-centered learning and performance assessments.

Harrison School District 2's Performance-Based Curriculum:
McGary Elementary School

Site Visit Dates: April 11-12, 1994

The development and implementation of a Literacy Performance Based Curriculum (PBC) represent a break from past practice in Colorado's Harrison School District 2 (HSD2). The new curriculum focuses on student outcomes and incorporates performance assessments.

The curriculum development process incorporates identifying, integrating, and assessing significant student outcomes for all students through the adoption of performance assessments. The district is developing its PBC in response to Colorado's House Bill 1313, which requires local education authorities to adopt standards for reading, writing, mathematics, science, history, and geography to guide curriculum, instruction, material selection, and assessment of student performance.

Performance-based Literacy Assessments

After the development and district-wide implementation of HSD2's writing performance assessment during the 1989-90 school year, the HSD2 began work on revising the K through 12 reading and language arts (literacy) curriculum, the first curriculum area mandated by the state to undergo the standards-based change. The cornerstone of this revision was the replacement of traditional forms of assessment with performance assessments.

During the curriculum review process, members of the district's reading and language arts curriculum committee developed performance assessments and benchmarks for student achievement. The literacy PBC was piloted during the 1992-93 school year, revised during the following summer, and implemented districtwide during 1993-94.

The literacy PBC is designed to set achievement targets for students, classes, and grade levels; to conduct ongoing assessment throughout the year; to design and carry out lesson plans that continuously teach outcomes and target achievement indicators; to teach students to become self-assessors by providing them with ongoing feedback through rubrics; and to insure that each unit of study has clearly stated outcomes as well as a means to assess results.

As part of achieving these goals, teachers administer three (out of five developed by the district) performance tasks each year. The performance tasks are activities that students engage in and complete to demonstrate that they can formulate the ideas and perform the skills identified in the target achievement indicator.

One of the five performance tasks (predetermined by the district) is utilized for school-building accountability purposes. Scores from the assessment are aggregated to the district, but are not high-stakes. Instead, they are intended to provide instructional direction.

In some cases, especially at the elementary levels, the literacy assessment tasks have a substantive focus that overlaps with other curricular areas, such as science and history, thereby encouraging cross-disciplinary instruction.

To invite classroom participation of both teachers and students, two scoring rubrics for each assessment task are included in the literacy PBC: the teacher scoring rubric and the student scoring rubric. While students are performing the task, the teacher frequently refers to the student scoring rubric and asks the students to reflect on their performance and to evaluate other students work based upon the student scoring rubric.

One important feature of HSD2's assessment reform effort is its homegrown "Assessment Academy." The Academy provides all district teachers with training in how to develop, administer, and interpret performance-based student assessments. HSD2 teachers expressed positive reactions to the training they received through the Academy. In the words of one teacher, "I haven't had to go outside the district to get the support I need."

McGary Elementary School

A parent interviewed for this study observed, McGary Elementary School is ". . . not your typical elementary school. It is way above average." The school has been described as an innovative school with a dynamic principal — a leader who also happens to have been one of the key figures in the adoption of performance assessments in the district. Several people in HSD2 agree that McGary has been the school in HSD2 that has most proactively embraced the new literacy curriculum, and, in fact, it is the furthest along in articulating and defining scoring rubrics and student proficiency levels beyond the literacy curriculum.

In 1993-94, McGary enrolled about 470 students whose ethnic composition was similar to that of the district as a whole. Approximately 47 percent of students were white, 26 percent were African American, 21 percent Hispanic, 5 percent Asian American, and 1 percent Native American.

Impact of PBC on Teachers, Students, and Parents

One teacher related that ". . . there was initially a mixed reaction among the faculty about the move to PBC and performance assessments. There were clearly some skeptics and some early adopters, but there was respect for the process of implementation undertaken by the district and for our principal." Most teachers interviewed agreed that PBC and performance assessments enhanced not only their pedagogical techniques, but the very nature of instruction in the classrooms. Teachers said that the use of performance assessments gives them information about their students and a better idea of what and why they are teaching.

Although most of the impact of PBC was perceived to be positive, some teachers expressed reservations and discomfort with the changes. One point of initial resistance to this change was from teachers who were attached to favorite units they had developed and used for years. Another point of resistance came from teachers who were not as capable as others at integrating the new curriculum into their classrooms.

Most McGary teachers realize that the curriculum reform process is going to take a long time to reach even a semi-final form. So far teachers and parents see the assessment effort as a "value-added" experience that has great potential for reaching its goals, but not until the distant future. Indeed, the district is well aware of the ouster of the majority of members of the Littleton, Colorado, School Board after parents and community members reacted to their perceived rapid movement toward adopting outcomes-based education. As one teacher said, "It is important that we go slow, because of what happened in Littleton."

In general, parents expressed their support of the performance assessments. One parent said, "The performance assessments have been well accepted by 98 percent of the parents, but I wouldn't want to see the traditional grading system go by the wayside." In general, parents and community members are comfortable with the change and feel that HSD2 is "adding something, not taking something away" with the implementation of performance assessments.

Teachers suggest that PBC and its scoring rubrics have helped to clarify standards for their students. One student said that she liked the scoring rubrics because they helped her write advanced-level stories. She said, "I look at the rubric to make sure everything is there. Mom helps me identify where I am on the rubric sometimes." Teachers are even more enthusiastic about the impact of the PBC and performance assessments on students than the students themselves are. One teacher noted, "I have never before had a student come to me with a B and ask, 'How do I get an A?'"

Future Plans

There is much that district- and school-level participants would like to see happen in upcoming years. They would like to see more thought go into the inclusion of students with disabilities in their classrooms. They would also like to see a better articulation between the curriculum and assessment at different grade levels and the continued refinement and adoption of performance assessments.

One 2nd-grade teacher summed up the opinions among HSD2 personnel about the reform efforts: "I think it will be worth it. I'm excited to be focusing on what we will be doing in future years. These are stressful changes, but we are getting support."

⁶ Facts about Pacesetter. The College Board, 1994.

-###-

[Chapter 3: Case Study Summaries Part 3 of 5] [Chapter 3: Case Study Summaries Part 5 of 5]