![[Contents]](/icons/up.gif){kind=icon}
[Chapter 3: Case Study Summaries Part 3 of 5] Assessment of Student Performance April 1997
In 1988, after six months of careful observation, the principal at Ni?os Bonitos Elementary School in San Diego, California, determined that children's needs — particularly those for learning the English language — were not being met. Students' Abbreviated Stanford Achievement Test (ASAT) scores were low, and staff morale was poor. The principal, thus, asked the full staff to take responsibility for developing the mechanisms of change.4
The development of the school's performance assessment system was heavily influenced by the school's demographic character. In the 1994-95 school year, Ni?os Bonitos served 924 students in grades Pre-K through 6. Thirty-nine percent of the students were of Southeast Asian heritage, 46 percent were Hispanic, 5 percent were African American, and the rest were East Asians. Seventy-seven percent of the students were identified as possessing limited English proficiency (LEP).
To initiate the school (and assessment) reform process, the principal developed a site-based committee structure of governance for the school. This committee proceeded to revamp the school curriculum and to reorganize the school and the school day. Students were organized into four non-graded but age-appropriate "wings." In the morning, all students within each wing, except those who are Spanish-speaking, are assigned to one of six levels of English proficiency for language arts and mathematics coursework. Spanish-speaking students are enrolled in a bilingual program.
Initially, emphasis was placed upon language arts improvement, historically the area of students' greatest weakness and of staff's greatest frustration. Teachers established clear entry and exit criteria and instructional goals for each age and language development level. For each level, the learning outcomes describe in detail what students should know and be able to do in terms of oral language, reading, and writing. Using learning outcomes, assessment has become an integral, daily part of instruction.
Written and audio portfolios of student writing, reading, and spoken language allow teachers to measure growth in language arts. Teachers have defined what each portfolio must contain for each language development level and age level. For instance, the primary-level portfolios for Spanish-speaking students should contain an audio sample of a story retold by the student, three daily journal samples, a minimum of three independent writing samples, and a reading checklist.
Teachers also developed scoring rubrics in order to objectively compare and communicate students' academic progress without having to "label" children with standard letter grades. The language arts scoring rubrics were modified and refined between the two years of the study; by 1994-95 the faculty had identified five performance levels for oral language and reading and six levels for written language that cut across language development levels and age levels. The rubrics were used to redesign the student report card in language arts, which was renamed the student "growth record." Students are assigned a "growth rubric" by their language arts teacher based upon a review of their written language portfolios and an assessment of their performance on reading and oral language.
Classroom Observation: In a 3rd/4th-grade ("middle-wing") classroom, groups of "transitional" students (those who are almost, but not quite, fluent in English) spent the morning working on six computers. Their teacher, who has received special training in the use of educational software, designed a performance task that required the students to describe and illustrate a book they had read about the difficulties Southeast Asian students (like themselves) experience as they assimilate into their new American culture. The teacher adopted the role of "coach" and circulated throughout the classroom as her students worked and helped them with their writing skills as well as with their computer skills. Students saved their work in both electronic and "hard-copy" versions for their language arts portfolios, which are shared with parents at parent-teacher conferences three times a year. |
In the second year of the study, teachers began to reevaluate the way in which the language arts portfolios are constructed. To date, teachers have selected the student pieces to be included in each portfolio. However, they are beginning to feel that students should have greater influence in the selection process, so that the portfolio becomes a tool for student self-reflection, as well as a method of measuring student progress from the teacher's perspective.
During the 1993-94 school year, Ni?os Bonitos staff began to research and write mathematics standards and rubrics, using the same process they had followed for language arts. The full staff, meeting in groups by wing, developed learner outcomes for each age level. These outcomes identified what students should know and be able to do to demonstrate mastery of each content and process skill area in mathematics. The mathematics curriculum committee then developed separate "observable student behaviors," which detailed the tasks each student must be able to perform, to accompany each set of learner outcomes.
In 1994-95, these learner outcomes and observable student behaviors were used to develop High Expectations Learning Plans for Students (HELPS). HELPS units require teachers to develop specific activities and assessment tools for each concept area (e.g., geometry and spatial sense or fractions and ratios) identified by the National Council of Teachers of Mathematics.
From the process and content skills they developed, the mathematics committee generated a scoring matrix that was used to evaluate student progress for the first time in 1994-95. The school received a waiver allowing it to use the new mathematics assessment matrix as a substitute for the mathematics portion of the standard district report card. Also in 1994-95, mathematics portfolios were introduced at all grade levels, although in differing forms and at differing paces.
Impact of Assessment Reform
Although no formal evaluations of Ni?os Bonitos' performance assessment system had been undertaken as of May 1995, the grassroots teacher involvement with developing learner outcomes and assessment methods seems to have ensured the success of reform efforts. Ni?os Bonitos' annual staff surveys indicate that, overall, teachers are very pleased with the new system and its results. The school's staff argues that their system of measuring student performance has more validity than do more traditional kinds of tests. They maintain that the portfolios and learner outcome tasks they have developed provide a complete picture of students' competencies.
In support of the central reform features (i.e., the curriculum and assessment systems), Ni?os Bonitos staff also have reduced individual class size and provided teachers with more weekly preparation time (teachers only teach one class during the afternoon rotation period). Some of the numerous grants the school has received allow teachers more release time.
The principal noted that the reformed system has improved classroom instruction — most teachers have moved away from the traditional teacher-centered approach and are providing more cooperative and experiential learning opportunities that focus on problem-solving skills. In addition, these new instructional and assessment methods have reduced disciplinary problems and provided teachers with a greater sense of accomplishment. One teacher noted that although the new methods may slow the pace of instruction, they reinforce the fundamentals of mathematics and language arts.
Teachers and administrators believe the effect of curricular and assessment changes on students has been noticeable. Students are doing a large amount of writing and have a greater sense of purpose, accomplishment, and enjoyment. During classroom observations, the principal has noticed a much higher degree of student engagement and "on-task" behavior. These improvements in student attitude and motivation have borne fruit in objective measures of student performance, particularly in the areas of oral and written language development for LEP students — ASAT writing scores continue to improve, and Ni?os Bonitos graduates are performing better at the junior high school.
Parent reaction to the new portfolio system and rubrics has been extremely positive, as measured by both the annual parent survey and the feedback received at parent outreach meetings held regularly throughout the year. The school's parent and volunteer coordinator noted that portfolios give parents a much more visual and concrete sense of their child's progress.
Future Plans
The future of Ni?os Bonitos' performance assessment system seems bright, since the school possesses strong leadership and has received support and accolades from the San Diego Unified School District. The school board member interviewed in 1994-95 anticipates that, with the continued and strong support of both Ni?os Bonitos' new principal and the district's new superintendent, Ni?os Bonitos and the district as a whole will steadily expand the use of performance assessments.
As part of an ongoing, district-wide restructuring effort, Prince William County (Virginia) Public Schools (PWCPS) has introduced Applications Assessments in mathematics, science, language arts, and social studies.
The district's work toward a reformed educational system has progressed rapidly. Applications Assessments in language arts, math, and science were pilot-tested in the fall of 1993, and were administered districtwide in the spring of 1994. During the summer of 1994, committees of district teachers developed performance standards for the three assessments and finished revising the district's curricula in most subject areas. During the 1994-95 school year, the new curricula began to be put in place, and the Applications Assessment in social studies was introduced.
The Prince William County Public Schools
Since the late 1980s, PWCPS has been planning and implementing a comprehensive restructuring effort encompassing:
The district decided to modify its assessment program when it realized that traditional methods of testing would not be able to accurately measure student progress toward the attainment of new standards. PWCPS, with the help of the Riverside Publishing Company, began developing a new assessment tool Applications Assessments. Riverside developed potential assessment items, and PWCPS teachers reviewed and revised the items. The assessment is designed to meet three additional objectives:
The Applications Assessments, which are administered to all 3rd, 7th, and 10th graders, constitute an on-demand performance assessment in the areas of mathematics, language arts, science, and social studies. Each assessment is contained within a booklet; a typical assessment has about a dozen tasks that call for anything from a phrase to a multiparagraph or multistep response. Each assessment task is accompanied by its own scoring rubric. The scoring is conducted by a company associated with Riverside Publishing.
| Prince William County Public Schools' Standards of Quality Students should be:
|
Staff Development
To acquaint district teachers with the Applications Assessments, the district compiled a Resource Kit, which was distributed to all schools in March of 1994 and again in the spring of 1995. The district also held a pair of two-day workshops for teachers who would be administering the assessments. Teachers who attended these workshops were responsible for bringing what they had learned back to other teachers at their schools. (The district did not anticipate repeating these workshops during the 1994-95 school year.)
Other PWCPS Assessments
The district also administers a state-level, open-ended assessment and a district-level, criterion-referenced, multiple-choice assessment.
Virginia's Literacy Passport is a high-stakes assessment comprised of three sections — reading, writing, and mathematics — and is taken by all 6th graders. Successful completion of all three sections is intended to reflect students' mastery of basic literacy and is required by the end of 8th grade if students are to be considered "full" 9th graders.
Since one of the six Standards of Quality (see box) focuses on basic skills, the district has adopted new Basic Skills Assessments. While the Applications Assessments aim to measure students' abilities to apply knowledge, the criterion-referenced, multiple-choice Basic Skills Assessments are more traditional in their assessment of students' mastery of subject matter. The Basic Skills Assessments are administered in grades 3, 5, 7, and 10.
The implication of the curriculum revision, completed during the summer of 1994, for the Applications Assessments is unclear. According to administrators and teachers, the Applications Assessments did not guide the curriculum revisions in any direct way. Therefore, it remains for the district's future evaluation to reveal how closely the revised curricula and the Applications Assessments correspond to one another.
Westgate Middle School
One of 12 middle schools in PWCPS, Westgate Middle School serves about 825 6th-, 7th-, and 8th-grade students. Students come primarily from middle class backgrounds; most students are white (81 percent), while others are African American (11 percent), Hispanic (5 percent), and Asian American (4 percent).
Westgate Middle School has been a central player in assessment reform and other education reform efforts introduced by the district. The school is operated by a school-based management team; over the past six years, the district has supported the adoption of school-based management districtwide. Westgate Middle School also has been using portfolio assessment schoolwide during the past four years. The portfolios are used to provide insights into students' learning processes and achievement, not for assigning grades.
Impact of Applications Assessments on Teachers, Students, and Parents
Most Westgate teachers have a favorable opinion of performance assessments in theory. However, their reactions to the Applications Assessments range from positive to ambivalent. Teachers found that the language arts and science assessments administered in 1994 were well aligned with the district's pre-revision curriculum and students' actual classroom experiences. However, teachers also said that the math assessment tested skills most 7th-grade students had not yet mastered. Teachers also were unclear about the alignment between the district's new curricula and the Applications Assessments, and they expressed concerns about the subjectivity inherent in the use of scoring rubrics.
Teachers applaud the district's efforts to include them in the process of developing the Applications Assessments, setting performance standards, and revising the curricula. However, one teacher warned, "Performance assessment is not the end all and be all. Educators tend to get caught up in fads."
Students interviewed during the 1993-94 school year said that the Applications Assessments were "easy" but that they preferred multiple-choice exams. Some students liked having the chance to express their opinions about issues on the language arts assessment, while others said that the themes carried throughout the assessments were "boring." Students interviewed in 1994-95 expressed similar reactions to those expressed by their classmates the previous year.
The two parents who participated in this study were not well informed about the Applications Assessments and, hence, had few opinions about them.
The Prince William County Public Schools have embarked upon a comprehensive restructuring effort, simultaneously reforming curriculum, assessments, and standards. The reform effort will require several more years to bring to fruition, and the effects of the various reforms, singular or cumulative, cannot yet be determined.
In 1988, the Vermont Department of Education began developing an assessment system consisting of portfolios and multiple-choice, uniform tests in mathematics and writing. Pressure from the business community and the Department's own belief that information on student performance is essential to improving student outcomes provided the impetus for this action. Thus, Vermont's portfolio system became the first of its kind in the U.S. Although administering the assessment system was not mandated, 59 of 60 Vermont Supervisory Unions were participating in it by the 1992-1993 school year.
Vermont's portfolio system is designed to foster local involvement and initiative in assessment reform. It is intended (a) to furnish data on student performance, (b) to encourage effective approaches to instruction, and (c) to foster equity in educational opportunity.
Vermont's Board of Education reasoned that the portfolio assessments would help achieve these goals by (a) enabling schools to compare scores, (b) building local capacity for districts to evaluate their mathematics and writing programs, and (c) helping teachers assess their instructional methods and students' progress.
Vermont Portfolios
In the school years between 1991 and 1994, each 4th- and 8th-grade student was required to maintain both a mathematics and a writing portfolio. In the 1994-95 school year, the writing portfolio was shifted from the 4th- to the 5th-grade level — a shift intended to lighten the portfolio burden on 4th-grade teachers.
Maple Leaf Teachers' Assessment of According to the Maple Leaf teachers present at an end-of-year scoring training session, the best aspects of the mathematics portfolios are:
The best aspects of the language arts portfolios are:
|
Mathematics portfolios must contain a set of 5 to 7 "best pieces" selected from mathematics assignments completed over the course of the school year. Examplars of three categories of mathematics problems — puzzles, investigations, and applications— must be represented among the best pieces.
Writing portfolios must contain work in specific genres. Samples of students' writing, selected from work completed over the course of the academic year, must include (a) a best piece of writing; (b) a letter from the student to the evaluator; (c) a story, play, or personal narrative; (d) a review of an event, book, or issue; and (e) some pieces of writing from classes other than language arts or English. Each final paper must be accompanied by dated drafts that show the progress of the work.
Classroom teachers score each piece of work contained in both of these portfolios using state-generated scoring rubrics. (In the summer of 1994, the language of the writing scoring rubric was simplified, and a prompt was added to help the scorer decide whether student work should be ranked on the upper or the lower part of the rubric.) A random sample of these portfolios is subsequently rescored at the state level, and teachers receive feedback from state scorers regarding the quality of their students' portfolios.
To facilitate use of the scoring rubrics, the Vermont Department of Education provides several training sessions each year to teachers through regional training networks.
Maple Leaf Middle School
One of the earliest participants in the portfolio assessment system is Maple Leaf Supervisory Union (MLSU), and Maple Leaf Middle School provides a lens through which to examine Maple Leaf teachers', students', and parents' responses to Vermont's portfolios.
In 1994-95, Maple Leaf Middle School served about 315 students in grades 6, 7, and 8, 98 percent of whom were white. Because MLSU does not operate a public high school, about 95 percent of the middle school students go on to attend the local private high school (the tuition for which is paid by MLSU).
Impact of Vermont Portfolios on Teachers, Students, and Parents
Although Maple Leaf Middle School teachers included in this study expressed doubts about the use of portfolios as an assessment system, most agreed the portfolios are pedagogically useful in some ways.
The 8th-grade mathematics teacher had not integrated portfolios into his classes for one main reason: he worried that because the required portfolio tasks stress writing, students may lose basic computational skills (such as the ability to calculate fractions and to remember multiplication tables). In addition, he expressed doubts about the reliability and validity of the portfolios as an assessment instrument. He was concerned that, in the future, portfolio scores may be used for high stakes accountability purposes.
On the other hand, this same teacher said that the portfolio tasks have helped him better understand children with limited facility in mathematics. In addition, he believes that the kinds of tasks required for Vermont's mathematics portfolios are the kind that help students learn about the application of mathematical concepts.
In contrast with the 8th-grade teacher, Maple Leaf's 7th-grade mathematics teacher was sufficiently impressed with mathematics portfolios as a pedagogical tool and had incorporated them into her daily teaching routine. However, she, too, expressed concerns regarding the emphasis on writing and said that she spent a considerable amount of time teaching writing (as opposed to math) skills.
In 1994-95, this teacher continued to use the portfolios, but with more misgivings. She concluded that teaching her students writing skills was taking time away from the teaching of mathematical skills; she was not able to devote as much time to teaching certain mathematical concepts as is required for students to truly grasp the concepts. In addition, she felt that students who were good in mathematics but not in English were beginning to develop negative attitudes towards mathematics.
The MLSU director of curriculum noted that these mathematics teachers are skeptical not so much of the portfolios per se, but of the NCTM standards — which stress communication and writing — in the mathematics scoring rubrics.
In contrast with the mathematics teachers, the language arts teacher was consistently positive about the language arts portfolios in both years. She believes that the portfolios are a powerful pedagogical tool that help teachers become better teachers.
This teacher's pedagogical approach now centers more on guiding students to think, to share thoughts through peer-conferences, and to edit during the writing process. She particularly likes the voice dimension of the language arts scoring rubric, as the emphasis on this dimension of writing has especially benefitted her female students. In addition, she was not at all concerned about scoring reliability.
However, in 1994-95, this language arts teacher and another one expressed concerns regarding the articulation of language arts standards and curriculum with the local private high school. The high school had complained that Maple Leaf Middle School graduates did not possess good skills in grammar, syntax, and spelling. Therefore, in 1994-95, these teachers began placing more emphasis on these components of writing.
In both years, students' responses to the portfolios were a blend of enjoyment and indifference. Students said that they enjoyed completing assignments for their writing portfolios. They spoke of writing more than they ever had before, and said they found the challenge stimulating. One student said, "You have to think more [about your work] to get a better grade." However, students said that the mathematics tasks were challenging but not "much fun," and they typically did not use the scoring rubric to evaluate their own work.
Teachers also noted that students felt vested in their work and were motivated to complete their portfolio assignments. One language arts teacher said that writing had become a habit with her students, and they did not think of it as being a chore.
In both years, parents interviewed said that they liked their children's portfolios but did not fully understand the purpose of the portfolio system. In 1994-95, the school board member interviewed expressed some strong reservations about the scoring reliability and content emphasis of the portfolio system; he believes that the scoring is too subjective and the portfolio requirements underemphasize rote content knowledge.
Maple Leaf Supervisory Union
Despite being cognizant of the technical problems associated with the portfolio system, MLSU official remain enthusiastic about its pedagogical implications. They have encouraged the use of portfolios at all grade levels. (In fact, according to one official, 4th- and 8th-grade teachers feel that they should be sharing the responsibilities of the portfolio system with teachers at other grade-levels for the state-wide system, largely because of the amount of work involved in instructing students in how to complete performance-based portfolio tasks.)
District officials, nonetheless, said that they realize that a tremendous investment of time, money, and support is required to implement the portfolio system. Furthermore, the new superintendent noted that the community will not continue to buy into the portfolio system if it is not used for school accountability.
Future Plans
The Vermont Department of Education is developing content and performance standards for its Common Core Framework to guide further development of its assessment system. The department also plans to extend the assessment system to other grade levels and to additional subject areas. MLSU officials also hope to incorporate the portfolio system at all grade levels.
However, teachers and officials realize that it will be difficult to achieve some of the goals of the portfolio system for several reasons. For example, it is difficult to establish a performance baseline to monitor student progress because the quality of the portfolio tasks teachers design is not uniform. In addition, the opportunities to learn are not uniform across the state.
Until these issues are addressed, the state's intention of monitoring student outcomes and helping districts evaluate their programs must remain on hold. On the other hand, because of teacher involvement in defining the system, portfolios are becoming more widespread and more accepted as a useful classroom instructional strategy.
4The Ni?os Bonitos principal who initiated the school's comprehensive restructuring process was chosen for the position of Principal in Residence with the U.S. Department of Education. As a result, the school acquired a new principal in 1994-95, the second year of this study.
-###-
[Chapter 3: Case Study Summaries Part 1 of 5] [Chapter 3: Case Study Summaries Part 3 of 5]