![[Contents]](/icons/up.gif){kind=icon}
[Chapter 8: Assessment Reform: Findings and Implications Part 1 of 2]
The myriad influences of performance assessments on teaching are complemented by their influences on student learning
(see Exhibit 7-3). The impact of performance assessments on students is due as much to the content of performance-based assignments as it is to the process of assessment itself. Principals, teachers, students, and parents spoke primarily about the influence of performance assessments on students' motivation to learn and on their writing and thinking skills.
Impact on Motivational and Social Skills
Many interviewees reported that students exhibit a greater motivation to learn and a greater amount of engagement with performance tasks and portfolio assignments than with other types of assignments. Both teachers and students mentioned that, in large part, this effect is due to the sustained attention and effort students must invest in their educational endeavors, as they simultaneously define the parameters of their work and determine its quality.
At several schools, including Hudson High, Maple Leaf Middle, Ni?os Bonitos, Cooper Middle, Breckenridge Middle, Crandall High, Ann Chester Elementary, Thoreau, Sommerville, Noakes, and Windermere, teachers noted that students are more motivated to learn with performance-based tasks and writing assignments than with homework exercises out of a textbook. For example, one Crandall student described performance-based assignments as "self-motivated learning." Student engagement is particularly true in the case of the portfolio writing systems, such as the one in Vermont, where students are encouraged to reflect and write personal opinions.
Ironically enough, at Manzanita High some students reported that the on-demand assessments did not challenge them to perform at their highest levels. They said that the tests were better suited for "low performing" students, meaning that the assessments did not require much intellectual effort.
Teachers at Maple Leaf Middle, Cooper Middle, Crandall High, Noakes Elementary, and McGary Elementary also reported that students no longer dispute the grades they receive on their assignments as much as students did in the past, and accept their assessed level of performance. Teachers attribute this effect to students' understanding of the scoring criteria; teachers in all of these schools share scoring rubrics with their students and encourage self-assessment.
Although teachers at ten schools noted that they were assigning more group-based projects to their students than they had previously, teachers at only three sites ? Cooper Middle, Ann Chester Elementary, and Sommerville High ? specifically noted that their students' ability to work cooperatively in groups had improved as a function of such performance-based group assignments. (However, several students and teachers noted that although most students enjoyed working together in groups, some also complained that they had to be responsible for other students' work.)
Writing and Thinking Skills
Interviewees at six sites ? Hudson High, Maple Leaf Middle, Ni?os Bonitos Elementary, Cooper Middle, Breckenridge Middle, and Thoreau High ? indicated that students are improving their writing skills and habits as a function of completing written assignments. (At most of these schools, one important focus of the assessments is the development and improvement of writing skills.)
In schools where assessments are geared towards evaluating research and problem-solving skills ? Hudson High, Cooper Middle, Ann Chester, Thoreau High, Sommerville High, and Windermere ? students and teachers reported that students have acquired good research and analytical skills in the process of completing assessment tasks. Students are better able to use resource materials for projects and also have developed project presentation skills, such as the ability to summarize their work for an audience.
However, the development of a good base of knowledge (better than that of students not exposed to performance assessments and attendant instructional techniques) was mentioned by teachers at only five sites ? Hudson High, Cooper Middle, Ann Chester Elementary, Sommerville High, and Noakes Elementary. Furthermore, student acquisition of the ability to generalize the knowledge and principles gained in one domain to another domain as a result of performance assessments and performance-based instruction was mentioned only by interviewees at Crandall High. (Indeed, several teachers agreed that teaching students to apply knowledge and skills across domains remains one of the most difficult pedagogical tasks.) On the other hand, the generalization of writing skills across different subject areas was mentioned by several teachers.
Most salutary effects on student learning were reported by respondents engaged in portfolio assessments and extended performance task assessments. Our data indicate that portfolios and extended performance tasks invite student and teacher engagement on a sustained basis, and, hence, are much more naturally integrated into instruction. Portfolios, in addition, provide an over-time record of student progress and achievement and allow reviews and revisits of old materials on a regular basis. Thus, they provide a powerful means for monitoring students' educational progress. Students themselves testified to how much "fun" it is to review their work and to see their academic progress over the year.
Impact on Children in Special Education Programs
The impacts of performance assessments on children in special education programs is not easy to assess. Because of the diversity of the categories and formats of assessment tasks used at the sample sites, and because of the diversity among the children served in special education programs (SEP), it is not feasible to report coherent findings regarding the impact of performance assessments on these students. Nonetheless, several teachers did talk specifically about the effects of assessments on some of the students in SEP.
Teachers at five sites, including Maple Leaf, Ni?os Bonitos, Cooper Middle, Noakes Elementary, and Park Elementary, noted a positive impact of assessments on some of their children in SEP. Respondents at these schools said that the use of portfolios and the stress on performance-based tasks has resulted in increased school success and learning for some of these students. For example, Cooper teachers noted that their children in SEP experience success with some performance assessments because many of these students are good with verbal skills and like performance-based projects that allowed them to do demonstrations and oral presentations. However, Cooper's special education teacher also said that assessments that place less emphasis on "show" also must be devised to capture the performance of the students in SEP who are shy and not verbal.
At both Ni?os Bonitos and Park Elementary, performance assessments were employed in order to individualize instruction (as much as teachers could) and also to better serve the needs of disadvantaged students and students in SEP. Hence, Ni?os Bonitos and Park Elementary teachers have used performance assessments primarily as diagnostic tools. In this regard, the use of performance assessments has been beneficial children in SEP.
Teachers at several schools also noted negative effects of performance assessments on their students in SEP. Interviewees at Maple Leaf, Cooper, Windermere, Westgate, and Noakes said that their students in SEP have difficulties completing performance tasks, as these children find some of the tasks to be too complicated. Windermere teachers and parents also found that some of their students in SEP suffer a sense of failure and frustration after their oral presentations (because they were assessed by strangers who were not necessarily aware of the students' disabilities.) Similarly, teachers at Westgate and Walters remarked that their students in SEP experienced a sense of failure after taking the on-demand assessments, as on-demand situations were too complicated and stressful for them.
At Breckenridge, in order to counter the possible negative impact of portfolio tasks on their students with disabilities (and bad publicity for the school, since assessment results are used for accountability purposes), teachers admitted to completing some of these students' portfolio assignments.
Our data show that several barriers must be overcome before performance assessments can have a positive impact on teaching and learning. Some of these barriers have already been discussed in Chapter 6. In this section, we describe the barriers perceived by teachers who are earnestly attempting to appropriate performance assessments and to change the pedagogical processes in their classrooms and, in some instances, also by teachers who like the idea of performance assessments, but have not appropriated them.
The two types of barriers to appropriation and effective classroom use of performance assessments are:
Concern over the lack of time to adequately plan and score performance assessments and to use the information emanating from the assessments was voiced by virtually all teachers included in this study
(see Exhibit 7-4). Despite heroic efforts on the part of teachers who had appropriated the assessments and modeled their classroom instructional practices on these assessments, the burnout and stress generated by the lack of time could not be avoided. At Sommerville High, teachers mentioned that they had experienced this problem, but had rectified it by obtaining from their principal an additional joint planning period. In addition, Pacesetter provides teachers with a curriculum, instructional units, and curriculum-embedded assessments, reducing (though not eliminating, according to Sommerville teachers) the amount of time teachers must spend designing their own assessments.
Teachers also mentioned a lack of time for participating in professional support and development activities focused on how to effectively integrate performance assessments into instruction. (In many cases, as discussed in Chapter 6, such professional development opportunities do not even exist.) This concern loomed large at Cooper, Breckenridge, Crandall, Ann Chester, and Windermere. In fact, Cooper teachers came to the conclusion that they did not possess the expertise and the know-how in teaching mathematics through performance-based methods, as their students were not learning mathematics as well as expected. Therefore, in 1994-95, they stopped teaching mathematics through thematic, performance-based methods and reverted back to traditional teaching methods.
Content and Performance Standards
A second major concern teachers voiced was the lack of clearly defined and understood content and performance standards. Teachers involved in using state-designed scoring rubrics were uncertain about how validly and accurately to judge student performance through these rubrics. Teachers at some state-level sites ? Maple Leaf, VT, Breckenridge, KY, and Crandall, OR ? expressed concern over the lack of clearly defined performance standards. (Teachers at Walters Middle and Manzanita High did not mention developing and scoring assessment tasks using any state-generated rubrics.) For example, at Crandall High in Oregon, some teachers were enthusiastic about the use of portfolios and scoring rubrics, but were unsure about what constituted an acceptable level of performance in student work for outcomes such as conceptual understanding and effective communication aspects of the mathematics and science scoring rubric. At Manzanita High in Arizona, the problem of performance standards was exacerbated by the fact that Manzanita teachers do not believe the ASAP assessments reflect the breadth of the curriculum teachers are supposed to teach.
Oregon, Vermont, and PWC are still defining curriculum frameworks and desired student outcomes or aligning the outcomes with concrete student performance benchmarks within the assessment system. In addition, teachers at the local level do not have sufficient information regarding the changes in the state's or district's curriculum frameworks. For example, in the case of Maryland, the state director of assessment expressed the view that Maryland teachers are still not sufficiently aware of the learning outcomes targeted by the state's assessment program.
At Hudson High and Cooper Middle, teachers were concerned about the rigorousness and fairness of the scoring criteria they had devised; they mentioned that they had little experience and professional training in designing scoring rubrics, and, hence, were not sure whether the rubrics were "good enough". However, teachers at some schools individualize performance standards and are not as concerned about scoring procedures.
A related concern was "sacrificing content to teach process;" several teachers and administrators, as well as parents and community members, believe that students are not being taught enough of the "basics" including multiplication tables, grammar, vocabulary, and "facts." For example, one Maple Leaf school board member was worried that students are not leaving the school system with ". . . a reservoir of basic information and education that they didn't have when they first started [school]."
Teachers at Manzanita High, Maple Leaf Middle, and Westgate Middle were especially concerned that the content of mathematics assessments may not be rigorous enough. For example, one Manzanita mathematics teachers said, "The 12th-grade math assessment covers only basic math skills. In 1994, one problem required that students graph a line, but no other 1994 and no 1993 problems required students to perform any algebra, geometry, trigonometry, or calculus."
Maple Leaf teachers believe that the emphasis in Vermont's mathematics portfolios on communication and writing skills interferes with the teaching of mathematics skills. Westgate teachers believe that this same feature of the Applications Assessments in math may be inappropriate for their students. One teacher expressed especially strong reservations about "all this writing in math" and is not certain that it helps students learn math skills and concepts. (Another Westgate math teacher, however, expressed positive views about the math assessment, noting that it is aligned with the standards developed by the National Council of Teachers of Mathematics.)
In several other schools, however, the National Council of Teachers of Mathematics Standards are well accepted and have permeated the classroom. These schools include Ni?os Bonitos Elementary, Breckenridge Middle, and Ann Chester Elementary.
At Sommerville, teachers feel that the Pacesetter curriculum, as presented by the developers, does not adequately incorporate the teaching of "math skills" into the task sets. Task sets comprise problems students work on in small groups. Teachers determine how they score and use these task sets.
The two strongest conclusions we can draw from these findings are that: (1) students are being asked to write and to do project-based assignments due, at least in part, to the use of performance assessments; and (2) students are more motivated to learn through project-based assignments then with rote exercises out of a textbook. In addition, the teacher's role has changed significantly in some schools; teachers are collaborating more than before and are taking the initiative to create new learning environments for their students.
All of the observed and reported changes in teaching and learning were mediated to a large degree by: (a) the type of the assessment task (e.g., portfolios or on-demand performance tasks); (b) the degree of integration of the assessment into the classroom (i.e., appropriation); and (c) the level of support provided to incorporate the assessment into routine classroom activities. Sites utilizing assessment systems comprised of portfolios and extended performance tasks exhibited the most extensive impacts on teaching and learning, while sites involved in implementing on-demand assessment systems exhibited the least number of changes.
The positive and intended changes in pedagogy are most evident for sites engaged in portfolio assessments because the portfolio format provides teachers and students some measure of control over the design of assessment products within a structure for documenting student progress on an on-going basis. In addition, assessments such as portfolios and extended performance tasks necessitate instructional change, as teachers must design and assign tasks that can be used for evaluative purposes. The use of portfolios and extended performance tasks, however, requires time and in-depth coverage of some topics and, therefore, may result in less-than-full coverage of the course curriculum.
Teachers, identified two major barriers to fully appropriating and effectively using performance assessments in their classrooms. The first barrier is a lack of regular time to devise, use, and score performance-based assessments. Teachers at practically all sites voiced concerns over how little time and support they have to do justice to the pedagogical reforms they have undertaken. A second barrier is that the linkages among content, performance, and assessment strategy are not wholly clear at the local level. Thus, at this point, the quality of assessment-influenced pedagogical changes is difficult to determine.
-###-
[Chapter 7: Cross-Case Analysis 4: Part 1 of 2]
[Chapter 8: Assessment Reform: Findings and Implications Part 1 of 2]