Portfolio Writing Assessment in Student Placement

Purpose

In recent years, testing student achievement has dominated the field of educational assessment, and this has raised fundamental questions about test validity and reliability. As the directors of this project note, there has been a continuing tension be tween the need to examine students' writing for placement and achievement, and the faculty's dissatisfaction with the means of doing so.

Miami University responded to this problem by becoming the first in the nation to explore portfolio assessment as a means of placing students in freshman composition courses. The portfolio method replaced standardized tests that did not judge actual writing, and one-shot, timed proficiency essays whose evaluation raised continual questions of accuracy and bias. Miami argued that a single piece of writing, produced under test conditions, has questionable content validity since the writing behaviors demonstrated in one essay do not constitute a representative sample of such behaviors. To enhance validity in assessing student writers, multiple pieces of writing composed at different times have to be considered.

Innovative Features

Miami embarked on its groundbreaking work by heeding the shift in composition instruction from transmitting knowledge about good writing to having students draft and revise. The project tackled hitherto unexplored questions of how portfolio assessment compares to holistic assessment of single essays, and how the gender of writer affects ratings. In addition, the reflective letter that students write to introduce the portfolio has brought forth a rich source of data for interpretation.

Several members of Miami's English Department consulted with local high school English teachers to develop an experimental portfolio program that would attract student participation. The Miami Admissions Office supplied listings of all freshmen and high school students likely to attend the University. Project staff designed guidelines for portfolio submissions and rating criteria, and trained portfolio raters. Staff then invited students to assemble their own portfolios, processed and scored them, and selected outstanding examples for new students to use as models.

Evaluation

Staff compared the new portfolio method to the traditional testing procedure by awarding placement and college credit to students admitted through the portfolio method while continuing to administer proficiency essays. The relative merits of the two measures were studied by tracking selected students who had been placed in courses based on either a high score on their portfolio and a low score on their proficiency essay, or a low score on their portfolio and a high score on their proficiency essay. Experienced raters scored these students' written work to learn which method better predicted student performance. In addition, to learn more about the strengths and weaknesses of the new method, staff collected self-report data from participating students, teachers and raters.

Project Impact

In a remarkable show of consensus and risk-taking, the Department of English voted to replace the proficiency essays with the new portfolios for the purpose of awarding advanced placement and credit. Consequently, data on participation from 1990 to 1992 showed an increase in the number of students submitting portfolios from 277 to 465.

In the first transitional year, students could achieve advanced placement and credit either by submitting a portfolio or by writing the proficiency essay, both of which were scored on a 1-6 scale. Eighty-five and a half percent of first and second readers of the essays recorded scores no more than one point different from one another, while 85.8 percent of first and second readers of portfolios recorded scores no more than one point different from one another. Thus, reliability in portfolio rating compares favorably to reliability in single essay rating. In addition, the portfolio method enhanced validity, since judgments derived from multiple and diverse writing sources.

The portfolios were found to have great appeal for faculty and administrators alike. Surprisingly, when reading time and rater fatigue were measured for both the portfolios and the proficiency essay, it was the essay that elicited more comments about fatigue, even though it took on average only two or three minutes to read, compared to seven to ten minutes for each portfolio. Raters worried that reading the same topic essays for hours distorted their judgment and sense of fairness, but this was not the case when reading the variety of writings in the portfolios.

A follow-up survey of raters suggests that the student s reflective letter introducing the portfolio had a powerful and personalizing influence on the rating of the whole. In fact, raters found the letter to be the most revealing about student attitudes toward their own writing-- a fascinating range of boastfulness, self-effacement, wit, and rambling." Staff speculate that the reflective letter positively affects reliability because it establishes the authority of the writer early, and prepares the raters to score the remainder of the portfolio. What is known is that both male and female raters perceived female students as significantly more successful than males in writing these reflective letters. Additional study is needed since portfolios and gender issues were found to interact in complex and subtle ways.

Project Continuation and Recognition

Portfolio assessment of writing has been institutionalized by the English Department at Miami. To assure continuity in program activities, other members beyond the original staff were enlisted into a Portfolio Assessment Sub-Committee on Composition.

In the third year of the project, Miami held a conference on portfolio assessment attended by 400 participants, many of whom were high school teachers and researchers. Teacher interviews and reports of instructional activities suggest the portfolio met hod may have affected the quality of composition teaching and writing in Ohio high schools.

Other institutions, including the University of Michigan and the University of Mississippi, have consulted with Miami's staff and begun their own placement portfolio programs.

Available Information

Information about the project is available from:

Donald A. Daiker
Jeffrey Sommers
Gail Stygall
Laurel Black
Department of English
Miami University
Oxford, OH 45056
513-529-7110

Besides the final report to FIPSE, the following project publications are available:

New Direction in Portfolio Assessment. Donald Daiker, Jeffrey Sommers, Gail Stygall, and Laurel Black, eds. Portsmouth, NH: Heinemann/Boynton-Cook. (With special chapters on gender: "Gendered Textuality", and "Writing Like a Woman and Being Rewarded For It? Gender, Assessment and Reflective Letters from Miami University's Student Portfolios").

"The Challenges of Rating Portfolios: What WPAs Can Expect." Jeffrey Sommers, Laurel Black, Donald Daiker, and Gail Stygall. WPA: Writing Program Administration. Vol. 17, Nos. 1-2 (Fall/Winter 1993).

Handbook for Portfolio Assessment: A Program for College Placement. Jeffrey Sommers, Laurel Black, Donald Daiker, and Gail Stygall. Oxford: Miami University, 1992 (or available through ERIC, Document #ED350616).

"Using a Writing Portfolio for Placement in College Composition," Jeffrey Sommers, Donald Daiker and D.J. Hammond. Notes from the National Testing Network in Writing. 10 (December 1991) 2-3.

[University of Connecticut] [Table of Contents] [University of Wisconsin at Madison]

FIPSE Home