Investigating a Linkage Between Reward for Good Teaching and Improved Student Performance

Purpose

To investigate the relationship between faculty incentives and student learning, seven departments (sociology, chemistry, social work, nursing, management, modern languages, and special education) elaborated plans to establish learning goals, assist students in reaching them, and assess the result of their efforts.

Departments used FIPSE funds in two ways: to compensate faculty through release time or monetary stipends for their participation in the project (these funds were made available throughout the course of the activities), and to reward those whose students made significant learning gains by granting support for professional development activities such as travel or equipment purchases.

Innovative Features

The project was departmentally based and, although strongly supported by the administration, developed from the bottom up. Faculty from the seven participating departments planned independent projects that differed significantly in their methodologies, proposed outcomes, and rewards.

Nevertheless, the seven departmental projects shared three features. Each established an intervention strategy aimed at enhancing student learning; each provided one or more assessment methods to document learning; and each specified faculty incentives for participation.

An Advisory Committee composed of at least one member of each participating department and various administrators, including the Vice President for Academic Affairs, monitored the departmental projects and voted on fund expenditures. Despite the variations in goals and assessment programs, departmental representatives used the Committee's regular meetings to learn techniques to stimulate learning that might be useful in their own departments, and to obtain assistance with problems of measurement or morale.

The departments of special education and modern languages did not complete the project. The five remaining departments succeeded in documenting greater learning gains by students participating in the project than by their non-participating peers or pre decessors and received the reward funds.

The following department-by-department summary of the projects should give an idea of the range of goals and strategies:

Sociology--The principal strategies for improving learning consisted of measuring student achievement and improving advising. The project achieved this by closely monitoring the aggregate and individual progress of its majors through a variety of instruments and by establishing a system of intensive student advising whereby faculty fed information about progress back to the student.

Chemistry--Department faculty met regularly to discuss ways of bringing students to a higher level of performance in such areas as recording laboratory data and in curricular matters such as better integration of courses.

Management--In the experimental sections of 19 courses, instructors adopted a Teaching and Learning by Objectives approach adapted from the Management by Objectives philosophy. In each course, a control section was conducted according to traditional pe dagogies. With the help of their advisers, students in the experimental sections each prepared an Individual Learning Plan, which was then modified through discussion with their course instructor. In addition, faculty met with each student approximately three times per semester to review progress and satisfaction.

Social Work--The faculty's goal was to improve the direct practice skills of students. To achieve this, faculty provided a special laboratory in which students did exercises involving such techniques as role playing, case presentations with small-group brainstorming, and problem- solving.

Nursing--The Department developed a non-graded seminar to foster the socialization and professional development of baccalaureate nursing students and to help them to create a healing relationship and environment for their patients. Students were encouraged to develop and enhance their skills of reflection, to identify their feelings about interactions with patients, and to explore the relationship between accountability and responsibility.

Evaluation and Project Impact

Departments developed several means of documenting student learning improvements. Many were multidimensional, and all sought to reflect the different goals of each department.

Sociology--Student progress was measured through a departmental examination developed during the project and the ETS major field examination, both of which were administered at the entrance, midpoint, and end of the major. In addition, students comple ted a research project at the beginning and the end of the major and videotaped presentations at the beginning and the end of the core courses in the major.

Seniors' scores on the ETS major field examination were significantly higher for the classes involved in the grant than for previous groups, and scores on the departmentally developed inventory improved during the course of the project. Compared to the performance of classes before the project, student papers evidenced more complex research designs and better writing. Oral presentation skills also improved.

Chemistry--Faculty administered standardized examinations, a self-assessment survey, and a questionnaire. They reviewed student laboratory work and papers in courses taken early and late in the major. They probed students' knowledge of the principles and facts of chemistry; their writing skills; their ability to solve problems, apply mathematics and interpret scientific data; their experience with computers, instrumentation and chemical literature; and their career goals.

Over the course of the project, faculty observed the following changes compared to prior generations of students: students dramatically increased their computer skills; they showed greater independence in laboratory work; more majors engaged in research under the direction of faculty members; more students made presentations at annual research conferences and submitted papers that won recognition in national competitions; student scores on standardized exams rose; and oral presentations showed imp roved organization, clarity, research, and use of visual aids.

Management--Participating students were evaluated on whether they met the objectives they had determined jointly with faculty. Pre- and post-tests were administered to the experimental and control sections of each course. Faculty developed comprehensive examinations to assess aggregate student learning levels, and to compare the performance of students who were involved in the project with those who were not.

Overall, the different measures demonstrated that the project made a significant difference in student learning of course contents and the development of managerial skills. An attitude survey revealed that students felt the project had been helpful to them.

Social Work--The practice instructor and the field instructor conducted pre- and post- assessments of five practice skill areas among experimental and control students. Self-assessments by students in both groups were also compared. The practice skill s of students who had completed the experimental laboratory showed significantly greater improvement in three of five areas than those of the control group. As a result, skill building content for several social work courses within the practice sequence was enhanced. During the school's last accreditation review, the site visit team of the Council on Social Work Education praised the strength of the practice sequence. In addition, the Masters of Social Work Program was revised two years ago to institutionalize the skills laboratory created as an experimental part of the FIPSE project. Because of the laboratory's success, two required social work practice courses have been increased from three credit to four credit courses, each requiring a skills laboratory component.

Nursing--Clinical faculty and the leader of the experimental seminar evaluated the extent to which experimental and control students had progressed toward desired professional goals. The results demonstrated that the clinical performance of the students in the seminar--as opposed to course grades or performance on other measures--was significantly higher than that of the control group. In clinical settings, seminar students exhibited greater self-awareness, responsibility and accountability than the controls.

In addition to the above assessments of student performance, a cross-departmental survey analysis of the entire project was conducted by members of the sociology faculty and is discussed below.

Lessons Learned

Departments made various uses of FIPSE funds. Some used funds at the end of the project to reward faculty for improved student learning with travel monies or equipment purchases. On the other hand, some used funds to compensate faculty for participation in the project (i.e., release time to teach an experimental laboratory section). Still others used part of their funds to finance student travel to conferences.

Given this range of uses of incentive funds, and given that there was no attempt to compare the performance of students of faculty who had been promised incentives with that of students whose instructors did not participate in the project, it is not possible to determine precisely the impact of faculty rewards on student learning.

The faculty themselves seemed to think that the effect of the rewards was minor. The survey of participating faculty conducted by the cross-departmental analysis team revealed that only one respondent felt that the prospect of rewards had constituted the entire motivation for participation in the project. A slightly larger number indicated that they were not at all influenced by incentive funds. Seventy-seven per cent of the respondents stated that the prospect of rewards had played some role in motivating their work.

In responses to the survey and to informal polling, many faculty reported that their participation in the project derived principally from a sense of professional obligation; others stated that they had been motivated less by the monetary incentives than by the formal commitment implied by the grant; and still others mentioned that the funds were too insignificant and too far removed in time from the work they were intended to reward to serve as major motivators. On the whole, the prospect of rewards seems to have provided some but by no means the entire motivation for faculty efforts to improve student learning.

In addition, the cross-departmental analysis yielded the following insights into the process of improving teaching and learning on a departmental basis:

1. Although all attempts to improve learning should identify goals, it is desirable for such goals to retain a certain flexibility, since the ability to continue to discuss and refine goals in the course of the project may contribute to faculty ownership of a program. If the program wishes to elicit broad faculty participation, a democratic, decentralized organization seems to foster satisfaction and retention.

2. A multidimensional approach to assessment is valuable not only because it ensures reliability and validity but also because a battery of measures can be scaled back, revised or augmented if practical problems or ideological shifts warrant. Home grown measures of learning created collaboratively by the faculty produce the sense of involvement that is crucial to faculty satisfaction with and participation in a project. Progress toward discipline-based cognitive learning goals may be easier to document than improvements in generic skills such as writing and speaking.

3. Student attrition in experimental groups may constitute a significant problem. Planners should allow for student incentives to ensure retention. Feedback to students is important for assessment projects to yield full fruit.

4. Large-scale changes in pedagogy may be less effective in improving learning than joint faculty reconsideration of general goals and increased opportunities for faculty/student contact.

5. If the program involves faculty in performing clearly delimited additional tasks (such as developing or teaching an extra course) it is more beneficial to remunerate faculty for their efforts by giving them release time or extra compensation than to promise end-of-program incentives.

Project Continuation

Even though participating departments did not continue all aspects of their individual projects, various strategies and activities originating from FIPSE funding continue. For example, the vice president for academic affairs has continued support of the sociology department's assessment efforts by paying for use of ETS examinations administered to graduating seniors. Part of at least one department meeting every year is devoted to a review of the ETS exams.

The "Teaching/Learning By Objectives" project implemented in the department of economics and management seems to be having some long-lasting effects. Some faculty members, particularly those who originally participated in the project, have embraced the Management By Objectives (MBO) philosophy as a methodology and as an instrument in their teachings. The attempts on the part of the project director and the department chair to disseminate the basic ideas of "Teaching/Learning by Objectives" (TLBOs) through briefings, individualized discussions, committees and departmental retreats have enticed some faculty members to internalize the basic TLBOs philosophy and integrate its ideas into their course syllabi, their teaching styles, and their relations with students.

Moreover, the departmental student advising system is now almost fully institutionalized. Advisers spend a great deal of time with their advisees, try to help them in developing both learning and career objectives, and guide them toward achieving their objectives.

Other continuing activities introduced during the FIPSE project include:

• Faculty still meet once a semester to discuss the progress of each of the majors and have expanded this practice to include physics and science education majors.
• One general advising meeting with the students is held each semester. Most of the faculty are present during these sessions, and the students are made aware of the requirements of the major, the educational goals and when various courses are offered.
• The college continues to emphasize computer skills, especially spreadsheet skills, which are now being utilized in the analytical chemistry and physics courses in addition to the physical chemistry course. Computer interfacing is slowly becoming more significant in laboratory courses, and heavy emphasis is placed on training students in the skills of data recording in the sciences.
• Microscale methods continue to be used in the organic chemistry course.

Available Information

Further information may be obtained from:

R.N. Keogh
Office of Research and Grants Administration
Rhode Island College
Providence RI 02908
401-456-8228

[University of Nebraska] [Table of Contents] [V. Improving Teaching and Learning]

FIPSE Home