Rhode Island: Measuring Contributions Support Professionals Make to Learning

Establishing specific goals helps students and professionals make progress

Teacher and her student sitting at a desk looking over a notebook

Teacher and student go through class notes together. Photo Credit: U.S. Department of Education

Amy Manchester has worked as a speech language pathologist for 12 years. Her caseload at Richmond Elementary School in the Chariho Regional School District in Rhode Island includes students with autism, dyslexia, language disorders, and other disabilities.

Her job is to make sure her students can access the curriculum. On any given day, that could mean teaching a non-verbal student to use a speech device to ask questions and make comments during class. Or, it could mean teaching a student who is on the autism spectrum the rules of conversation and methods to interpret social cues and body language to help them participate in classroom discussions.

She knows the work she does affects her students’ learning. Until this year, however, her performance evaluation did not put a lot of emphasis on her effectiveness.

But, during school year 2013­-2014, Rhode Island piloted a process by which support professionals—library media specialists, school nurses, reading specialists, counselors, psychologists, social workers and language and speech pathologists—were evaluated based, in part, on whether their students achieved specific learning goals and outcomes.

In the past, for example, Manchester might have had a goal of helping students improve their ability to understand words as they were being spoken to them. But the amount of improvement did not affect her performance evaluation. Now, she has specific numerical targets (called student learning outcomes, or SLOs) for lowering the number of pronunciation errors a student makes and for increasing his ability to identify distinct sounds within a word. SLOs are long-term academic goals established for groups of students, which help them understand their progress and helps support professionals like Manchester understand what work still has to be done. Another of Manchester’s goals was to collaborate more with classroom teachers. “A lot of times it is easy to do our work in isolation and the collaboration piece is key [to student learning]”, she said.

Manchester and other support professionals also need to set specific targets for increasing access to student learning (called student outcome objectives, or SOOs) to measure their impact in a different way. Reducing truancy, for instance, is an SOO.

The formal title of the process is the Rhode Island Model Support Professionals Evaluation and Support System. It is an extension of the State’s redesigned teacher evaluation and support system, which puts greater emphasis on student learning and goal setting. Both systems were developed with support of the State’s Race to the Top grant from the U.S. Department of Education.

Measuring the Impact of Support Professionals on Learning

The graphic shows a process for identifying and using SLOs or SOOs. The header of the graphic says: Identifying and using SLOs or SOOs to guide and improve practice involves four steps. And the process shown below includes four steps: Prepare, Develop, Implement, and Reflect.  The first step (prepare) includes: Review standards, units of study, previously provided services, and effect on students’ access to learning, Review the assessments used for assigning grades and monitoring progress, Review past student data, and Determine which services and skills should be prioritized in current school year. The second step (develop) includes: Get to know students by collecting and analyzing data, Re-evaluate which services or skills those students need, Draft SLOs/SOOs in collaboration with supervisor or evaluator and colleagues and submit, and Receive approval for SLOs/SOOs from supervisor or evaluator. The third step (implement) includes: Monitor student learning or progress toward outcomes, Discuss progress with colleagues and evaluators, Revise support and interventions in collaboration with evaluator if students are not progressing as expected, and Collect, analyze and report on SLO and/or SOO results. The fourth step (reflect) includes: Review results with evaluator and Reflect on results to improve practice.

Support professionals go through a four step process to establish SLOs and SOOs. For more information, see the Rhode Island Model Evaluation and Support System Guidebook for Support Professionals.

“The roles of the support professionals are different from those of teachers, but what they do has an impact on student learning—whether it is attendance, social-emotional issues or health,” said Lisa Foehr, director of the Office of Educator Quality and Certification for the Rhode Island Department of Education (RIDE).

RIDE convened a small working group of representatives of each profession and conducted focus groups around the State to design the system. In talking to support professionals across Rhode Island, State officials learned that the performance of many support professionals wasn’t being evaluated at all, Foehr said.

Under the new system, Rhode Island support professionals collaborate with their supervisors to establish SLOs and SOOs. Both SLOs and SOOs should be “specific and measurable, based on student information, and aligned with standards, as well as any school and district priorities where applicable,” according to the System’s guidebook. Depending on their role, some support professionals will identify two SLOs, two SOOs, or one SLO and one SOO.

The new system goes into full use for the school year 2014-2015.  It requires school districts to evaluate support professionals in three areas:

  • Professional practice, which includes collaboration with colleagues and the quality of services delivered.
  • Professional responsibilities, which includes fulfilling school responsibilities, communications, and professionalism.
  • Student learning.

Setting Goals and Measuring Progress

The student learning component will count the most, Foehr said. But support professionals contribute to learning in a variety of ways, some directly, some indirectly, and some both. A library media specialist, for example, can teach students research skills and a school nurse can teach good health practices. Recognizing the diversity of ways in which professionals can impact student growth, the support professionals are expected to be actively involved in collaborating with their supervisors to determine how their effectiveness is to be measured.

Teacher sits at the front of the classroom on the floor with two students and they go over work.

Teacher leads classroom in group work. Photo Credit: U.S. Department of Education

A social worker, for example, might set a goal of reducing the amount of class time chronically truant students miss by checking in with them more often to see how they’re doing and establishing stronger relationships with their families to understand student challenges and develop strategies to promote more consistent attendance.

Mary Ann Canning McComiskey, a veteran social worker at Lincoln High School in Lincoln, is on the working group that designed the new approach. One of her objectives this year is helping her special needs students regulate their behavior in class, follow directions and understand social cues, so that they are less disruptive in class and interfere less with their own as well as their classmates’ learning. Many of the students she works with have social cognition and behavioral issues. To measure her students’ progress, she created an observational tool that allows her to quantify changes in their behavior.

Encourages Reflection

The goal-setting process expects “you to be reflective in your practice and maybe change or modify the interventions,” she said. It also helped her set priorities for how she uses her time.

She acknowledged that many support professionals may be fearful of the new evaluation process. But “it’s not as scary as people think it is,” she said. “I find that it really helps to document the importance of our roles in schools.”

Patrick Cozzolino, a school social worker at Westerly Middle School in Westerly, said one of his objectives this year is to reduce bullying. So, he is coaching teachers on strategies for improving behavior and how to gather data on students’ progress. The process has challenged “me to look at things through a data-driven perspective and to take on more of a leadership role,” he said.

Some supervisors appreciate the consistency in rubric language across the various fields.  “I have a clearer understanding of what the expectations are across each discipline and this allows for stronger feedback and a more aligned, common vision for support professionals,” said Andrea Spas, assistant director of special education for the Chariho Regional School District.

This graphic shows an example of an outcome objective. The title of the student outcome objective is "Prevention Student Outcome Objective."  Content Area -- None Grade Level -- K-5 Students -- All Students  Interval of Service -- One School Year  For the first section, the essential question is: What is the most important outcome that will enable students to have better access to education through your services? The main criteria for this essential question is "priority of content" and there are two elements: objective statement and rationale. The objective statement is: Improve the overall health, wellness, and safety of students. The rationale includes four statements:  A constant aspect of my work is supporting students’ overall health, wellness and safety. I will focus on three major areas of my role including immunization records, vision screenings, and in-service trainings.  Immunizations are a key to primary prevention of disease from infancy through adulthood. I am in a critical position to create awareness and influence action related to mandates and recommended immunizations in the school community. Routine screening is a prevention measure to help children who need services gain access to them in order to prevent the occurrence of more severe problems later.  By providing high quality in-service trainings to staff and families it will help bring awareness to the many health problems of our students while providing strategies and coping mechanisms to those that interact with students.   For the second section, the essential question is: Where are my students now with respect to the objective? The element is: Baseline Data/Information. The description is: There are currently 400 students enrolled at our school. Last school year 100/400 or 25% were non-compliant with state requirements. Of the 100 non-compliant students 35 were in Kindergarten, 20 in 1st grade, 10 in 2nd grade, 10 in 3rd grade, 10 in fourth grade, and 15 in 5th grade. I administered a fall survey to all staff to gauge their level of confidence with understanding of how to handle the daily needs and potential crisis of our diabetic students. No staff responded feeling very confident and only 10% were confident.    For the third section, the essential question is: Based on what I know about my students, where do I expect them to be by the end of the interval of service? How will I measure this? The main criteria for this section is "Rigor of Target" and “Quality of Evidence.” Next to Rigor of Target, there are two elements: Target and Rationale for Target(s). Next to Target, there are two descriptions, one for Immunization and one for Staff Training. Description for Immunization: This school year we have 90/420 non-compliant or 21%. I will reduce the number of students not in compliance with state required immunizations to 67/420 or 16%. This is an overall improvement of 5%. Description for Staff Training: 100% of faculty will be somewhat to highly confident in their understanding and ability to handle the daily needs and potential crises of diabetic students. Next to Rationale for Target(s), there are two descriptions, one for Immunization and one for Staff Training. Description for Immunization: While 5% may not seem that rigorous of a target, looking at historical data will support this choice. For the past five years, the percent of non-compliant students has remained stagnant at 21%..This year I will focus my attention to our younger students in Kindergarten and 1st grade (the two with the most need). The number out of compliance will decline over time as I implement targeted strategies. Description for Staff Training: Last year we had two incidences where diabetic students collapsed because their blood sugar went too low. While 100% may seem too rigorous it is vital to the safety of students that the staff knows how to handle the needs of diabetic students. After initial training I will survey faculty, and those who still do not feel confident will receive additional training and by the end of the school year all faculty will feel somewhat to very confident in their understanding and ability. Next to Quality of Evidence, there is one element: Evidence of Source(s). Next to Evidence of Source(s), there are two descriptions, one for Immunizations and one for Staff Training. Description for Immunization: I will use Aspen (student information system) to track student records. I will enter all information in the fall and monitor the impact of my strategies. I will take note of any trends and use this information to drive additional outreach. I will be able to have a final record through ASPEN to provide evidence.  Description for Staff Training: I will use end-of-training staff surveys as an evidence source for this target. The last element is Strategies and includes two descriptions, one for Immunizations and one for Staff Training. Description for Immunization: Assessment of student immunization records, contact with parents through phone calls, notes, and school mailings. In addition to initial contact, I will follow up with families where students still have missing records except those who are exempt.  I will send home dual-language brochures that highlight the seriousness of vaccine-preventable diseases and have them in other places around the school including the front office where parents frequently visit. Description for Staff Training: I will present a highly engaging staff development at a faculty meeting in late fall and then continue with smaller or one-on-one training throughout the winter for those who need it.

Adapted from a longer version prepared by Rhode Island educators. For additional examples of SOOs and SLOs, please visit Rhode Island Department of Education.

One of the biggest challenges is identifying goals that best reflect what the professionals actually do to help students, as well as confirming that reaching those goals has a direct benefit for them, said Jessica Waters, Education Specialist in the Office of Educator Quality and Certification at the Rhode Island Department of Education who is working closely with support professionals to implement the new system.

For example, administering flu vaccines is part of a school nurse’s job, but counting the number of vaccines she administers is not a fair measure of her overall effectiveness. “To have more than half your evaluation based on a single function seems very inauthentic and unfair,” Waters said. “Broadening [student outcome objectives] so they reflect more of the work support professionals do is what we’re working on now.”

While the model is still under development, Waters and members of the working group are optimistic about its direction. “Support professionals are extremely excited that they’re finally being recognized for their important work in schools,” she said. “Very rarely did our principals talk with nurses about how they could support student learning; these conversations are now starting to happen,” Waters added.

 

Takeaways

  • Identifying goals that accurately reflect the full scope of a support professional’s job is a key challenge.
  • Clarifying expectations helps evaluators provide more meaningful feedback to support professionals.
  • Evaluators need to fully understand that support professionals contribute to student learning in a variety of ways. Rubrics and articulated expectations help.
  • Support professionals value the focus on goal-setting and increased use of data to help them improve their practice and increase their impact on student learning.
      

Q&A with Jessica Waters, Race to the Top Education Specialist

 Q. What kind of feedback have you gotten so far from support professionals?
 A. Support professionals are extremely excited that they’re finally being recognized for their important work in schools. They’ve spent a lot of time not feeling valued.

Q. What are the biggest challenges with implementation of the new model?
A. The big question we’re grappling with is how to make sure that student outcome objectives are truly representative of a support professional’s job. We are in the process of figuring out how to develop goals that reflect the full spectrum of their role.

Q. What changes are you noticing as this work unfolds?
A. Everyone in the building is now taking responsibility for student learning. Support professionals feel much more a part of the learning community.

 

 Resources