A r c h i v e d  I n f o r m a t i o n

Biennial Evaluation Report - FY 93-94

Chapter 313

Special Studies

(CFDA No. 84.159)

I. Program Profile

Legislation: Individuals with Disabilities Education Act (IDEA), P.L. 101-476, as amended, Part B, Section 618 (20 U.S.C. 1418) (expires September 30, 1995).

Purposes: The purposes of the Special Studies program are as follows:

Funding History

Fiscal Year Appropriation Fiscal Year Appropriation
1977 $l,735,000 1987 3,800,000
1980 l,000,000 1988 3,638,000
1981 l,000,000 1989 3,594,000
1982 480,000 1990 3,545,000
1983 480,000 1991 3,904,000
1984 3,100,000 1992 4,000,000
1985 3,100,000 1993 3,855,286
1986 3,170,000

Awards may be made to State and local education agencies, institutions of higher education, public and private nonprofit organizations, and private profit organizations when necessary because of the unique nature of the study.

II. Program Information and Analysis

Performance Indicators

This program conducts evaluation studies, including studies to assess (a) State and local programs in serving preschool children (Goal 1); (b) educational outcomes of students with disabilities including status of high school exit (i.e., graduation or dropping out) (Goal 2); and (c) the effect of education reforms on the achievement of disabled students (Goal 3).

Population Targeting

Studies are conducted on evaluation issues relating to the provision of special education and related services to infants, children, and youth, ages birth to 22.

Services

Federal Evaluation Studies

Study of Anticipated Services for Students with Handicaps Exiting from Schools

Longitudinal Study on a Sample of Handicapped Students

Key findings of the study are that:

The Center for Special Education Finance

State Evaluation Projects

State Agency-Federal Evaluation Studies Projects

Projects Funded in FY 1993

The Colorado Department of Education is studying the feasibility of developing a conceptual framework that can be used effectively to examine three configurations of special education services. The conceptual framework developed for the feasibility study will focus on teacher roles and responsibilities, curriculum, instructional methods, and environmental components (e.g., student-student interaction, classroom climate, time-on-task, etc.) in three instructional settings (i.e., services in general education classes with special education consultation, services in co-taught classes, and services in resource classes).

The District of Columbia Public Schools, in collaboration with the Department of Psychoeducational Studies at Howard University and the National Center on Educational Outcomes (NCEO) at the University of Minnesota, is examining the feasibility of using the NCEO conceptual model of educational outcomes and indicators to develop outcome measures for special needs students in the District of Columbia Public Schools. Through this current study, the District of Columbia Public Schools will serve as a test site for implementation of the NCEO model of outcomes and indicators for disabled students.

The Hawaii State Department of Health, in collaboration with the Hawaii University Affiliated Program, is conducting a feasibility study to determine the best ways to identify needs of families involved in early intervention programs under Part H of the Individuals with Disabilities Education Act (IDEA). The study will also evaluate the relative merits of different ways of tracking how well these needs are being addressed through the existing system of early intervention services.

The Kansas State Board of Education, in collaboration with the Center for Educational Testing and Evaluation (CETE) at the University of Kansas, is carrying out a study to determine the feasibility and validity of using existing statewide student assessment procedures and practices to test students with disabilities. Recent test data indicate that students with disabilities are under-represented in the Statewide testing initiative. The current study is exploring the reasons for this, determining the extent to which students with disabilities and the programs provided to them could benefit from the State testing program, and to recommend any necessary modifications so that the testing program can be effective for students with disabilities as well as for all students.

The New Hampshire State Department of Education, in collaboration with the Center for Resource Management, Inc. (CRM) is assessing the outcomes of high school students with disabilities in regular education placements, and identifying those factors associated with student success. The study is designed to address national and State initiatives that emphasize the need for increased accountability at the school level in monitoring student performance and outcomes, and assessing the positive and negative impacts of various factors.

The North Carolina State Department of Human Resources is investigating the inclusion of infants, toddlers, and preschoolers with disabilities, and their families, who receive early intervention services under Part H or Part B of IDEA, into the State's comprehensive early childhood system of services, called Smart Start. The fundamental question addressed by the study is "What happens to preschoolers with disabilities and their families as a function of community early childhood programs developed by the local partnerships?"

In February, 1990, Ohio implemented the Infant Hearing Screening and Assessment Program (IHSAP) requiring hospitals to use a questionnaire to identify infants at risk for hearing loss. Hospitals must then either provide hearing assessments of all at-risk infants or give their parents a list of facilities performing these assessments. The Ohio Department of Health is conducting a feasibility study to develop and test a conceptual framework for a full-scale evaluation of IHSAP that will answer the questions: "Is IHSAP successful in identifying infants who are at risk for hearing impairment? Is Ohio's early intervention system working to ensure that infants identified with hearing impairment are enrolled in services by the time they are 12 months of age?"

The Colorado Department of Education is developing a model for measuring social competence and is determining the feasibility of constructing a measurement system to evaluate progress toward social competence for students identified as seriously emotionally disturbed. Major activities include: development of a model for measuring social competence; development of items which incorporate the interactive relationship of the identified specific social intents of students and the requirements of the social context; and determination of the feasibility of constructing an instrument to measure progress toward social competence utilizing these items.

The Oregon Department of Education, in collaboration with Portland State University, is currently conducting a full evaluation of the State's Comprehensive Program Plan for Supported Education. This full evaluation is using information from observations, interviews and surveys to identify attitudes and perceived outcomes for students with disabilities who receive special education and related services in regular education settings. Under this new FY 1993 award, the Oregon Department of Education will determine the feasibility of extending the full study beyond an examination only of attitudes and perceptions to include measures of direct student outcomes, and compare and contrast the effects of supported education using outcome measures.

The Pennsylvania Department of Public Welfare, in conjunction with the Allegheny-Singer Research Institute, is evaluating the differential characteristics and effects of family-oriented approaches to early intervention. The purposes of the study are to: operationally differentiate among at least three family-oriented approaches (family-allied, family-focused and family-centered); and to relate these differences, in turn, to differences in child, parent, and family functioning. The study will seek to determine whether different family-oriented approaches do have differential outcomes and, if so, to discern some of the processes underlying these differences.

In 1991, South Dakota initiated a process of school restructuring throughout the State in response to articulation of the National Education Goals. The State intends that restructuring be achieved within the whole school and on behalf of all students, including those receiving special education services. In conjunction with this Statewide initiative, the South Dakota Department of Education and Cultural Affairs is conducting an evaluation study to examine the extent to which special education programs and students are involved in the reform efforts, and to determine the effects of the reforms on special education programs, services, instruction, personnel, and students.

The Vermont Department of Education, in collaboration with Trinity College and the University of Vermont, is carrying out an evaluation study to assess the impact of recent changes in State legislation and policy on local school policies and practices. The study will measure the impact of changes in local policies and practices on delivery of special education services, and determine the impact of both State and local changes on outcomes for students with disabilities and their families. Finally, the study will measure the success levels of students currently served through special education programs.

The Oregon State Department of Education, in cooperation with the State licensing authority (Teachers Standards and Practices Commission) and Western Oregon State College's Teaching Research Division, is evaluating the effectiveness of the National Teacher Examination (NTE) as a vehicle to add special education endorsements to the basic teaching certificate. The present study builds upon the results of a feasibility study conducted to identify a usable data base and to provide the foundation for a comprehensive evaluation effort.

Findings of Studies Completed in FY 1992

Connecticut State Department of Education: "Internal Construct Validity of the Connecticut Mastery Test: Special Education Applications." The Connecticut Mastery Tests (CMT), a curriculum-based, criterion-referenced test in language arts, writing, and mathematics, is used to monitor the academic outcomes for students across the State by assessing skills that should be mastered by most students by the end of the third, fifth, and seventh grades. In order to determine the validity of using the test for special education populations as well as general education students, the Connecticut State Department of Education sought to establish the internal construct validity of the CMT for special education students with mild disabilities. All students in grades 4, 6 and 8 are required to take the CMT, with the exception of special education students who have been exempted by their Pupil Placement Teams.

The research questions addressed by this study included: "Is there a floor or ceiling effect that limits the sensitivity of the CMT for testing special education students?" "Is the CMT measuring the same performance dimensions for special education and general education test takers?" "Do total test and object level performance characteristics of special education test takers vary by exceptionality or by LRE related variables?" And, "Do the characteristics of CMT items (e.g., difficulty, discrimination) vary based on the exceptionality of the students?"

In summary, the findings of this study were that:

Based on the results of these analyses, the researchers concluded that the Connecticut Mastery Test is a valid and reliable assessment tool for the special education students who took the test.

The Connecticut State Department of Education also concluded a feasibility study entitled "Measuring Student Attitudes and Attributes in Special Education". This feasibility study was designed to develop a methodology for incorporating student attitudes and attributes of self-concept into the Statewide Evaluation of Special Education Services. The goals of the study were to: identify and define constructs related to student attitudes and attributes that are considered important features of, or important outcomes of, special education programs; build conceptual models that describe the role played by these constructs in special education programs and the relationships between these variables and academic outcomes; describe the desired outcomes on these constructs for students in special education programs; and make recommendations about measurement strategies, data collection procedures, and suitable evaluation questions for this project.

While the steering committee was not able to construct a formal model to describe the role played by attitudes in special education programs and the relationships between attitudes and academic outcomes, several elements of the model were specified. Three primary sources of variance were identified that contribute to student achievement in school: important student attributes (e.g., ability, motivation, self-concept, social competence, family support); school resources, including instructional variables such as mission, curriculum, school climate, and instructional resources; and teachers' skills, training, and commitment. The model presumes that schools and teachers combine to act on students to produce desired outcomes.

The steering committee made three sets of recommendations concerning the future of the Statewide assessment: (1) areas appropriate for instrument development were identified, including student judgments of academic and social competence, social integration, and involvement in educational decision making, and consumer satisfaction; (2) general guidelines for the evaluation to follow were to minimize both intrusion on student instructional time and data collection burden on teachers and administrators, and to meet the American Psychological Association basic requirements for valid, reliable, and ethical assessment; and (3) to help shape the project, develop an attitude assessment strategy that may be integrated with the main structural features of the Plan for Statewide Evaluation of Special Education Services, obtain data for non-disabled students whenever possible for purposes of comparison, and ensure that all data collection efforts are cost effective in use of fiscal and human resources.

The Minnesota Department of Education: "Linking Costs to Multiattribute Outcomes in Special Education". This study was designed to examine the program costs and outcomes of special education for students with moderate to severe mental disabilities under three different administrative structures used in Minnesota to deliver special education services: an independent school district, an intermediate school district (representing consortia of independent districts offering services to students with low incidence disabilities), and a special education cooperative representing small to medium independent districts sharing delivery of special education services. The main focus of the study was on estimating the relative efficiency of the three alternative administrative structures in serving students with moderate to severe mental disabilities. Two products resulting from the study include a cost accounting framework for district level cost analysis, and an evaluation framework for assessing the outcomes resulting from delivery of the special education services through the use of a stakeholder group.

Study findings suggest that costs do not appear to be strongly influenced by the type of administrative structure for most program areas, but are influenced by other factors, such as differences in teacher salaries and teacher tenure. For example, average teacher salaries in the urban independent district were over 25 percent higher than in any of the other districts. Independent of costs, the intermediate district's program for students with moderate and severe mental disabilities was found to be most effective, with the independent urban district's program the least effective, according to the multiattribute outcomes identified by the stakeholder group. The findings indicate that the intermediate and special education cooperative districts were about twice as cost effective as the urban independent district model. Average costs were generally lower and average measures of effectiveness were generally higher for the two multidistrict models.

North Carolina Department of Human Resources: "The Abilities Project: Developing Descriptors for Characterizing Infants and Preschoolers with Disabilities". The ABILITIES project was a cooperative effort between the North Carolina Department of Human Resources and the Frank Porter Graham Child Development Center of the University of North Carolina at Chapel Hill. The purpose of the project was to investigate the utility of the ABILITIES Index, an alternative instrument for describing the functional abilities and limitations of young children with disabilities, independent of etiologic bases and/or manifestations of handicapping conditions. The project consisted of five related studies to examine the reliability, utility, and consumer perceptions of the instrument for use in early intervention.

The Reliability Study examined the extent to which parents and various professionals consistently rated children with the ABILITIES Index. The two central questions addressed the extent to which ABILITIES ratings are consistent across raters, and for an individual rater, how consistent they are across time.

The Team Consensus Study more fully examined the reliability of the ABILITIES Index among a group of experts from different disciplines. Two questions were addressed: (1) do members of an interdisciplinary assessment team rate the same child in the same way? and (2) what is the relationship between individual team member ratings and a team consensus rating?

To establish the validity of the ABILITIES Index, the Criterion Study sought to determine the extent to which ratings on the ABILITIES Index could account for variability in developmental markers, intervention variables and traditional categorical labels. Two questions were addressed: (1) what is the relationship between functional characteristics, as measured by the ABILITIES Index, and developmental markers, assessed by the Battelle Developmental Inventory? and (2) to what extent can the ABILITIES Index complement or extend the differentiation of children grouped on the basis of traditional descriptors such as categorical labels or etiologic markers?

The Consumer Validation Study was conducted to determine the extent to which consumers perceive the ABILITIES Index to be understandable, acceptable, and useful. The final study was the Descriptive Study to use the index to describe all the children in a given service delivery system.

The studies concluded that the ABILITIES Index was found to be a reliable, valid, useful, and acceptable way to describe the functional needs of children with disabilities. At a systems or population level, aggregate ratings on the ABILITIES Index can be used to describe populations in such a way that groups with different characteristics can be clearly differentiated from one another based on functional characteristics. Findings from this project also suggest that a system such as the ABILITIES Index could be used at the individual level as an alternative to categorical labeling.

National Studies

A cooperative agreement was awarded to the University of Minnesota in FY 1990 to support a National Center for Outcome Assessment (NCEO). The NCEO mission is to work with Federal and State agencies to facilitate and enrich the development and use of indicators of educational outcomes for students with disabilities. Responsible use of such indicators will enable those students to achieve better results from their educational experiences.

Major NCEO activities include:

Program Administration

In FY 1993, a total of 13 awards were made under the State Agency-Federal Evaluation Studies program.

Management Improvment Strategies

III. Sources of Information

  1. Fifteenth Annual Report to Congress on the Implementation of The Individuals with Disabilities Education Act (Washington, DC: U.S. Department of Education, 1993).

  2. Sixteenth Annual Report to Congress on the Implementation of The Individuals with Disabilities Education Act (Washington, DC: U.S. Department of Education, 1994).

  3. Program files.

IV. Planned Studies

The Office of Special Education Programs plans to support one cooperative agreement to study the progress being made to implement the transition services mandated by IDEA. There is considerable State and local variation with respect to the implementation of these expanded requirements. Very little information exists on the nature and extent of State and local implementation, including policies, procedures, and practices. Moreover, policymakers, administrators, and educators at the Federal, State, and local levels lack information regarding the nature of student participation and the impact these services have on student outcomes, the extent that other agencies are involved in the transition process, and the degree that transition services access and use information and services available from a variety of Federal programs.

The specific goals of the evaluation are to describe Federal, State, and local implementation, including policies, procedures, and practices associated with transition services, to identify barriers to effective implementation, and to evaluate the impact of transition services on student outcomes.

V. Contacts for Further Information

Program Operations:
Lou Danielson (202) 205-8119

Program Studies:
Susan Thompson-Hoffman, (202) 401-3630

-###-
[Captioned Films, Television, Descriptive Video, Educational Media for Individuals with Disabilities] [Table of Contents] [Secondary Education and Transitional Services for Youth with Disabilities]