A r c h i v e d  I n f o r m a t i o n

Technology and Education Reform: Technical Research Report - August 1995

3. Study Aims, Questions, and Methodology

As stated in Chapter 1, this study was designed to provide insights both into how technology can support constructivist learning activities at the classroom level and into the practical and organizational factors that promote or hinder technology implementations within schools. For this reason, research questions were specified at both the classroom and the school levels.

Research Questions

Classroom Teaching and Learning Questions

What examples does the classroom offer of using technology to support long-term, student-centered projects?

School-Level Implementation Questions

Overview of the Methodology

Site Selection

Given the resources to conduct case studies at just nine sites, the research team devoted considerable effort to choosing cases that would provide a range of worthwhile examples for other schools. We were quite aware that no site was likely to prove to be exemplary in all respects, and that all schools, including those that are pioneers, experience difficulties and unevenness in their programs. We looked for sites that had a history of using technology not for its own sake, but rather as a support for constructivist learning and a broader education reform agenda.

Further, both OERI and our research team felt that it was important to pull illustrations of exemplary instructional uses of technology from public school classrooms serving students from diverse backgrounds and from low-income homes. We sought to document the fact that technology-supported constructivist learning activities can unfold not just in affluent suburban schools but in classrooms facing all of the funding and social issues besetting so much of the nation's education system.

We collected ideas for potential case study sites through a review of the literature, through discussions with practitioners and education technology experts at a national conference, from our project advisors, and from our own network of school and technology contacts. We conducted telephone interviews with nearly 40 potential sites to collect information regarding the following criteria:

Applying these criteria to the potential sites for which we had conducted phone interviews, we made recommendations to OERI and negotiated a final case study sample, described in Table 1. (The school names appearing in the table and throughout this report are pseudonyms.)

Table 1
Case Study Sites

Site Level Student Body Setting Region Key Features
Free/Reduced-
Price Lunch		 Demographics
Bay Vista Elementary E 25% 89% minority;
25% ESL		 Suburban West State model technology school site for science
TeacherNet
(Network of 462 schools)		 All Varies Varies across schools Includes rural, urban, and suburban Midwest Partnership of 54 school districts across 2 states participating in network activities
South Creek Middle M 65% 60% low-income Hispanic Suburban Southwest Reopened in 1991 as model restructured school with high level of connectivity
Nathaniel Elementary E 85% 95% minority;
59% LEP		 Urban West Inner-city school involved in classroom projects including communal databases for cooperative learning and video-supported science and language arts curricula delivered through satellite dish
Progressive E 23% Wide SES range;
61% minority		 Urban West Charter school with team-taught classes, project-based instruction and 1 computer for every 2 students
John Wesley Elementary E 100% 86% Hispanic including many children of migrant workers; 64% LEP Suburban West Technology introduction initiated by teacher team working on curriculum and instruction as part of school's active restructuring effort
School of the Future M 80% Wide SES range;
67% male		 Urban North Central Designed as "break the mold" school incorporating technology; course offerings designed around student interests
East City High
(School-Within-
a-School)		 S 40% 35% African-American Urban Midwest Apple Classroom of Tomorrow (ACOT) within urban secondary school
Maynard
(School-Within-
a-School)		 M
4-6		 77% 71% African-American; 27% Hispanic Urban Northeast Mini-school provides students with extensive access to computer lab and wide area network resources

On-Site Activities

With the exception of the use of video to document our interviews and classroom observations, our data collection procedures followed standard qualitative procedures. Two-person evaluation teams visited each of the nine sites for a period of 3 to 8 days over a 2-year period. At each site, we did initial brief observations of a broad range of classrooms in order to pick two classrooms for more intensive observation and videotaping.(2) Our criteria for choosing these classrooms were a combination of the theoretical and the pragmatic--from the early discussions with administrators and teachers, we tried to select classrooms that were using technology in tool-like ways to support complex, student-centered activities. At the same time, we were constrained by schedules, trying to select classrooms that would be doing something interesting on their technology-supported projects on the particular days we would be present to observe them. These more intensively studied classrooms were typically observed over repeated days, sometimes on multiple visits. More extensive interviews were conducted with the teacher or teachers, and typically one of our two student focus groups was conducted with students drawn from this class. In addition, as our data collection proceeded, we found it useful to interview individual students or small groups as they developed or exhibited their technology-based work or demonstrated how they used particular pieces of software. We observed and videotaped classes, school activities, teacher meetings and training, and other key events related to technology use in these classrooms.

In addition to the classroom-based data collection, we interviewed a wide range of other school respondents, including principals, project coordinators, and school technology coordinators. Moving out from the school, we then interviewed representatives of other institutions that were pivotal in the school's reform effort. These might include district personnel, researchers, representatives from business partners, leaders from parent groups, or education consultants. Our final selection of respondents depended on the school's particular implementation history and its perception of the key players within it. For individual sites, we also interviewed a school board member, a union leader, and a state administrator.

Planning for Cross-Site Synthesis

National studies involving multiple sites require advance planning and structuring of the data collection effort so that information can be systematically collated and synthesized for cross-case analysis. To this end, we planned a strategy that included site visitor training, the use of interview guides for each type of respondent (e.g., teachers, principals, technology coordinators), development of debriefing forms for the school and classroom levels, and the use of qualitative analysis software (Seidel et al., 1988) to facilitate qualitative data analysis.

We used our research questions as a general blueprint for designing both interview protocols (lists of topics to be covered) and debriefing forms (case study outlines). The purpose of a debriefing form is to provide a standardized framework for writing a case study report. This is especially important when multiple sites and multiple researchers are involved. We used two debriefing forms for our study, one for schools and one for classrooms. The school-level debriefing form took a broad view that included a review of the educational context of the site; demographic information; educational indicators; history of educational reform at the site; levels of involvement at the district, state, and federal levels; history of technology applications, including incentives for use, when and how the applications started, technologies used, target grades and curricula, key school players, and key outside players; overview of the way the technology is used by students and teachers; implementation details, including problems encountered, strategies for overcoming barriers, and facilitators and costs; impact of the technology use on students, teachers, and the school climate and processes; the way the technology use is evaluated; and respondents' reflections and advice.

The classroom debriefing form was similar in scope but focused on what was observed in the classroom during the site visit. Site visitors were prompted to write about the classroom context, features of reform that they observed, the classroom activities that took place, the technologies involved (e.g., microcomputers, wide area networks, hypermedia, animation, simulation), how the technologies were used by students and teachers, and intended and actual benefits of the technology use from the perspective of students and teachers. The debriefing forms for our schools and classrooms are presented in Volume 2: Case Studies.

A special feature of our cross-case synthesis plan involved the use of software for qualitative data analysis, in our case, THE ETHNOGRAPH. The software facilitates the analysis process by searching for and retrieving data marked by code words or combinations of code words. It prints out text organized by the code or codes specified in a search procedure. The printout then can be assembled in a way that allows the researcher to read all the text pertaining to a particular topic, concept, or variable across all sites or a subset of sites. A critical step in using such software is the generation of a set of codes for labeling segments of text. We began the process of developing codes concurrently with designing the debriefing forms. Details of this procedure and a listing of our analytic theme codes are provided in Volume 3: Technical Appendix.

Software-Supported Cross-Case Analysis

After the majority of write-ups were complete, the research team members read one another's write-ups and met as a group for a full day to share impressions and begin the process of interpreting the findings from a cross-site perspective. We began by focusing on individual cases and then worked across cases. Our shared conceptual framework, exemplified by the debriefing form, helped to structure the discussion, but by this time, we were thinking beyond the debriefing framework to look for higher-order patterns and issues that we had not recognized when the debriefing forms were designed. We focused especially on successful sites and what made them so, and the apparent reasons some supposedly "exemplary" sites hadn't turned out to be so exemplary after all. As we generated observations about our sites, we began to identify potential cross-site themes and corresponding theme codes. The methodological volume (Volume 3) contains a full listing of theme codes, as well as the heading codes from the debriefing forms.

Once the debriefing forms were converted to ASCII files, the next step was the insertion of codes. Embedded heading codes were inserted on-line by clerical staff; theme coding and other more complicated coding were inserted by researchers on hard-copy printouts and then inserted into THE ETHNOGRAPH files by clerical staff.

Once all the codes were inserted, searches were run. A search pulls out all segments of text that are coded with the code word being searched. (Volume 3 contains examples of search output.) Multiple code searches can be done on all files or a subset of files, so that researchers have limitless ways to explore the data.

The search output then was organized into six major categories:

The search output filled four large notebooks. Within each notebook, the printouts were further organized by code word. For example, the Technology Implementation notebooks encompassed 29 codes beginning with HISTORY. Text coded with HISTORY was organized by site, after which the next set of printouts would appear. This way of organizing the printouts was selected because it enabled the researcher to read across sites while staying focused on a particular aspect of the technology application--in this case, the history of each implementation.

In addition to facilitating the consideration and elaboration of themes across the sites, THE ETHNOGRAPH also facilitated quick counts and status checks regarding the occurrence of selected variables within each of the sites. This provided an additional means for the researchers to verify and summarize their findings. For example, the researchers were quickly able to assess the number of classrooms within and across sites that reported specific intended benefits (e.g., higher-order thinking skills) and teacher-observed effects (e.g., increased student motivation, greater collaboration) in relation to the integration of technology. The findings of the cross-case analysis supported by THE ETHNOGRAPH are presented in Chapters 5 through 10.

2 The single exception was the "site" that was actually a network of over 450 schools. For this site, we visited four schools and selected a single electronic research class for more detailed observation and description. Hence, there were 17 classroom activities in our final sample of detailed vignettes.


-###-

[Review of Literature (part 2 of 2] [Table of Contents] [Case Study Profiles]