A r c h i v e d  I n f o r m a t i o n

Improvements in Student Performance

Teachers for the classes and activities observed at the case study sites were nearly unanimous also in reporting that students were able to handle more complex assignments and engage in higher-order thinking skills because of the supports and capabilities provided by technology. Teachers report that students are more able and willing to edit their texts and engage in critical rereading when writing at the computer. Math tools, such as spreadsheets and LOGO programs, can assist students in gaining greater conceptual understanding through concrete interaction and feedback. Science simulations allow students to observe and manipulate multiple aspects of complex microworlds. Database software provides students with a tool for gathering and analyzing large amounts of information in different ways. Across all of these applications, the technology itself poses a problem-solving challenge as students learn to master the features of the tool to accomplish their desired goal. Further, the computer tasks are often done in collaboration with peers, which in turn adds a new layer of complexity through the feedback and the communication requirements that working with others entails. Exhibit 12 provides some classroom examples of technology's role in supporting students as they undertake activities calling for higher-order thinking. One of the teacher's from the CSILE class described in Exhibit 12 talked about ways in which the structure of the communal database her students used supported their learning of important concepts:

There's a lot of high-level knowledge embedded in it that they're not even aware that they are using. They have to produce summaries... All these things you used to teach out of a workbook and it didn't make any sense. Here it's a direct application. They need to be able to do this in order to access something. So it's automatic that they learn how to do it because they want to be able to access. Like with the Book Talk, they apply key words to their summaries, so that someone can scroll through and see, "Now here is a book on family problems..." So making that kind of link, being able to synthesize your thoughts...there's actually an application on CSILE for linking, so they can link one note to another, talk about what the connection is.

--Elementary school teacher using CSILE

Another student outcome that many teachers attributed to technology use was the production of higher-quality products. This outcome was cited in relation to many different types of technology-supported activities, such as video production, multimedia projects, animation, and research. Students in a fifth-grade classroom were in the process of editing and producing a videotape on multicultural heroes, which they planned to market to other classrooms. Students involved in a city-building project at the Progressive School put together a multimedia product incorporating carefully edited text and QuickTime video for presentation to visitors and at conferences. At the School of the Future, middle school students created complex, professional-looking animations with HyperCard. At one of the TeacherNet high schools, where students use the Internet to research report topics, both the students and the teachers said that the reports were of higher quality in terms of the breadth, recency, and comprehensiveness of the information incorporated.

Writing and desktop publishing were widely regarded as areas that had been positively affected by the use of computers. Teachers reported that students produced higher-quality texts when writing with the word processor and often were more willing and able to edit their texts in this context.

Exhibit 12 Examples of Technology-Provided Supports for Advanced Thinking Skills A Middle School Math Class The math teacher reports that technology supports enable students not only to produce higher-quality graphs but also to understand graphs at a deeper level and to be able to examine the relative strengths and weaknesses of different graphic representations. In an activity designed to introduce students to the use of spreadsheets to calculate, analyze, and present quantitative data, students first estimated the length of various parts of their own bodies (e.g., wrist circumference) and then made actual measurements. Both sets of data were entered onto a spreadsheet, from which students then began to experiment with different ways of representing the data. Because the physical production of the graphs was handled by the computer, students could focus on making the conceptual link between the spreadsheet data and the visual representation. They worked collaboratively to determine what information to display and how best to display it. After trying a variety of representations (e.g., pie, line, and bar graphs), they were able to discuss the advantages and disadvantages of each for different types of data sets (e.g., pie graphs can display only one variable). Students are asked to further reflect on their problem-solving process through the activity of writing narratives at the word processor describing their approaches and their results. Through a series of activities such as this, the teacher reports that students have gained an understanding of graphs that has extended across subject areas. At the request of other teachers who have observed this increased skill on the part of students participating in the math class, the math teacher has agreed to conduct a workshop on computer-supported graphing for the entire faculty. A Long-Term Elementary School Project In the Archeological Dig Project at Nathaniel Elementary School, the use of CSILE helped students organize and extend their thinking through collaboration with their peers. Students and teachers generated challenging topics about the culture under study (e.g., "What did the ancient Egyptians do to write numbers?" "What were their laws and government?"). A student who initiated a topic would be prompted by CSILE to figure out specific things that needed to be learned to address the topic. By posting the inquiry on-line, the student would get help from other students in figuring out ideas (actions) and resources (where to go for more information). Students would make on-line notes of information on the topic as they collected it, stimulating other students to add more information and interpretations. As students became more adept with the process of collaborative knowledge building through CSILE, their dialogues took on an impressive level of sophistication. For example, the group developing a number system for a hypothetical culture posted an illustration of the system with the linked text entry: This is the slaminan's number system. It is a basic 10 number system too. It has a pattern to it. The number of lines increase up to five then it goes upside down all the way to 10. --Student group in CSILE class The group developing rituals for the same culture studied the entry and wrote a response pointing out the need to extend the system: We all like the number system, but we want to know how the number 0 looks like, and you can do more numbers not just ten like we have right now. --"Ritual group," CSILE elementary school class.

Teachers and students alike commented that the capability of creating written products similar in appearance to published print is a source of real satisfaction and pride for students.

If you're doing something like eighth grade research, you want it to look nice. You don't want childish, sloppy writing.

--Middle school student

For some of them, it makes them feel like they're finished. It looks really good-better than if they had done it with handwriting. It looks professional. . . . In that respect...I know the feeling myself...it puts a new luster on things. I think that if they didn't have that, they would not feel so big or so professional-that they have gone through everything that a real writer does. To have it published at the computer and typed up like that is more than if they had just done it by hand. It's like the difference between third and fifth grade.

--Elementary school teacher

Other frequently cited benefits of technology-based activities included enhanced creativity, the development of design skills, and improved thoughtfulness about the needs and interests of a potential audience. Experiences in developing the kinds of rich, multimedia products that can be produced with technology, particularly when the design is done collaboratively so that students experience their peers' reactions to their presentations, appear to support these qualities.

Multiple media give students choices about how best to convey a given idea (e.g., through text, video, animation). In part because they have the capability to produce more professional-looking products and the tools to manipulate the way information is presented, students in many technology-using classes are reportedly spending more time on design and audience presentation issues.

They also do more stylistic things in terms of how the paper looks, and if there is something they want to emphasize, they'll change the font...they're looking at the words they're writing in a different way. They're not just thinking about writing a sentence; they're doing that, but they are also thinking about, "This is a really important word" or "This is something I want to stand out." And they're thinking in another completely different way about their audience.

--Elementary school teacher

Although most teachers were positive about the design consciousness that technology fosters, a potential downside was also noted by a few teachers. It is possible for students to get so caught up in issues such as type font or audio clips that they pay less attention to the substantive content of their product. We observed one computer lab in which several students with a research paper assignment spent the entire period coloring and editing the computer graphics for the covers of their as-yet-unwritten reports, pixel by pixel. A middle school student reported that when writing at the computer she often got caught up in worrying about the kind of type to use ("You think about what kind of writing you want it to be in, plain or . . ."). Teachers are developing strategies to make sure that students do not get distracted by some of the more enticing but less substantive features of technology, for example, by limiting the number of fonts and font sizes available to the students.

The greater use of outside information sources was an effect of technology use cited by teachers from 10 classrooms. This effect was most obvious in classrooms that had incorporated telecommunications activities, but other classes used technologies such as satellite broadcasts, telefacsimiles, and the telephone to help provide access to external sources of information. Schools with links to the Internet reported that the network brought "the outside world" into the classroom, enabling students to gather data directly from a wide variety of sources and to learn about life beyond the classroom walls through interpersonal communication with e-mail participants from around the world. Students at the Maynard Computer Mini-School studying Ireland for their schoolwide multicultural fair, for example, used the Internet to interview Trinity College students and to obtain information on historical artifacts from the College's database.

Sometimes children...are very isolated because they don't get to go many places. But through this communication over the computer they are able to relate that Ireland isn't just a place on a map but that there are people living there, and through e-mail they get a response. A lot of them have pen pals all over the world.

--Elementary school teacher

I think the availability of all the different resources, DIALOG in particular, to access all the different databases...it's really a help to teachers because they get to bring things in and share them with their classes.... Instead of information that's 2, 3, 4, or 5 years old, the computer can get the most current and up-to-date information. I have had endless numbers of teachers come to me and say, "I am such a better teacher now. I have access to current information. I got it at home last night."

--Secondary school principal

Students too find that the currency of information they can gather over the Internet gives them a greater sense of being part of the world of national and global events:

I have looked at articles [on the Internet] that aren't 20 minutes old.... For example, the day Kurt Cobain's body was found, I went home and I heard it on the radio. The first thing I did was I called [the local network connection] and looked at the UPI.... There were already four or five different articles...like his impact on society...actual new accounts, different experiences of the band members...a lot of up-to-date information.

--High school student

A number of classrooms using either wide area or local area networks extensively reported positive effects on students' inclination to consider multiple perspectives. When students conduct searches on the Internet, for example, they are exposed to the idea that there are many different sources of information and varying perspectives to be brought to bear on a particular subject. Engaging in electronic-mail communication with individuals from distant locations further exposes students to perspectives that differ from their own.

Teachers at the Maynard Computer Mini-School, where students make extensive use of Internet resources in conducting their research projects, suggest that the active engagement in finding and querying information resources, especially human information resources, leads students to develop a different stance toward information. Rather than something inert that is cut and dried and captured for all time in a textbook, information is something that exists all over the world, changes constantly, and can be viewed from multiple perspectives.

The kids were able to contact Trinity College in Ireland [over the Internet] and ask for information from college students.... It makes kids realize that information is happening right now and it's not just in a textbook. And textbooks can be wrong. And there's always a perspective on a textbook... let's get it from someone who is really living there instead of from a textbook...

--Fifth-grade teacher

Changes in Student and Teacher Roles

An impact of technology cited by a great majority of teachers is an increased inclination on the part of students to work cooperatively and to provide peer tutoring. The CSILE classrooms, described above, were designed explicitly to take advantage of network technology as a support for cooperative work. As multiple groups of students worked concurrently on their respective portions of their hypothetical culture, CSILE helped the groups keep in touch with each other's progress, an important activity because the various aspects of a culture need to be consistent.

Student groups created graphic and text notes to explain what their cultural universal was. Then other students could access that and comment back right away and say, "Wait a minute; you can't make a boat out of a tree because our culture doesn't cut down trees!"

--Elementary school teacher from CSILE classroom

While many of the classrooms we observed assigned technology-based projects to small groups of students, as in the CSILE Archeological Dig Project, there was also considerable tutoring going on around the use of technology itself. Collaboration is fostered for obvious reasons when students are assigned to work in pairs or small groups for work at a limited number of computers. But even when each student has a computer, teachers note an increased frequency of students' helping each other. Technology-based tasks involve many subtasks (e.g., creating a button for a HyperCard stack or making columns with word processing software), leading to situations where students need help and find their neighbor a convenient source of assistance. Students who have mastered specific computer skills generally derive pride and enjoyment from helping others.

In addition, the public display and greater legibility of student work create an invitation to comment. Students often look over each other's shoulders, commenting on each other's work, offering assistance, and discussing what they are doing.

I've also seen kids helping each other a lot at the computer. The ones that pick it up faster, they love teaching it to someone that doesn't know it yet.

--Fourth-grade teacher

The ones who have used it from the beginning have become peer coaches.

--Fifth-grade teacher

We observed only one school where many students preferred working individually on technology-based projects without interacting with each other, even sitting with nothing to do rather than pairing up at a computer if there were not enough to go around. At this school, a fear of disclosing one's password and making computer files vulnerable to destruction by other students appeared to work against the inclination to collaborate. But even within this school, we observed individual classes in animation and home design where the teachers encouraged collaboration, and students viewed and commented on each other's work ("Cool!") and offered assistance with each other's designs. In general, unless students are expressly directed to avoid discussion during computer work or are put into a physical environment that makes discussion difficult (for example, put in carrels and given headsets for computer-based work), technology appears to promote interaction rather than isolation.

Many of the teachers also report that the introduction of technology creates many more occasions on which students are teaching teachers. At one school we observed a trio of students working with their teacher after school, patiently guiding her step by step through her first lesson in word processing. Such activities support a shift in student and teacher roles toward the kind of "community of learners" that many educational theorists advocate (Brown et al., in press).

Going along with this change in roles, teachers see a positive effect of technology on students' inclination to regulate their own learning. Several teachers commented that the computer allows students to engage in independent learning activities and to work at their own pace.

Teachers are often pleasantly surprised by their students' ability to manage long-term, complex technology-based assignments. In addition, technology can provide supports for self-regulation. CSILE, for example, is designed to support students' learning by providing a structure that facilitates student reflection on their own and others' thinking. One teacher noted that students become more adept at identifying the gaps in their own thinking and understanding through their participation in CSILE, as they make their knowledge explicit and receive specific feedback from others.

More generally, the network configuration and interface set up for a number of sites provided supports for students to take responsibility for their own learning.

Students know that if they are absent, they have immediate access to look up materials on the network without the need to stop the teacher from instruction to the rest of the class. Students can mail in assignments on the network. Items are dated and time stamped when they are turned in. Teachers can collect materials at their convenience rather than taking away valuable learning time while with the students.

--Secondary school technology coordinator

The network also supports greater student responsibility for managing their own behavior by allowing students to work on assignments from any location. A student in a social studies class who has finished her assignment can log on to the server and work on her assignment for another class. Technology can function as a tool for monitoring growth and self-progress through feedback and the automatic maintenance of a running record of a student's work. A variety of technologies offer opportunities for students to critically examine their own and each other's work, for example, through viewing videotapes of exhibitions or performances or through electronic transmission of one another's texts for review and editing.

Some teachers also report improved behavior in areas such as attendance and time on task. A number of the technology-using classrooms we studied reported having fewer behavioral problems than other classes and more students who were willing and able to stick with a task for long periods of time. In part, this effect seems related to motivation and the degree of absorption that typifies technology-based tasks:

Technology contributes to the sense that they always have something that is high-powered to do, which is motivating.

--Elementary school teacher

One of the teachers we interviewed perceived an additional advantage of computer-based activities in calming students who are having emotional difficulties or trouble concentrating:

The computer room is a place where a lot of the kids, if they are having trouble with their behavior or emotions, they will ask to go.... I think it soothes them, and there is no pressure.... It focuses them off whatever was bothering them and gets them back on track.... It can take a child who is very unfocused and having distress in the classroom, and you can see them...they are not breathing heavy anymore...even just logging on.

--Fifth-grade teacher in inner-city school

For many sites, the school's engagement in technology-based activities had attracted outside attention and numerous visits. Some of the sites have hosted hundreds and even thousands of visitors in a single year. Past visitors have included prominent individuals, such as state legislators, members of the National School Board Association, a state governor, the Chairman of the Federal Communications Commission, and the President of the United States. Teachers from two classrooms noted that as a result of experiences in showing visitors how technology is used within the school, students had gained confidence and improved oral communication skills. Given the facility displayed by a number of the students we interviewed, this contention is easily believed.

Finally, nearly every teacher cited the concrete technical skills that students had acquired and the advantages that such skills would give them in later education or employment settings. The level of expertise acquired by students varied both within and across sites, in relation to the level of access that was provided as well as in relation to individual student interest. Students with the most sophisticated skills were often those who engaged in frequent self-selected technology-supported activities, in addition to their regular coursework. What was striking overall, however, was the large numbers of students who were quite comfortable with the equipment and had achieved a basic level of competency in using technology as a tool.

They are not going to have the phobias we have [about technology]. They have discovered a lot.

--Elementary school teacher

When I first came to [this school] I really didn't know anything at all about the Macs other than there's a little switch in the back-you turn it one way, it turns on.... That's basically all I knew. I first started out with just the word processor...and then I started getting into games and I took a HyperCard class and that really got me started.

--Middle school student

Evaluations

A number of the case study sites have conducted self-studies or evaluations of their programs, either in response to state or other funding accountability requirements or as a tool for improving their programs. Several of these were quite detailed (e.g., South Creek, School of the Future, John Wesley) and included data from student, teacher, and parent surveys as well as the kinds of statistics normally collected by districts. Other programs maintained surprisingly little data; this tended to be true particularly for the schools-within-schools, which were not required by districts or states to collect data on the subset of students they served. Impediments to conducting systematic evaluations, of course, include lack of funding and expertise. Two of the schools with the most complete evaluations (the School of the Future and John Wesley) both received foundation funds to support this activity.

Even when the data maintained by a school were extensive, however, they did not provide a simple answer to the question, "Does technology work?" In every case, the site was implementing many more changes than simply the introduction of technology, and students, teachers, and other school variables were not completely comparable with those of other schools within the district or state. As Joan Herman (1994) points out, we should not be surprised by this situation. By their very nature, education reforms are multifaceted. We need to ask not the simplistic question of whether technology "works" but rather more detailed questions about specific effects of specific uses and kinds of technology. Nevertheless, to provide a sense of how the case study sites measure up in terms of traditional education indicators, we have provided a summary of available data in Table 9.

These data were collected by the schools or districts, and the available comparison group (other schools in the district, students not in the mini-school, performance prior to the innovation) varies by site. In reviewing the data, the reader should remember that we specifically chose to look primarily at sites serving substantial numbers of students who live in economic poverty. Even so, the sites vary considerably in the demographics of their student populations. A rough indicator--the proportion of students receiving free or reduced-price lunches--is provided. Overall, the site indicators are positive . Out of the 8 single-school sites, 7 reported lower-than-average rates of teacher turnover; 6 reported high student attendance rates; and 5 had higher standardized test scores than some comparison group. Of the 6 schools providing reports regarding rates of student discipline problems, 5 could point to lower rates of disciplinary incidents than experienced by some comparison group. Although these variables were not measured in the same way across all the schools and in some cases there may be questions about the suitability of the comparison group, the overall picture is an encouraging one. In the final column of the table, we present some of the key variables other than technology per se that might contribute to the measured outcomes.

Table 9
Indicators for Case Study Sites

Site	%Free/ Reduced Lunch	Test Scores	Attendance	Discipline Problems	Teacher Turnover	Other Outcomes	Nontechnology Factors
Bay Vista	25%	Students showed >1 year's growth on CTBS science subtests in '87-'88; significant gains on tests of science process skills in '89	Above average for district	Below district average for district	Low		Hands-on science curriculum; highly involved parents; closeknit teaching staff
East City High School	40%	Much higher percentage pass state proficiency tests than in rest of school, but test is given in fall of 9th grade	Higher than rest of school	Not available	Low		Smaller class size; interdisciplinary, student-centered approach. Students must apply for program; 37th percentile on CTBS required
John Wesley	100%	Scores fluctuate across grades and years between 30th and 50th percentiles on state standardized tests	Improving over past 3 years	Declining over past 3 years	Low		High percentage of students with limited English; poor match between program and standardized tests; integrated thematic instruction; emphasis on emotional as well as cognitive growth
Maynard Computer Mini-School	77%	Mini-school students gained more than school's other students on one reading and three math subtests for '90-'91	Not available	Better than rest of school	Lower than rest of school	Mini-school students do well on city, state, and national competitions (e.g., poetry, chess)	Students selected form those whose parents apply; mini-school has smaller classes and more cohesive program than rest of school
Nathaniel Elementary	85%	Second-graders scored in 7th-14th percentiles on CTBS tests in '92; sixth-graders at 19th-30th percentiles	High for district	Low for district	Low		Very large classes; high student mobility; high proportion of limited English
Progressive School	23%	Higher scores than other schools in the area despite more ethnic and SES diversity	High	Not available	Low		Extremely close-knit, dedicated staff; cohesive, student-centered curriculum; high parent involvement
School of the Future	80%	Standardized test scores in mathematics declined over first two years; by 92' scores returned to level of original incoming class (slightly above national average)	High	Varied across time	High		Standardized tests not matched to curriculum; following negative publicity about test scores, school has gained reputation as a "school of last resort"
South Creek Middle	65%	Higher scores on state tests than schools serving comparable populations; 45%, 59%, and 72% of seventh-graders mastered state objectives in reading. math, and writing, respectively, compared with 42%, 42%, and 62% for similar schools and 53%, 52%, and 69% for state as a whole in '93	High	Low compared with year of opening	Low	0.5% dropout rate compared with 3.9% for state; high teacher satisfaction shown on survey; at end of school's first semester, its students ranked 2nd in district on math objectives despite the fact that it was lowest in district in SES	Teach 90 minutes each of math and language arts daily; teacher team approach; high level of limited English among students

-###-

[Effects on Students (part 1 of 2)]

[Effects on Teachers]