![[Table of Contents]](/icons/up.gif){kind=icon}
Computer-Assisted Instruction --Meta-analyses of studies at the elementary school (Kulik, Kulik & Bangert-Drowns 1984; Niemiec & Walberg 1985) and secondary school (Bangert-Drowns, Kulik & Kulik 1985; Kulik, Bangert & Williams 1983; Samson, Niemiec, Weinstein & Walberg 1986) levels generally show a significant advantage for computer- assisted instruction. Kulik, Kulik, and Bangert-Drowns (1985) found that on average, CAI students at the elementary school level outperform their counterparts without CAI by .47 standard deviations. The relative advantage of computer-assisted instruction in these reports appears stronger for disadvantaged and low-ability students (Bangert-Drowns, Kulik & Kulik 1985; Samson et al. 1986) and for males (Niemiec & Walberg 1985).
When Clark (1985) reexamined samples of the studies included in earlier meta- analyses, however, he found that effect sizes were much smaller when the same teacher provided instruction in both treatment and comparison groups and were absent when instructional method was controlled (such that the study measured the effect of instructional delivery medium only). Effects were larger in shorter-term studies, suggesting that novelty effects boost performance with new technologies in the short term but tend to wear off over time.
Videodisc and Multimedia Technologies--Advantages of interactive videodisc over lectures have been reported (e.g., Nelson, Watson & Busch 1989). Fletcher (1990) conducted a meta-analysis of 47 studies comparing instruction via computer-controlled interactive videodisc (IVD) with conventional instruction in military training, industrial training, and higher education settings. On average, those who learned through IVD had achievement scores that were .50 standard deviation higher than those of students taught conventionally. Bosco (1986) reviewed eight IVD studies conducted in school settings; some of the studies found an advantage for the videodisc presentation, while others reported no significant difference.
Several studies have found positive effects in having students develop their own curriculum materials using hypermedia. When asked to draw "concept maps" of the Enlightenment, eleventh-grade history students who had studied the period using a hypermedia corpus called ACCESS (American Culture in Context: Enrichment for Secondary Schools) had more information within their maps and used more abstract concepts to organize the information they had than did their peers who had not used the hypermedia materials (Spoehr 1992). In research conducted by Richard Lehrer, ninth-grade students were retested a year after they had studied the Civil War, some by developing hypermedia presentations and others through traditional approaches. Those with the hypermedia experience had a more realistic understanding of the role of the historian, recalled more Civil War facts, and had more elaborated concepts (Lehrer, Erickson & Connell 1992).
There are relatively few studies providing evidence regarding the effects of new information storage technologies such as CD-ROM. In one study, the term papers of eighth graders using a computer-based videotex encyclopedia were judged to show greater knowledge than those of students using a print-based encyclopedia (Krendl & Fredin 1985).
Distance-learning --Although there is a voluminous literature on distance-learning, there is very little empirical evidence of effects on student learning (Moore 1989). Because distance-learning is generally implemented in situations in which face-to-face instruction in a particular subject area is either infeasible or more expensive, proponents have sought to show that it is equal to, rather than better than, traditional approaches. An evaluation of a two- way interactive television project in Iowa found that students in television classes performed equivalently to students in other sections of the class taught by the same teacher (Nelson 1985). Similarly, a series of studies of ITV in rural Minnesota found no significant achievement differences when students were compared with those in conventional classes (Kitchen 1987).
As Clark (1985) points out, if you really want to assess the comparative effectiveness of the technology medium per se, you need to hold everything else constant. When Clark reexamined a sample of the CAI studies reviewed in earlier meta-analyses, he found that instructional method was equated in only half of the comparison studies. When those studies using the same instructional approach in both groups were analyzed separately, there was no effect of presenting the instruction via computer.
It is not clear, however, that the purity of experimental design espoused by Clark would prove very useful for policy-oriented research. To hold everything constant except the medium used to deliver instruction, studies have to sacrifice representativeness, looking at only a very circumscribed piece of content taught by the new technology and through a more traditional medium (e.g., lecture or textbook). Thus, for example, if a teacher designs a special lesson using specific instructional objectives, diagnostic routines, branching rules, and feedback patterned after those used in a CAI program, and we find that students in her classes perform equivalently to those using CAI, what does this tell us about the effectiveness of CAI compared with typical classroom teaching?
On reflection, in most cases, we are really not interested in whether or not there are effects of the delivery medium per se. We almost never implement a change in medium only. Particularly when we want to understand how technology can support education reform, we want to change the content and the instructional strategy as well as the medium. In such cases, we need to look at specific effects of various facets of the innovation and at the implementation process and how students and teachers use technology, rather than simply comparing two different delivery media in terms of a single outcome measure.
On the dependent-variable side, issues can be equally thorny. Many studies, particularly those examining longer interventions, compare treatments in terms of outcomes on standardized tests. Because the treatment conditions are rarely equated in terms of instructional content, the tests that are used as learner outcome measures are usually more congruent with the objectives and content of one treatment than with those of the other. Thus, the comparison is inherently unfair. Because the tests are usually multiple-choice measures of basic skills, those applications that are most similar to the tests (e.g., drill and practice in basic skills) have tended to show the largest gains (Bangert-Drowns, Kulik & Kulik 1985). A comparable bias may exist in those studies that have designed their own learner outcome measures around the content and presentation format used in the new technology (Samson, Niemiec, Weinstein & Walberg 1986). Students taught different content by a different approach would not be expected to do as well on these measures, regardless of the medium used to deliver the instruction.
The accumulation of comparative studies biased in their choice of control groups or outcome measures does little to help us understand what features of the treatment are critical for producing the desired effects. Although still common in the evaluation literature, these studies are being superseded by more elaborate approaches, as discussed below.
[Contextualized Research]
This page was last updated December 27, 2001 (jca)