A r c h i v e d  I n f o r m a t i o n

Where Do I Start?

Starting Early

Planning an evaluation starts with planning for technology. It is best to plan your evaluation at the same time that the technology program itself is being planned. Early involvement will help you for two reasons. First, it will allow you to collect baseline data. Second, it will allow you to structure professional development and/or purchases to the evaluation design. For instance, in Example A above, teachers were randomly assigned to different kinds of professional development so that different types of teachers (e.g., novice vs. veteran) were distributed roughly equally among both types. To implement random assignment, the evaluator had to be involved in program planning.

What if the program is already underway? Is it too late? No, it's not too late. Though you probably won't have as many options for structuring the evaluation, you can still start a good evaluation after the program has begun.

Ok, so where do I start? Start by learning as much as you can about the background and goals of the program. You can do this by reviewing a number of different documents:

Helpful Documents

• Technology plans. Technology plans will give you the history and intended uses of technology in the area. They will help you understand how plans have changed over the years, where people believe technology should be at different points in time, previous data collection activities, and what people believe will be the long-term benefits of technology.

• Information from previous evaluations. Was information collected prior to implementing the program? Was a "needs assessment" done? Were recommendations followed? It will be worth looking at this information so that you do not duplicate efforts and can learn from previous experiences.

• Curriculum and/or standards. Does the state, district, or school have a set curriculum or standards? Is technology a separate topic or is it integrated into different content areas? Looking at these documents will give you information on how technology is presently being infused into teaching.

• School board minutes. Reviewing school board meeting minutes may give you an idea of board member plans for technology use in schools.

Talk to People

In addition to reviewing documents, you will also want to speak with people who were involved in planning the program. They will be able to tell you who the major stakeholders are as well as the motivation behind the program.

Other Resources

Finally, you may want to look at other evaluations. Have any previous surveys been done at your school or district? Are there similar schools or districts doing a technology evaluation? You should talk to persons who planned the program, call your state technology coordinator to find out if other districts in your state are doing evaluations, and use the Internet to see what other states and districts are doing. Indeed, technology itself is a wonderful time-saving tool when designing and conducting evaluations. Many associations, states, and school districts have placed evaluation designs and examples of surveys on the Internet. An example of a good place to start is the U.S. Department of Education's website (http://www.ed.gov/Technology).

Where Did Rivers Start? The program was already underway before Kathy was asked to do the evaluation, so she was not able to provide input into planning. In addition, the district had not collected any technology information to date. Thus, her next step was to assess the different pieces of documentation and the human resources available to help her devise an evaluation strategy. The following is a list of the documents she was able to obtain: Document Comments State Technology Plan Plan developed two years ago by state tech coordinator Committee formed (teachers, tech people, school board) Major tasks - wiring, hardware, and professional development Wiring to be completed in two years (NOW!) Goal is to use technology in new media centers to get students connected to the world District Technology Plan Written last year by committee of teachers Plan to have wiring done this year, hardware next year Focus on elementary schools first, then middle, then high Goal is to use technology to improve reading achievement Curriculum/Standards State standards and curriculum are somewhat different from the district curriculum. The state curriculum consistsof a set of skills that students are expected to master: among these skills are technology competencies. The district curriculum does not mention technology much, but when it does it is in the context of sample lesson plans or activities to foster a concept (e.g., incorporates using computers for writing in English and for conducting a science fair project). After reviewing documents, Kathy talked to a few people to get more information. First, she called the state technology coordinator to get some background on the state plan and information on which other districts in the state may be evaluating technology programs. She found out that the state plan was written mostly by the technology coordinator with the input of a technology committee. She also found out that a technology coordinator from a district about two hours away had also recently called requesting evaluation guidance. Kathy got this person's number, called him, and they decided to share information with one another and to meet regularly. Finally, Kathy talked with the principal, the superintendent, and the professional development coordinator. They gave her a brief overview of district efforts and pointed her to some printed resources. She then went to the library in the district office to look up these resources and look for additional information on other district evaluations. Using the Internet and a CD-ROM database, she found many examples she could draw on, including sample surveys.

"Where Do I Start? " Worksheet

What documents are available (e.g., state and/or local technology plans)?

____________________________________________________________________________

____________________________________________________________________________

Ask questions about the plans. What major stakeholders were involved in creating the documents that were collected? Why were the plans written? Are the state and local plans similar (e.g., have similar timeframes)? Are the plans being implemented? How have plans changed over the years? What are some major goals in the plans (e.g., "use technology to improve student writing skills")?

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

____________________________________________________________________________

How is technology integrated into the curriculum and/or standards?

____________________________________________________________________________

____________________________________________________________________________

Whom could you talk to about the evaluation?

____________________________________________________________________________

____________________________________________________________________________

Who else may be collecting data you can use?

____________________________________________________________________________

Are there any other resources that you learned about by talking to people?

____________________________________________________________________________

____________________________________________________________________________

What does the Internet have to offer?

____________________________________________________________________________

____________________________________________________________________________


-###-

[What is an Evaluation Anyway?]

[What Questions Should I Ask?]