Your evaluation team's research questions will drive its selection of research methods. Selecting methods before identifying questions is like putting the cart before the horse. Research methods fall into two general categories: quantitative and qualitative. Once your evaluator helps your team choose the approach that is most appropriate, you can go on to identify specific research methods that will help answer the evaluation questions.

Quantitative vs. Qualitative Approaches

Generally speaking, quantitative approaches to data collection deal with numbers and answer the questions who, what, where, and how much. Qualitative approaches deal with words (stories) and answer the questions why and how.

For example: Quantitative: Disciplinary reports reveal a 10 percent decrease in incidents of physical fighting on school premises. Qualitative: According to a participant in the Peers Making Peace program, "I have learned a lot about negotiation and mediation in this program, and I've actually managed to help some students resolve their conflicts peacefully here at school. It's a really good feeling!"

Quantitative data can be counted, measured, and reported in numerical form. This approach is useful for describing concrete phenomena and for statistically analyzing your results (e.g., calculating the percentage decrease of cigarette use among 8th grade students). Some examples of quantitative data include test scores, attendance rates, drop-out rates, and survey rating scales.

Benefits of collecting quantitative data include the following:

Tools can be delivered more systematically.	Data are easily compiled for analysis.
Tools can be used with large numbers of study participants.	Findings can be presented succinctly.
Tools tend to be standardized, allowing for easy comparison within and across studies.	Findings are more widely accepted as generalizable.

Qualitative data are reported in narrative form. Examples include written descriptions of program activities, testimonials of program effects, comments about how a program was or was not helpful, stories, case studies, analyses of existing files, focus groups, key informant interviews, and observations. You can use qualitative information to describe how your program functions and what it means to the people involved. Through qualitative data, you can place your program in context, and better understand and convey people's perceptions of and reactions to it.

Benefits of collecting qualitative data include the following:

It promotes understanding of diverse stakeholder perspectives (e.g., what the program means to different people).	Stakeholders may find quotes and anecdotes easier to understand and more appealing.
It may reveal or shed light on unanticipated outcomes.	It can generate new ideas and/or theories.

Click here for a detailed overview of different quantitative and qualitative data collection methods.

Benefits of a Mixed-Method Approach

The ideal evaluation uses a combination of quantitative and qualitative methods. A mixed-method approach offers a range of perspectives on your program's processes and outcomes. Benefits of this type of approach include the following:

• It increases the validity of your findings by allowing you to examine the same phenomenon in different ways. This approach -- sometimes called triangulation -- is most often mentioned as the main advantage of the mixed-method approach. Think of a stool; there must be at least three legs for it to be stable. Each leg represents a different approach to or method of data collection, holding up the program as represented by the seat.

• It can result in better data collection instruments. For example, it is beneficial to conduct focus groups to inform the development or selection of a questionnaire.

• It promotes greater understand your findings. Quantitative data can show that change occurred and how much change took place, while qualitative data can help you understand why.

• It offers something for everyone. While some stakeholders may respond more favorably to a presentation featuring charts and graphs, others may prefer anecdotes and stories.

Although it may increase the expense and complexity of your evaluation, a mixed-method approach is still -- resources permitting -- the way to go. By using different sources and methods at various points in the evaluation process, your evaluation team can build on the strengths of each type of data collection and minimize the weaknesses of any single approach.

Protecting Program Participants

Before gathering data, your evaluation team should decide how to address the following important issues:

• Consent. Everyone participating in your evaluation must be informed about the study's purpose, what will be expected of them, and possible benefits and drawbacks of participation. If they agree to participate, they must be given the opportunity to skip questions or stop participating at any time, with no negative consequences.

Active Parental Consent Any data collection effort funded by the U.S. Department of Education that asks minors about the following topics requires active, or written, consent from their parents/guardians: political affiliations or beliefs of the student or the student's family, mental and psychological problems of the student or the student's family, sex behavior or attitudes, illegal, anti-social, self-incriminating, or demeaning behavior, critical appraisals of other individuals with whom respondents have close family relationships, legally recognized privileged or analogous relationships, such as those of lawyers, physicians, and ministers; religious practices, affiliations, or beliefs of the student or student's family, income (other than that required by law to determine eligibility for participation in a program or for receiving financial assistance under such program).

• Confidentiality. This means that participants' responses will not be shared with anyone outside your evaluation team, unless the information shows that a participant has an imminent intent to harm himself or herself or others. Confidentiality protects the privacy of participants and thus increases the likelihood that they will respond to your questions candidly.

• Anonymity. Whenever possible, try to collect data in a manner that allows participants to remain anonymous. This means that no one, including members of your evaluation team, has the capacity to match participants to their responses.

It is important that your evaluation team becomes familiar with the U.S. Department of Education's (USED) Family Educational Rights and Privacy Act (FERPA) and Protection of Pupil Rights Amendment (PPRA). These federal laws, which apply to schools and programs that receive USED funding, are designed to protect the rights of parents and students during the research process. Click here to visit the USED Web site to learn more about these important provisions.

Discussion Questions Please think about the questions below and share your responses, comments, and/or any questions about today's material in the Discussion Area. How do you think the various stakeholders in your school and community would respond to quantitative vs. qualitative approaches to collecting and presenting evaluation data? What approach(es) and method(s) has your evaluator recommended, and why? Are you using, or do you plan to use, a mixed-method approach to the evaluation of your school's prevention activities? If so, what are the various methods you are using or might you use? How do they complement one another? What steps has your team taken, or will it take, to protect the privacy of students and families during the data collection process?

This completes today's work. Please visit the Discussion Area to share your responses to the discussion questions!

References for Day 4 materials:

Central Center for the Application of Prevention Technologies. Approaches to prevention evaluation. Available on-line at: http://www2.miph.org/capt_eval/.

Frechtling, J., Sharp, L., & Westat (Eds.). User-friendly handbook for mixed-method evaluations. Available on-line at: http://www.ehr.nsf.gov/EHR/REC/pubs/NSF97-153/start.htm.

McNamara, C. Basic guide to program evaluation. Available on-line at:

http://www.mapnp.org/library/evaluatn/fnl_eval.htm#anchor1581634.

Wandersman, A., Imm, P., Chinman, M., Kaftarian, S. Getting to outcomes: Methods and tools for planning, self-evaluation, and accountability. Volume 1 available on-line at: http--www.stanford.edu-~davidf-GTO_Volume_I.pdf. Volume 2 available on-line at: http--www.stanford.edu-~davidf-GTO_Volume_II.pdf.