ࡱ > - / $ % & ' ( ) * + , 3 bjbj}+}+ q A A F l B B B ^ v v v
І 4 T
d ( Ȝ Ȝ Ȝ 6 * < f ' ) ) ) ) ) ) $ ~ r M ] v " Ҟ " M b v v Ȝ Ȝ b b b v Ȝ v Ȝ ' b ' b b v j , v v [ Ȝ X t}6
t # 2 [ 0 U b [ b
v v v v FAmerican Institutes for Research
A Review of the Literature in Adult Numeracy: Research and Conceptual Issues
Prepared By:
The American Institutes for Research
1000 Thomas Jefferson Street, N.W.Washington, D.C. 20007
Prepared For:
Office of Vocational and Adult EducationDivision of Adult Education and LiteracyU.S. Department of EducationWashington, D.C.
Project Staff:
Principal Investigator:Larry Condelli, AIRCo-Project Director:Kathy Safford-Ramus,
Saint Peters College,
Jersey City, New JerseyCo-Project Director:
Renee Sherman, AIR
Diana Coben, Kings College,
London
Iddo Gal, University of Haifa,
Haifa, Israel
Anestine Hector-Mason, AIR
March 27, 2006
American Institutes for Research
1000 Thomas Jefferson Street, NW | Washington, DC 20007-3835
Table of Contents
TOC \h \z \t "H2,1,H3,2,H4,3" HYPERLINK \l "_Toc130973115" 1. Introduction PAGEREF _Toc130973115 \h 1
HYPERLINK \l "_Toc130973116" Organization of the Review PAGEREF _Toc130973116 \h 2
HYPERLINK \l "_Toc130973117" 2. Issues in Conceptualizing Adult Numeracy PAGEREF _Toc130973117 \h 5
HYPERLINK \l "_Toc130973118" Definitions of Numeracy PAGEREF _Toc130973118 \h 5
HYPERLINK \l "_Toc130973119" Formative Phase PAGEREF _Toc130973119 \h 6
HYPERLINK \l "_Toc130973120" Mathematical Phase PAGEREF _Toc130973120 \h 7
HYPERLINK \l "_Toc130973121" Integrative Phase PAGEREF _Toc130973121 \h 7
HYPERLINK \l "_Toc130973122" Theories on Learning and Knowing Mathematics PAGEREF _Toc130973122 \h 11
HYPERLINK \l "_Toc130973123" Behaviorism PAGEREF _Toc130973123 \h 11
HYPERLINK \l "_Toc130973124" Constructivist Theories of Learning PAGEREF _Toc130973124 \h 11
HYPERLINK \l "_Toc130973125" Socio-Cultural Perspectives on Mathematics PAGEREF _Toc130973125 \h 13
HYPERLINK \l "_Toc130973126" Numeracy and Cognition: Experience and Situations PAGEREF _Toc130973126 \h 14
HYPERLINK \l "_Toc130973127" Affective Factors, Anxiety, and Learning Styles PAGEREF _Toc130973127 \h 16
HYPERLINK \l "_Toc130973128" Learning Styles PAGEREF _Toc130973128 \h 18
HYPERLINK \l "_Toc130973129" Brain Research: A New Science of Learning PAGEREF _Toc130973129 \h 19
HYPERLINK \l "_Toc130973130" Summary PAGEREF _Toc130973130 \h 20
HYPERLINK \l "_Toc130973131" 3. Adult Numeracy and Mathematics Instructional Approaches and Interventions PAGEREF _Toc130973131 \h 21
HYPERLINK \l "_Toc130973132" Professional Society Instructional Standards PAGEREF _Toc130973132 \h 21
HYPERLINK \l "_Toc130973133" National Council of Teachers of Mathematics Standards PAGEREF _Toc130973133 \h 21
HYPERLINK \l "_Toc130973134" Crossroads PAGEREF _Toc130973134 \h 22
HYPERLINK \l "_Toc130973135" Other Frameworks PAGEREF _Toc130973135 \h 22
HYPERLINK \l "_Toc130973136" Research Evaluating Instructional Approaches PAGEREF _Toc130973136 \h 23
HYPERLINK \l "_Toc130973137" Methodology for Identifying Research PAGEREF _Toc130973137 \h 23
HYPERLINK \l "_Toc130973138" Quality of Research PAGEREF _Toc130973138 \h 25
HYPERLINK \l "_Toc130973139" Effective Instruction for Adult Learners: Research Findings PAGEREF _Toc130973139 \h 25
HYPERLINK \l "_Toc130973140" Use of Technology in Instruction PAGEREF _Toc130973140 \h 26
HYPERLINK \l "_Toc130973141" Computer Use in ABE Instruction PAGEREF _Toc130973141 \h 27
HYPERLINK \l "_Toc130973142" Computer Use in Correctional Education PAGEREF _Toc130973142 \h 28
HYPERLINK \l "_Toc130973143" Computer Use in GED Instruction PAGEREF _Toc130973143 \h 29
HYPERLINK \l "_Toc130973144" Computer Use in Developmental Mathematics Instruction PAGEREF _Toc130973144 \h 30
HYPERLINK \l "_Toc130973145" Research Evaluating Instruction Based on Constructivist Theories PAGEREF _Toc130973145 \h 31
HYPERLINK \l "_Toc130973146" Cooperative Learning PAGEREF _Toc130973146 \h 31
HYPERLINK \l "_Toc130973147" Discovery Learning PAGEREF _Toc130973147 \h 32
HYPERLINK \l "_Toc130973148" ABE Instructional Interventions PAGEREF _Toc130973148 \h 33
HYPERLINK \l "_Toc130973149" Summary PAGEREF _Toc130973149 \h 34
HYPERLINK \l "_Toc130973150" 4. Assessment Issues in Adult Numeracy PAGEREF _Toc130973150 \h 35
HYPERLINK \l "_Toc130973151" The Purposes of Assessments in Adult Numeracy Education and Related Research PAGEREF _Toc130973151 \h 35
HYPERLINK \l "_Toc130973152" Purposes of Assessment PAGEREF _Toc130973152 \h 35
HYPERLINK \l "_Toc130973153" Related Research PAGEREF _Toc130973153 \h 36
HYPERLINK \l "_Toc130973154" Overview of Widely Used Instruments for ABE Numeracy Assessment PAGEREF _Toc130973154 \h 38
HYPERLINK \l "_Toc130973155" TABE PAGEREF _Toc130973155 \h 38
HYPERLINK \l "_Toc130973156" CASAS PAGEREF _Toc130973156 \h 39
HYPERLINK \l "_Toc130973157" A Critical Look PAGEREF _Toc130973157 \h 40
HYPERLINK \l "_Toc130973158" Defining the Meaning of Assessment and How It Can Be Improved PAGEREF _Toc130973158 \h 42
HYPERLINK \l "_Toc130973159" Considerations for Developing Improved Assessments for Adult Numeracy PAGEREF _Toc130973159 \h 44
HYPERLINK \l "_Toc130973160" What Should Be Assessed in Adult Numeracy Instruction? PAGEREF _Toc130973160 \h 45
HYPERLINK \l "_Toc130973161" Conclusions PAGEREF _Toc130973161 \h 47
HYPERLINK \l "_Toc130973162" 5. Professional Development in Adult Numeracy PAGEREF _Toc130973162 \h 49
HYPERLINK \l "_Toc130973163" Issues in Professional Development PAGEREF _Toc130973163 \h 49
HYPERLINK \l "_Toc130973164" Need For Professional Development in Numeracy PAGEREF _Toc130973164 \h 50
HYPERLINK \l "_Toc130973165" Overlapping Relationship Between Literacy and Numeracy PAGEREF _Toc130973165 \h 52
HYPERLINK \l "_Toc130973166" Research in Professional Development and Adult Numeracy PAGEREF _Toc130973166 \h 53
HYPERLINK \l "_Toc130973167" International Efforts PAGEREF _Toc130973167 \h 56
HYPERLINK \l "_Toc130973168" Summary PAGEREF _Toc130973168 \h 57
HYPERLINK \l "_Toc130973169" 6. Summary and Implications for Future Research PAGEREF _Toc130973169 \h 59
HYPERLINK \l "_Toc130973170" Definitions and Theories of Adult Numeracy PAGEREF _Toc130973170 \h 59
HYPERLINK \l "_Toc130973171" Instructional Approaches and Interventions for Adult Numeracy PAGEREF _Toc130973171 \h 59
HYPERLINK \l "_Toc130973172" Implications for Further Research PAGEREF _Toc130973172 \h 61
HYPERLINK \l "_Toc130973173" Further Research on Instruction PAGEREF _Toc130973173 \h 61
HYPERLINK \l "_Toc130973174" Research on Mathematics Instruction for Children PAGEREF _Toc130973174 \h 62
HYPERLINK \l "_Toc130973175" Development of Improved Assessment PAGEREF _Toc130973175 \h 63
HYPERLINK \l "_Toc130973176" Research to Improve Professional Development PAGEREF _Toc130973176 \h 65
HYPERLINK \l "_Toc130973177" Conclusion: Moving Forward PAGEREF _Toc130973177 \h 66
HYPERLINK \l "_Toc130973178" References Cited PAGEREF _Toc130973178 \h 67
HYPERLINK \l "_Toc130973179" References Consulted PAGEREF _Toc130973179 \h 84
HYPERLINK \l "_Toc130973180" Appendix A Definitions of Numeracy A- PAGEREF _Toc130973180 \h 1
HYPERLINK \l "_Toc130973181" Appendix B Professional Society Standards B- PAGEREF _Toc130973181 \h 1
HYPERLINK \l "_Toc130973182" Appendix C List of Web Sites and Journals Searched C- PAGEREF _Toc130973182 \h 1
HYPERLINK \l "_Toc130973183" Appendix D Summary of Mathematics and Numeracy Intervention Studies Reviewed D- PAGEREF _Toc130973183 \h 1
1. Introduction
The increasing need for numeracy skills in all aspects of adult lifefamily, employment, communityhas made numeracy a requisite skill for success in todays society. In the context of Adult Basic Education (ABE), more emphasis is needed on providing quality numeracy instruction to adults to help them achieve the mathematical knowledge and skills that will enable them to adjust to this growing societal demand. The Office of Vocational and Adult Education (OVAE) recognized the need for learners to improve their numeracy skills when it identified mathematics as a core academic area for the development of rigorous content standards. As adult learners are encouraged to move beyond the General Educational Development (GED) and into postsecondary education, the development of numeracy skills will become more critical. However, as educational assessments have shown, 35 percent of all U.S. students are scoring below basic on the National Assessment of Educational Progress (NAEP) (NCES, 2002, Math Assessment), with even higher proportions of Hispanic, African American, and low-income students scoring below basic.
This problem is of significant concern to adult educators because an increasing number of 18- to 25-year-olds are enrolling in adult education programs: the very same students who lack numeracy skills. The concern about the numerous skill deficiencies in todays adult learners is exacerbated by the fact that adult education programs are not adequately prepared to provide numeracy education to a diverse student population that bring different needs, interests, skills, and behavior and attitudes toward numeracy.
Although numeracy instruction plays a significant role in adult education in many countriesnotably Australia, the Netherlands, and, more recently, the United Kingdomthe United States has experienced limited attention to numeracy instruction and little research on how local adult education programs teach mathematics or numeracy. There are many reasons for this lack of focus: little agreement on what constitutes numeracy; poor professional development in numeracy; limited understanding of how adults with diverse characteristics, needs, and backgrounds obtain numeracy skills; and the lack of alignment among content standards, curricula and instruction, and assessments. Schmitt (2002) points out that GED preparation has been the driving force in mathematics instruction in most adult education programs. Workbooks focusing on standard computational rules, fractions, whole numbers, decimals, percentages, and prealgebra drive instruction. Exercises tend to emphasize repetitive problems, word problems, and problems with real-life applications.
The Adult Numeracy Initiative is the first major effort of the U.S. Department of Education to improve the research and practice of adult numeracy. This project has several goals:
Develop a thorough understanding of the current state of the field of adult numeracy.
Identify the gaps in knowledge about common strategies for teaching adult numeracy and how these strategies differ across different types of learners.
Identify the type of professional development and teacher certification that should be required for teachers of adult mathematics.
Identify the type of assessment instruments that might be appropriate for measuring adult quantitative skill acquisition.
The project will achieve these goals through two phases of activity. The first phase, consisting of a literature review, an environmental scan, a technical working group convened to identify critical issues, and commissioned papers to address some of these issues, will distill the limited body of knowledge about current research and practice on adult numeracy instruction, assessment, and professional development. The information from this first phase will inform the second part of the project, which entails the design of curriculum and professional development materials and activities and the design of a research-based intervention or demonstration program that can be tested with a rigorous methodology.
These activities of the Adult Numeracy Initiative seek to answer the following research questions posed by OVAE for the project:
How does adult numeracy develop and how does it differ from the development of quantitative literacy in children?
What are the social variables that affect quantitative skill acquisition in adults? How should programs address these social variables to enhance skill acquisition?
What instructional practices exist in mathematics education for adult learners that are worthy of replication?
What outcomes are most important to address in the evaluation of adult education programs in mathematics? What are the best tools or assessments for evaluating these outcomes?
What practices exist in professional development and certification requirements for teachers of adult mathematics education that are worthy of replication?
What types of programs have been implemented at the state and local levels through federal funding that incorporate or focus on adult mathematics instruction?
What types of programs have been implemented at the state and local levels through federal funding that focus on adult mathematics instruction related to adult English language acquisition learners?
This review addresses the first five research questions by summarizing key findings from the literature. The report that will incorporate the findings from the environmental scan will address the final two research questions concerning mathematics instructional programs.
Organization of the Review
This literature review is the first report of the Adult Numeracy Initiative and lays the foundation for the other substantive activities in the projects first phase. It strives to answer the first five research questions posed by the Statement of Work. The review summarizes the definitions, theories, and research around adult numeracy to organize what is known and to point the way for future research and development. We have organized the literature review into five parts:
Issues in Conceptualizing Adult Numeracy addresses the first and second research questions and presents an overview of the competing approaches to defining adult numeracy. The section then reviews the main theoretical approaches toward teaching and learning mathematics for adults that reflect these definitions. The concepts and theories in this section provide a background and context for the research in instruction and assessment presented in subsequent sections.
Adult Numeracy and Mathematics Instructional Approaches and Interventions reviews the small number of studies done on ABE students, supplemented with an also small number of studies on adults in community college developmental education programs, on the effects of different types of instructional approaches on mathematics learning among adults. In response to the third research question, this section presents the instructional approaches, findings, and methodologies of these studies.
Assessment Issues in Adult Numeracy endeavors to respond to the fourth research question as it summarizes the existing knowledge base regarding assessment in adult numeracy, reviews the uses of assessment, analyzes the nature of assessment and how it can be improved, and summarizes the principles for designing effective assessments for adult numeracy.
Professional Development in Adult Numeracy deals with the fifth research question and discusses the state of professional development in ABE around numeracy. The section notes the low-level background of ABE teachers to teach mathematics, presents professional development approaches, and briefly discusses research on the characteristics of effective professional development.
Summary and Implications for Future Research summarizes the findings of the review and suggests future research and how the field of adult numeracy practice and research might progress. It also briefly suggests how research on childrens learning of mathematics may inform research on adults, thus also addressing the first research question.
Writing a literature review is one of the researchers more difficult tasks. Unless it is to be a lifes work, lines must be drawn around the topic to identify what is important, what is to be included, and how it is to be analyzed. This task is even more difficult in the field of adult numeracy, which Diana Coben has aptly called a moorland, where the lines, where they exist at all, are often indistinct and vaguely drawn. The Adult Numeracy Initiatives research questions identified the general topics we were to cover. Within these topics, two goals guided our approach: (1) to identify the areas of greatest interest to furthering research and practice of adult numeracy instruction and professional development within the U.S. adult education system and (2) to set the stage for the other activities of the project.
The review suggests areas to pursue further in the environmental scan and commissioned papers, including ongoing work on the development of instruction approaches and curricula and professional development. The technical working group and the commissioned papers will also address many of the conceptual and theoretic issues identified in the review.
2. Issues in Conceptualizing Adult Numeracy
The construct numeracy does not have a universally accepted definition, nor agreement about how it differs from mathematics (Gal, van Groenestijn, Manly, Schmitt, & Tout, 2005).
This statement, by the authors of a recent international report on adult numeracy, indicates a fundamental problem for anyone reviewing the research literature in this area: there is as yet no consensus about the nature of adult numeracy. Numeracy is a deeply contested concept, beset by terminological confusion, especially when referring to adults. A plethora of similar and loosely related terms compete for attention: mathematical literacy, techno-mathematical literacy, quantitative literacy, functional mathematics, mathemacy, and so on. The resultant complexities are discussed in depth in Adult Numeracy: Review of Research and Related Literature ADDIN EN.CITE Coben20033244 DC Library0605-Converted.enlEndNote324427(Coben, 2003).
The issues of the definition of numeracy may seem to be an academic exercise, with little practical value. However, how numeracy is defined has profound implications for all issues of concern to the Adult Numeracy Initiative. Definitions of numeracy have implications for what adults need to know, what should be taught, how students should be assessed, and what professional development teachers need, as a recent international comparative study of adult numeracy frameworks makes clear (Hagedorn et al., 2003). In this section, we summarize the conceptualizations of numeracy and learning theory related to how adults learn mathematics and numeracy.
Definitions of Numeracy
The term numeracy originated in the United Kingdom in the Crowther Report on the education of children ages 1518. As the mirror image of literacy, numeracy was a way of bridging scientific and literary cultures ADDIN EN.CITE Ministry of Education1959133 DC Library0605-Converted.enlEndNote13327(Ministry of Education, 1959, . 389). The definition entailed not only the ability to reason quantitatively but also some understanding of scientific method and some acquaintance with the achievement of science. Literacy and numeracy, at a basic rather than an advanced level, have been yoked ever since, with numeracy often subsumed within literacy.
Definitions of numeracy have proliferated. One view equates numeracy with mathematics and computational skills, in much the same way that literacy is viewed as mastery of basic reading and writing. A much broader view focuses on peoples capacity and propensity to interact effectively and critically with the quantitative aspects of the adult world (Gal, 2002a). Similarly, in relation to literacy, some argue that numeracy is subsumed in literacy, whereas others argue that debates about numeracy within the context of literacy limit the full operationalization of both concepts. Gal and Schmitt (1994) reported that some people prefer to use the term mathematical literacy, believing that numeracy is toolimiting in scope. Others feel just the opposite, taking numeracy to be the mirror image of literacywhile viewing mathematical literacy as a sub-area of mathematics (p. ii). Appendix A presents a sampling of definitions of numeracy.
Maguire and ODonoghues (2002) organizing framework (Exhibit 1), developed through discussions with researchers and practitioners in Adults Learning Mathematics A Research Forum (ALM), offers a way of bringing some order into the conceptual confusion surrounding adult numeracy. In the framework, concepts of numeracy are arranged along a continuum of increasing levels of sophistication. In the formative phase, numeracy is considered to be basic arithmetic skills; in the mathematical phase, numeracy is in context, with explicit recognition of the importance of mathematics in everyday life. The third phase, the integrative phase, views numeracy as a multifaceted, sophisticated construct incorporating the mathematics, communication, cultural, social, emotional, and personal aspects of each individual in context.
Exhibit 1.Adult Numeracy Concept Continuum of Development
Phase 1Phase 2Phase 3Increasing levels of sophisticationFORMATIVE
(basic arithmetic skills)MATHEMATICAL
(mathematics in context of everyday life)INTEGRATIVE
(mathematics integrated with the cultural, social, personal, and emotional)A continuum of development of the concept of numeracy showing increased level of sophistication from left to right (from Maguire & ODonoghue, 2002)Formative Phase
Conceptions of numeracy following the Crowther Report lost the sophistication of the original definition. For example, numeracy first appeared in the UNESCO International Standard Classification of Education in 1997 as Literacy and numeracy: Simple and functional literacy, numeracy. The designations simple, with respect to content and skills, and functional, with respect to purpose and application, are telling: numeracy in these definitions refers to basic mathematical, or sometimes specifically numerical or quantitative, skills, which adults are deemed to need to function effectively in society. In this view, numeracy is a basic skill normally acquired in childhood; in some versions of numeracy, what adults are deemed to need is simple arithmetic. Evans ADDIN EN.CITE Evans2000164 DC Library0605-Converted.enlEndNote1646(2000) calls this the limited proficiency model of numeracy, a hangover from the Victorian period when the 3Rs of reading, (w)riting, and (a)rithmetic held sway in elementary education. As a corollary, because the content is seen as simple, numeracy may also be thought to be easy to learn, a view roundly rejected by Ma ADDIN EN.CITE Ma19992534 DC Library0605-Converted.enlEndNote25346(1999). This view of numeracy is located in Maguire and ODonoghues formative phase.
Mathematical Phase
A broader view of numeracy may also be traced back to the United Kingdom, in the 1982 Cockcroft Report. This view epitomizes Maguire and ODonoghues mathematical phase, with its emphasis on the use of mathematics in daily life:
We would wish numerate to imply the possession of two attributes. The first of these is at-homeness with numbers and an ability to make use of mathematical skills, which enable an individual to cope with the practical mathematical demands of his everyday life. The second is ability to have some appreciation and understanding of information which is presented in mathematical terms, for instance in graphs, charts or tables or by reference to percentage increase or decrease. ADDIN EN.CITE DES/WO1982111 DC Library0605-Converted.enlEndNote11127(Department of Education and Science/Welsh Office, 1982,. 39)
In this phase, numeracy often includes number, money, and percentages; aspects of algebraic, geometric, and statistical thinking; and problem solving based on the mathematical demands of adult life. This view of numeracy has been influential in the Anglophone world, including the United Kingdoms Adult Numeracy Core Curriculum (Basic Skills Agency, 2001). In the United States, this approach appears as part of functional literacy approaches, exemplified in the CASAS framework (2004), in several states mathematics instructional content standards, and in the National Adult Literacy Survey (NALS) (NCES, 1992) and it successor, the National Assessment of Adult Literacy (NAAL) (NCES, 2005). These national surveys measure quantitative literacy, a concept that clearly falls within this applied mathematics phase.
However, the issue of functionality is not straightforward. Numeracy could be functional with respect to a wide range of contexts and purposes, and the practical mathematical demands of everyday life may require more than basic numeracy. This complexity is acknowledged in Maguire and ODonoghues integrative phase.
Integrative Phase
All of the most recent, influential approaches to defining adult numeracy fall into Maguire and ODonoghues integrative phase. In this phase, numeracy is viewed as a complex, multifaceted, and sophisticated construct, incorporating the mathematics, communication, cultural, social, emotional, and personal aspects of each individual in context. FitzSimons and Coben (in press) argue that numeracy in this sense may empower individuals as knowledge producers as well as knowledge consumers(that is, to become technologically, socially, personally, and/or democratically numerate.
Steen ADDIN EN.CITE Steen19902536 DC Library0605-Converted.enlEndNote253617(1990) exemplifies this phase when he outlines five dimensions of numeracy, distinguished in terms of their purposes and associations:
Practical, concerning mathematical and statistical skills that can be put to immediate use in the routine tasks of daily life
Civic, where the focus is on benefits to society
Professional, because many jobs require mathematical skills
Recreational, for the appreciation of games, puzzles, sports, lotteries, and other leisure activities
Cultural, concerned with mathematics as a universal part of human culture
Maguire and ODonoghues integrative phase also encompasses critical concepts of numeracy that eschew any automatic association with low-level mathematics. For example, Johnston (1995) proposes that
To be numerate is more than being able to manipulate numbers, or even being able to succeed in school or university mathematics. Numeracy is a critical awareness, which builds bridges between mathematics and the real world, with all its diversity. [...] in this sense ... there is no particular level of mathematics associated with it: it is as important for an engineer to be numerate as it is for a primary school child, a parent, a car driver or gardener. The different contexts will require different mathematics to be activated and engaged in. (p. 34)
Coben (2000a) also emphasizes the individuals judgments about the use (or not) of mathematics in a given situation:
To be numerate means to be competent, confident, and comfortable with ones judgments on whether to use mathematics in a particular situation and if so, what mathematics to use, how to do it, what degree of accuracy is appropriate, and what the answer means in relation to the context. (p. 10)
Although integrative conceptions of adult numeracy dominate almost all current theorizing and thinking in adult numeracy, this view has only just begun to move beyond a limited core of numeracy researchers and practitioners. Most mainstream practice continues to reflect formative and mathematical approaches to numeracy. However, more integrative approaches to numeracy have become influential over the last few years, as illustrated by projects to define numeracy instructional content standards, the Program for International Student Assessment (PISA), and the Adult Literacy and Lifeskills (ALL) Survey. The numeracy definitions in these projects specify the intended cognitive outcomes of adult numeracy education and/or emphasize the need for the individual to adjust to the increasing technological demands of the knowledge economy.
The introduction of the National Council of Teachers of Mathematics Curriculum and Evaluation Standards for School Mathematics (NCTM, 1989) helped fuel the instructional standards reform movement in mathematics and numeracy (discussed further in the next section). These standards emphasized conceptual understanding and the development of problem-solving and decision-making skills, rather than rule-based learning. This view of mathematics and numeracy has had an impact not only in the teaching of mathematics to children but also in adult education and has helped prompt a movement toward the development of content standards for teaching adult mathematics. The state of Massachusetts (Leonelli & Schwenderman, 1994) and the Adult Numeracy Networks (ANN) mathematics standards framework were among the first attempts at developing an integrative numeracy framework for adult instruction. The following seven themes were proposed to serve as the foundation for the development of numeracy standards (Curry, Schmitt, & Waldron, 1996):
Relevance/Connections
Problem-Solving/Reasoning/Decision-Making
Communication
Number and Number Sense
Data
Geometry: Spatial Sense and Measurement
Algebra: Patterns and Functions
Several states have moved toward developing mathematics content standards, using basic computational, functional, or integrative approaches, and eight states have already developed mathematics standards. OVAEs Adult Education Content Standards Warehouse project (http://www.adultedcontentstandards.org) has supported states efforts to develop standards, as has the National Institute for Literacys (NIFL) Equipped for the Future (EFF) project. EFFs Math Content Standard states that adults should be able to Use Math to Solve Problems and Communicate (see Exhibit 2) after participation in adult basic education.
Exhibit 2.EFF Standard: Use Math to Solve Problems and Communicate
Understand, interpret, and work with pictures, numbers, and symbolic information.
Apply knowledge of mathematical concepts and procedures to figure out how to answer a question, solve a problem, make a prediction, or carry out a task that has a mathematical dimension.
Define and select data to be used in solving the problem.
Determine the degree of precision required by the situation.
Solve problems using appropriate quantitative procedures and verify that the results are reasonable.
Communicate results using a variety of mathematical representations, including graphs, charts, tables, and algebraic models.From National Institute for Literacy (2000)
The Organization for Economic Co-operation and Developments (OECD) PISA also focuses on using mathematics. It is designed to assess the readiness of 15-year-olds for life beyond school, focusing on the extent to which students are able to use their knowledge and skills to meet real-life challenges. This reflects a change in curricular goals and objectives in many countries, which are increasingly concerned with what students can do with what they learn at school (OECD, 2003).
Mathematical literacy is defined in PISA as
an individuals capacity to identify and understand the role that mathematics plays in the world, to make well founded judgments and to use and engage with mathematics in ways that meet the needs of that individuals life as a constructive, concerned and reflective citizen. (p. 24)
It encompasses four domains or subscales:
Space and shape, which includes recognizing shapes and patterns
Change and relationships, which includes data analysis needed to specify relationships or translate between representations
Quantity, which focuses on quantitative reasoning and understanding of numerical patterns, counts and measures
Uncertainty, which includes statistics and probability
The second PISA survey, which included the United States, covered reading, mathematical and scientific literacy, and problem solving, with a primary focus on mathematical literacy. Conducted in 2003 in 41 countries, it is usually referred to as PISA 2003. The United States ranked 28th out of 40 countries in terms of the percentage of students at each level of proficiency on the mathematics scale in PISA 2003.
In the most recent international survey of adult numeracy (including the United States), the Adult Literacy and Lifeskills (ALL) Survey, also conducted under the auspices of the OECD, the emphasis is on using mathematics in real contexts, including, but not limited to, everyday life. Numeracy is considered in the ALL Survey as the knowledge and skills required to manage and respond effectively to the mathematical demands of diverse situations; in addition,
Numerate behavior is observed when people manage a situation or solve a problem in a real context; it involves responding to information about mathematical ideas that may be represented in a range of ways; it requires the activation of a range of enabling knowledge, factors, and processes. (Gal et al., 2005, p. 142)
The issue of level of difficulty experienced by the individual adult when engaging in numerate behavior is tackled in the ALL Survey through an analysis of assessment items in terms of their textual and mathematical complexity. The inclusion of a textual dimension allows for the fact that numeracy is often mediated through text and difficulties in reading may impede numeracy performance. Text may be presented in digital form or on paper, and in the integrative phase, numeracy is often associated with information and communication technologies (ICT). For example, Steen (2001) describes numeracy as a natural tool for the computer age, encompassing the capacity to communicate using digital data.
Theories on Learning and Knowing Mathematics
Each of the main definitions of numeracy has ties to adult learning theory. In turn, the learning theories have profound implications for the content of mathematics instruction, its pedagogy, and how learning should be assessed ADDIN EN.CITE Forman1999193519356Forman, Susan L.Steen, Lynn ArthurBeyond Eighth Grade: Functional mathematics for life and work1999Berkeley, CA(Forman & Steen, 1999). Definition, theory, and instruction are thus tied together: ones view of what numeracy is leads to a theory of learning, and this theory affects preferred approaches to instruction. There remains controversy around the implication of theory to practice because there is little empirical research demonstrating the effects of instructional approaches implied by the theories on how adults know and learn about mathematics.
Behaviorism
Up until the mid-1990s, behaviorist approaches dominated adult mathematics instruction. In the behaviorist approach, learning is defined as a change in behavior observed when a stimulus results in a response. In behaviorist mathematics instruction, the teacher conveys knowledge, such as a number fact embedded in a word problem (the stimulus), to the students who absorb it and produce a solution (the response). Learning is considered to have occurred when the correct solution is given consistently. Learning mathematics in this mode entails immediate recall, retention, and transfer, and understanding is equated with computation and operations, as measured by achievement tests or performance tasks. This approach is associated with an absolutist view of mathematics, that is, the belief in the certainty and truth of mathematics. In the absolutist view, mathematics is a set of absolute truths determined by authority; doing mathematics means following the rules correctly ADDIN EN.CITE Coben20001041045Coben, DianaGal, IddoNumeracy, mathematics and adult learningAdult Numeracy Development: Theory, research, practice33-502000Cresskill, NJHampton Press(Coben, 2000a). Behaviorist methods dominated U.S. educational practice until the late 1950s.
Constructivist Theories of Learning
The last 10 years have brought a major shift in ideas about learning mathematics, from a behaviorist perspective to a constructivist perspective (Kieran, 1994) so that constructivism now has a markedly greater influence on contemporary mathematics education. The shift to constructivist theories of learning corresponds to the adoption of integrative definitions of adult numeracy. For example, constructivism underpins the NCTM mathematics standards discussed above. The keystone of constructivism is the notion that all knowledge is constructed by individuals acting upon external stimuli and assimilating new experiences by building a knowledge base or altering existing schemas. Learners actively construct knowledge by integrating new information and experiences into what they have previously come to understand, revising and reinterpreting old knowledge in order to reconcile it with the new ADDIN EN.CITE Billett19964497449717Studies in Continuing Education43-581811996(Billett, 1996). Two main strands in constructivism have emerged, following, on the one hand, Piaget (focusing on ways in which individual learners make sense of mathematics) or, on the other hand, Vygotsky (seeing learning as an activity in which shared mathematical meanings are constructed socially). Jaworski (1994) notes that debates between radical and social constructivists parallel debates between these two positions.
Piagetian theories in adult numeracy focus on the importance of an individuals cognitive developmental stage in learning. Piaget proposed four major developmental stages through which a child progresses intellectually from birth to adolescence. Much of the research at the K8 level has proposed materials and methods that promote student learning at the penultimate stage, the concrete level, and progress toward the formal operations level, the final stage. The concrete operational and formal operations levels have been the subjects of a few studies specific to adult populations. These studies include Mayta (1990), who correlated achievement in mathematics to the concrete stage among a group of incarcerated males, and Brockbader ADDIN EN.CITE Bockbrader19924791479132Bockbrader, B.Characteristics of adult rational number understanding. Unpublished doctoral dissertation1992ToledoUniversity of ToledoUMI No. 9220860(1992), Wolfe (1999), and Martelly ADDIN EN.CITE Martelly19984792479232Martelly, D. I.Effects of using manipulative materials to teach remedial algebra to community college students on achievement and attitudes towards mathematics. Unpublished doctoral dissertation1998UMI No. 9826062(1998), who found the same relationship among community college students enrolled in developmental mathematics courses. Although these studies do not suggest either materials or instructional approaches, they do refute, to varying degrees, Piagets conviction that the evolution to formal operations is complete by age 15; they also validate the use of concrete materials and manipulatives for adult students.
Another aspect of Piagets theory of intellectual development has received less attention in the adult numeracy field, his notion of intellectual growth as involving three fundamental processes: assimilation, accommodation, and equilibration. Assimilation is the process through which new events are incorporated into pre-existing cognitive structures. Accommodation involves changes in these structures to accommodate new information. Through this dual process of assimilation-accommodation, the learner forms schemata. Equilibration refers to the balance the learner strikes between his or her schemata and the environment and between assimilation and accommodation. A new experience causes disequilibrium until the learner is able to assimilate and accommodate the new information, thereby attaining equilibrium. Llorentes (1996) study of the problem-solving behavior of adults in Argentina with little formal education in work situations uses Piagets theory of equilibration to highlight the interactive and constructive nature of everyday knowledge and the social constraints that influence problem solving.
Vygotsky (1978) emphasized the social aspect of learning and the interplay of speech and action in childrens learning activities. To Vygotsky, affect, motivation, and will are central to learning. Two of his major contributions to constructivist theory were the ideas of a zone of proximate development (ZPD) and scaffolding, although the latter name was not actually used by Vygotsky (Wilson, Teslow, & Taylor, 1993). Vygotsky defined the ZPD as the distance between the actual developmental level as determined by independent problem solving and the level of potential development as determined through problem solving under adult guidance or in collaboration with more capable peers. Scaffolding is the process in which the student masters a skill under the guidance of an expert. There are four basic steps in the process (Wilson, Teslow, & Taylor, 1993):
The student observes the teacher modeling an activity.
The student tries the activity under the guidance of the teacher.
The teacher prompts with cues only when needed.
The student is free to practice the skill independently.
Vygotskys work has many applications to the teaching of mathematics and has been referenced by advocates of cooperative learning and problem-solving activities. Like Piaget, Vygotsky studied children, but his theories of ZPD and scaffolding both translate smoothly to the design of instruction for adult mathematics students.
Constructivists maintain that learning is an active process, that collaboration enhances learning, that learning should be situated in realistic settings, and that testing should be integral to the experience, not a separate activity ADDIN EN.CITE Merrill19914793479317Merrill, M. D.Constructivism and instructional designEducational TechnologyEducational Technology45-533151991(Merrill, 1991; Wilson, Teslow, & Taylor, 1993). At its most radical sense, constructivist theory holds that each person discovers truth and constructs his or her own unique knowledge base ADDIN EN.CITE von Glasersfeld1990389738975von Glasersfeld, ErnstDavis, Robert B.Maher, Carolyn A.Noddings, NelAn exposition of constructivism: Why some like it radicalConstructivist Views on Teaching and Learning Mathematics1990Reston, VANCTM(von Glasersfeld, 1990). Constructivism is associated with fallibilist views of mathematics, in which mathematics is seen as a social construct and therefore value laden, culturally determined, and open to revision. Benn (1977) argues that fallibilist approaches lead to more inclusive, adult-friendly teaching and learning; by contrast, the absolutist view is associated with the product view of mathematics, in which mathematical skills and concepts are seen as external to the learner. In constructivist epistemologies of mathematics education, mathematics education is viewed as a process whereby knowledge of mathematics is gained by doing mathematics.
Socio-Cultural Perspectives on Mathematics
The belief in a socially constructed base of knowledge does not negate the idea of the universal truth of mathematical ideas. But although the concept that 2 + 2 = 4 is true wherever and whoever you are, Bishop (1991) illustrates concepts such as the idea of negative numbers and the angles of a triangle adding to 180 degrees (as opposed to 100 or 150) as evidence of the cultural basis for mathematics. Bishops work is an example of ethnomathematics. Ethnomathematicians address the question of whose knowledge counts, challenging the hegemony of the Western model of mathematics from a variety of perspectives ADDIN EN.CITE Powell199734934928Powell, Arthur B.Frankenstein, MarilynEthnomathematics: Challenging Eurocentrism in Mathematics Education4401997Albany, NYState University of New York Press0 7914 3351 X(Powell & Frankenstein, 1997). Bishop has identified six pan-cultural mathematical activities: counting, locating, measuring, designing, explaining, and playing; the Western model of mathematics represents one approach to these, but there are others. Against this background, research and practice in ethnomathematics focus on the mathematics of cultural groups and the development of pedagogies that take these different forms of mathematics into account, establishing comparisons between academic mathematics knowledge and local knowledge and analyzing the power relations involved in the use of both kinds of knowledge ADDIN EN.CITE Knijnik19967607605Knijnik, GelsaKjrgrd, T.Kvamme, A.Lindn, N.Intellectuals and social movements: Examining power relationsNumeracy, Race, Gender, and Class: Proceedings of the third international conference on the Political Dimensions of Mathematics Education (PDME III)90-1131996Lands, NorwayCaspar Forlag(Knijnik, 1996).
An interesting example relative to ABE was conducted by Masingila (1992) in a work-place literacy setting. She contrasted the mathematics practices that carpet layers use when estimating and installing their product with the school-based knowledge of general mathematics students. Masingila found that the carpet layers engaged in conceptually deep mathematical thinking as they solved problems encountered during installations. These constraint-filled situations differed substantially from the textbook area problems that the students were required to solve. The straightforward school problems did not prepare the novice carpet layers for the realities of area, ratio and proportion, and measurement experienced on the job.
Feminist studies of mathematics represent another approach from the ethnomathematical, social constructivist view. This work has been influential in the mathematics education of girls and women and in research in gender studies. Becker (1995) sets out two types of knowing developed in feminist research: separate knowing (associated with men and concerned with such things as logic, rigor, abstraction, rationality, and certainty) and connected knowing (associated with women and concerned with intuition, creativity, hypothesizing, conjecture, and experience). She contends that in mathematics and science, separate knowledge is valued over connected knowledge and that this disadvantages girls. She advocates a connected approach to teaching and learning mathematics, through which, she contends, more girls would enjoy mathematics, succeed at it, and choose to study it to advanced levels. Feminists have also taken issue with assumptions about womens allegedly unmathematical minds and sought to explain womens and girls underachievement, where it exists or seems to exist, in cultural rather than biological terms, while seeking to understand and improve the teaching of mathematics to women and girls and their take up of and performance in mathematics ADDIN EN.CITE Burton19952528252817Burton, LeoneMoving towards a feminist epistemology of mathematicsEducational Studies in MathematicsEducational Studies in Mathematics275-291281995(L. Burton, 1995).
A representative study in this vein is Crittenden (2000), who followed eight women, volunteers from an intact class, as they progressed through a preprofessional mathematics review course designed to prepare them for placement in nontraditional jobs in the building trades. Crittenden reports the following:
The women had difficulty learning mathematics in a classroom setting.
Although work-related mathematics tasks are often more complex than classroom-based mathematics problems, the women in the study had an easier time learning and using mathematics on the job.
The women in the study did not perceive any difficulties using mathematics in common budgeting and shopping chores because of the repetitive nature of the tasks.
The women in the study were unable to use relevant mathematics skills for nonroutine personal finance tasks, such as the evaluation of investment options for retirement planning, because they had either incomplete or unhelpful schemas for the financial services industry.
Ethnomathematics and feminist approaches demonstrate that school mathematics and street mathematics differ substantially. Further, the affective environment of the traditional classroom setting can impede student learning despite the best-intentioned efforts of the instructor. Both points should be taken seriously when interventions are designed for the adult numeracy student.
Numeracy and Cognition: Experience and Situations
Despite the importance of understanding cognitionwhat and how people know what they knowfor instruction, such studies in adult numeracy or mathematics education are rare; most studies of cognition and numeracy/mathematics in the education and psychology fields have been developed through research with children. However, there is clear research evidence that mathematical knowledge develops both in and out of school, for adults and children, and is profoundly influenced by experience and cultural practice, as socio-cognitive theorists have shown ADDIN EN.CITE Lave19882752756Lave, JeanCognition in Practice: Mind, mathematics and culture in everyday life214adults, cognition, culture, everyday mathematics, feelings, mathematics education, power, practice, problem-solving, social relationships, transferability of learning, values1988CambridgeCambridge University Press0-521-35734-9Saxe19913693696Saxe, GeoffreyCulture and Cognitive Development: Studies in mathematical understanding1991Hillsdale NJLawrence Erlbaum Associates0805802738Schliemann198937537517Schliemann, A. D.Acioly, N. M.Mathematical knowledge developed at work: The contribution of practice versus the contribution of schoolingCognition and InstructionCognition and Instruction185-22161989(Lave, 1988; Saxe, 1991; Schliemann & Acioly, 1989). Such studies emphasize the ability of people to control and regulate their own behavior in relation to their experience in their environment, rather than react automatically to stimuli, as behaviorist psychologists predict.
Adults bring this prior knowledge and life experience to the classroom and apply it to their use of mathematics in a wider range of situations. Effective instruction must be responsive to these experiences, but there is little theory designed toward understanding prior experience and how these experiences affect learning. Duffin and Simpson (1993, 1995) have developed a theory of learning that attempts to classify learners experiences into three categories: natural, conflicting, and alien (Exhibit 3).
Exhibit 3.Adult Experiences and Learning Responses
ExperiencesResponses to ExperienceNaturalFits the learners mental structures
Is expected and unsurprisingStrengthens the current way of thinking
Extends the scope of the internal mental structureConflictingIs inconsistent with the learners internal mental structures
Jars with expectations
Highlights limitations or contradictionsDestroys an internal mental structure
or
Limits a way of thinking
or
Merges two mental structuresAlienHas no connection with the learners internal mental structures
Is meaningless for the learner
Cannot be coped withIgnores the experience
or
Avoids the experience
or
Absorbs the experience as a new mental structureFrom ADDIN EN.CITE Simpson19957857855Simpson, AdrianDuffin, JanetCoben, DianaMACS A model for university-wide mathematics supportMathematics with a Human Face: Proceedings of ALM-2, the Second International Conference of Adults Learning Maths - A Research Forum (ALM-2) held at University of Exeter, 7-9 July 199560-641995LondonGoldsmiths College, University of London, in association with ALMSimpson & Duffin (1995)
Duffin and Simpson suggest that the natural-based learner aims to build on existing internal mental structures, developing connections between new and previous experiences. Such learners are likely to develop highly flexible, widely applicable learning because of their highly interconnected mental structures, but they are likely to build up their understanding of complex methods slowly. Meanwhile, for an alien-based learner, new experiences are accepted separately and either are left unconnected or are connected only later in response to any conflicts that may arise. Such learners should be able to master techniques quickly, without the need to build on existing structures, but such skills would be bound by the context in which they were learned and would be difficult to reconstruct if they went unused and became forgotten.
Gal (2000) takes a different approach, beginning from the learners perspective. He notes that real-life numeracy situations are always embedded in the life stream with personal meaning to the individual involved. Adults need numerate skills to enable them to manage diverse types of quantitative situations, and he identifies three types of numeracy situations that adults must manage. Generative situations require people to count, quantify, compute, or otherwise manipulate numbers and generate a response. Examples are dealing with simple operations, such as calculating a total price of products while shopping or measuring a shelf; dealing with multistep operations embedded in text, such as completing a tax form; and making reasonable decisions, such as choosing the best home mortgage loan. Resulting responses have clear right or wrong answers.
Interpretive situations require people to make sense of verbal or text-based messages based on quantitative data but do not require them to manipulate numbers. Examples include interpreting a chart in a newspaper article reporting crime statistics or reading a report of a survey with poll results. The response expected in such situations is an opinion or the activation of critical questions that have no clear right or wrong answers.
Decisions situations require people to find and consider multiple pieces of information to determine a course of action. Such situations include identifying ways to use limited resources, such as money or time, and choosing among alternatives (renting the right apartment, purchasing the right car, or deciding on the best insurance, for example). Not only is there no clear correct answer in such situations, but the person may sacrifice precision or accuracy to save time or mental load when deciding on a response and may reach the response in an inefficient or nonstandard way (Gal et al., 2005).
The three numeracy situations are not necessarily distinct categories, and Gal (2000) notes that other types of numeracy situations and hybrid situations are possible. Numerate behavior is also enabled by dispositional elements (prior belief, attitudes, and habits) that motivate and support effective behavior in any situation.
Affective Factors, Anxiety, and Learning Styles
Dispositional elements are central to constructivist theories, which not only posit the importance of experience and situations in learning but also include characteristics of the learner as integral to learning. Research on the role of learners affect, attitudes, and beliefs in learning mathematics, although widespread for children, has been limited on adults, focusing on attitudes and anxiety about mathematics. Singh (1993) surveyed adults attitudes toward mathematics and found that abstraction and perceived lack of relevance are common reasons students cited for their dislike for mathematics and that fear of failure induced by instruction and testing in mathematics is a main cause of anxiety.
Sheila Tobias has conducted extensive research on math anxiety in adults. She authored Overcoming Math Anxiety, in which she explores the economic impact, both personal and societal, of poor mathematics skills. Using her work with math-phobic individuals, particularly women, Tobias shares her insights into the causes of math anxiety and the myths of math ability (or not), and she suggests ways to help students overcome their anxiety and open the door to disciplines and occupations that require strong mathematical skills. She recommends the following:
Students need to take charge of their mathematics learning, refusing to be intimidated by their history and the culture of previous mathematics classrooms.
Teachers must create environments where math anxiety can be discussed openly, help students recognize their mathematics strengths, and provide opportunities for success and difficulty within their zone of proximal development.
Mathematics books are not an easy read, and part of the desensitization process compels teachers to help students develop appropriate reading skills.
Teaching styles must be adjusted to include methods that recognize differences invoked by gender and culture.
Talking and writing about feelings and strategies must permeate the course. (Tobias, 1993)
Several other studies have investigated the existence and effect of anxiety on learning in adult students. Altieri (1987) surveyed developmental studies students (n = 89) at a community college using Kolbs Learning Styles Inventory followed by interviews with 17 of the students and four faculty members. Although the study was not specific to mathematics and anxiety was not the focal question, analysis revealed that anxiety and remembering were the central learning problems of the students. The researcher proposes that the first may be a root cause of the second. Both the students and the faculty named anxiety as the dominant impediment to learning, an impediment that fostered problems with remembering, course pacing, and testing.
Using the Brief Math Anxiety Rating Scale (BMARS) and the Learning Style Inventory-Adapted (LSI-A), Cook (1997) found a connection between anxiety level and perceptual learning style in more than 500 community college students. Cook found that students with audio and tactile/kinesthetic learning styles were more likely to have math anxiety. Female students had a higher math anxiety level, but age was not significantly correlated.
Jost (1997) studied the way that anxiety about mathematics and computer-assisted instruction affected 40 students in an adult education class. He used the Computer Attitude Scale and Mathematics Anxiety Rating Scale along with a demographic questionnaire and conducted analyses to determine the interaction of computer anxiety and demographic variables. He found significant gender differences for computer experience, more negative attitudes, and higher computer anxiety. There was no gender difference for achievement as measured by the final exam in the course.
There are other, correlational studies of math anxiety on adults in developmental mathematics courses. Peskoff (2001) evaluated the relationship between students level of math anxiety and the strategies they employ to cope with it, using 279 developmental mathematics students. A multivariate statistical analysis related the effects of math anxiety, gender, and course enrollment on 10 coping strategies rated for frequency of use and helpfulness. Peskoff found that students with low math anxiety use and value a wider variety of coping strategies than peers with high math anxiety. High-anxiety students used tutoring services and met with their counselors significantly more than low-anxiety students, and males used the avoidance strategy of exercising or engaging in physical activity significantly more than females. However, students and the faculty considered this strategy one of the least useful strategies. Two strategies considered more helpful by all students and the faculty, completing homework assignments on time and letting the instructor know they do not understand the material, were used significantly more often by females. Students identified two additional strategies that they felt were more helpful: asking questions in class and allowing extra study time before exams.
Parker (1997) interviewed 12 developmental mathematics students who previously had math anxiety to describe the steps they took to overcome their anxiety. Analysis of the data identified six stages in the transition from math anxious to confident. First, each spoke of a personal realization that he or she had to become comfortable with mathematics because some personal goal depended on it. Often that goal was related to job advancement, although that was not the only reason cited. Next, students made a personal commitment to achieving the goal. For some of the subjects, past success mastering difficult tasks spurred belief that they could also be successful at mathematics. They spoke of determination and positive thinking as keys to victory. They then mapped out a strategy to master the subject and took action. Many described the recognition of a significant turning point in both their aptitude and their attitude, which led to positive feelings about themselves and about the value of mathematics, eventually becoming a part of the math support system to help other struggling with mathematics.
Duffin and Simpson (2000) have also explored the tensions between the cognitive and the affective aspects of learning, as has Evans ADDIN EN.CITE Evans20001641646Evans, JeffErnest, PaulAdults Mathematical Thinking and Emotions: A study of numerate practicesStudies in Mathematics Education3202000LondonRoutledge/Falmer, Taylor & Francis Group0-750-70912-X (pbk)
00750709138 (cased)(2000), who goes beyond the focus in some earlier studies on math anxiety to focus on the interrelationship between adults mathematical thinking and both positive and negative emotions. He argues that thinking and emotion are inseparable, so mathematical activity is always emotional and teachers should encourage adult students to seek to understand what might be emotional blocks to actively seeking out possible applications of their learning.
Learning Styles
Along with affect, recent research has examined the role of individual learning styles in adult education. Two influential examples are David Kolbs (1984) learning style inventory that he termed experiential learning and Howard Gardners (1993) theory of multiple intelligences (MI). Kolb describes learning as an ongoing, circular process based in concrete experiences that the learner reflects upon, abstracts concepts from, and then actively experiments with to enrich the learning base. Learners are classified according to the point in the process where they seem most comfortable. Divergers ponder concrete experiences, imagine possibilities, and ask What if? questions, whereas convergers start with concepts and seek out a solution. Assimilators focus on concepts, reflecting on their abstract qualities with little concern for practical applications. Finally, accommodators take concrete experiences and experiment with them to create new experiences and build knowledge.
Gardners theory combines psychology with neuroscience and identifies eight intelligences: musical, bodily-kinesthetic, logical-mathematical, linguistic, spatial, interpersonal, intrapersonal, and naturalist. These, he contends, are located in different regions of the brain. Someone who is left brained is thought to be logical and structured, whereas someone who is right brained is more creative and spontaneous. The idea is that by teaching to these intelligences, teachers can make their lessons more effective.
Learning styles approaches are popular among some adult numeracy educators, and a group of adult literacy teachers in New England has investigated the utility of MI theory to their work. Two of the teachers focused on mathematics instruction, reporting benefits for both teacher and student from the project ADDIN EN.CITE Costanzo2001172217225Costanzo, Meg RybackSchmitt, Mary JaneSafford-Ramus, KatherineAdult Multiple Intelligences and mathA Conversation between Researchers and Practitioners. Adults Learning Mathematics - 7. Proceedings of ALM-7 the Seventh International Conference of Adults Learning Mathematics - A Research Forum104-1082001Cambridge, MANational Center for the Study of Adult Learning and Literacy (NCSALL), Harvard University Graduate School of Education, in association with Adults Learning Mathematics - A Research Forum (ALM)(Costanzo, 2001).
However, recent research by Frank Coffield and colleagues warns against stereotyping people on the basis of their learning styles ADDIN EN.CITE Coffield20044730473027Coffield, FrankMoseley, D.Hall, E.Ecclestone, K.Should we be using learning styles? What research has to say to practice2004LondonLearning and Skills Research Centre559(Coffield, Moseley, Hall, & Ecclestone, 2004). Coffields team surveyed more than 70 instruments designed to identify peoples learning styles. They then undertook a rigorous scrutiny of 13 of these instruments, together with a literature review of the main theories on learning styles. They concluded that the idea that various types of intelligence are located in diverse parts of the brain is not confirmed by neuroscience. Although certain parts of the brain do seem to control particular activities, the brain is far more flexible and more robust than some theorists had assumed. They also found that some of the most widely used instruments have low reliability, poor validity, and a negligible impact on teaching and learning. Similarly, John White describes learning styles approaches as deterministic and potentially leading people to restrict their own possibilities ADDIN EN.CITE White1998478947896White, JohnDo Howard Gardner's multiple intelligences add up?1998LondonLondon University Institute of Education(White, 1998).
Brain Research: A New Science of Learning
The study of cognitive factors and individual differences that affect learning has recently taken a radically different turn, toward brain research. The question of how our understanding of adult numeracy and mathematics teaching and learning might relate to this new research in cognitive neuroscience has been explored since 1999 in the Numeracy Network, which is part of OECDs Brain and Learning project. This project has several aims:
Develop a new science of learning through creative dialogue between cognitive neuroscience, psychology, education, health and policy.
Discover what insights cognitive neuroscience might offer to education and educational policy and vice versa.
Identify questions and issues in the understanding of human learning where education needs help from other disciplines. ADDIN EN.CITE OECD/CERI20044503450312OECD/CERI2004ParisOECD(Organization for Economic Co-operation and Development [OECD] 2004, p. 9)
In the present, second phase (200206), the project is focusing on three main issues: literacy, numeracy, and lifelong learning; the findings and policy recommendations will be published in 2006.
The Numeracy Network focuses primarily on brain mechanisms related to fundamental educational skills that enable comprehension of mathematical thought. This includes basic work on numeracy skills and symbolic thinking, with emphasis on the cognitive psychology and neuropsychology of mathematics operations.
An example of work discussed in the project that may have far-reaching implications for understanding mathematics learning at all ages is Dehaenes triple code theory. This theory describes a modular system of brain areas that are active when a child is learning or performing arithmetical operations: addition, subtraction, multiplication, and division. The basic idea is that when manipulating a number, a child does one of three actions, each involving a different region of the brain:
Performs some visual manipulation (seeing the number as a visual digit, such as 3),
Performs some linguistic manipulation (hearing or reading the number as a word, such as three), and
Represents it as a quantity (such as 3 is bigger than 1) (OECD 2004, p. 64).
On the basis of this theory, Dehaene ADDIN EN.CITE Dehaene19924799479917Dehaene, S.Varieties of numerical abilitiesCognitionCognitionCognition1-42441992(1992) contends that:
Adult human numerical cognition can therefore be viewed as a layered modular architecture, the preverbal representation of approximate numerical magnitudes supporting the progressive emergence of language-dependent abilities such as verbal counting, number transcoding, and symbolic calculation. (p. 35)
If Dehaene is right, his theory may explain why some adults have difficulty in one or more of these areas: recognizing, manipulating, or representing numbers. It also tells us that we should not assume that an adult who can do one of these actions will be able to do the others and shows that language is deeply implicated in some, but not all, arithmetical operations. This is just one example; the work of the OECD Numeracy Network shows that there is much debate among cognitive neuroscientists, psychologists, and educationalists. For example, Karmiloff-Smith (2004) argues that modules might pre-exist in the neonate, but that this cannot be assumed from studies of children or adults because domain specificity can emerge over developmental time. Also, it is a long way from producing a theoretical model to proving its explanatory power and then working out the implications for teaching and learning. It is important to recognize the present limitations, as well as the power and potential of brain research with respect to education, and to avoid a determinism that sees brain research as the final arbiter in matters of education. Nevertheless, the development of a new learning science is promising, especially if it takes the form of a creative multidisciplinary collaboration.
Summary
This brief review of the competing conceptualizations of adult numeracy and learning has identified a rich and active body of theoretical work. There is substantial debate within the field on how to define and characterize adult numeracy, and we used Maguire and ODonohues conceptual framework to organize the competing definitions. Integrative definitions of numeracy have the most influence on recent instructional frameworks and standards developed for adult numeracy. These frameworks also reflect constructivist views of learning that posit that learners actively construct knowledge by integrating new information and experiences into what they have previously come to understand. There is a substantial body of research inspired by constructivist theories, and we reviewed relevant research on the role of learners prior experience, numeracy situations, math anxiety, and learning styles on adult mathematics learning, along with new, promising brain research.
3. Adult Numeracy and Mathematics Instructional Approaches and Interventions
As just illustrated, there is a rich and lively debate on definitions of adult numeracy and on how adults learn mathematics. Integrative definitions of numeracy and constructivist theories have been particularly influential, and current approaches to numeracy instruction for adults reflect this thinking. In this section we briefly review the predominant instructional frameworks about teaching mathematics to adults. We then present a review of the empirical research evaluating instructional approaches toward teaching mathematics to students in adult education and literacy classes.
Professional Society Instructional Standards
Several professional societies concerned with mathematics instruction have developed standards that influence recent practice in ABE instruction. These standards define instructional content, including the specific facts or subjects to be covered; skills needed, such as problem solving and critical thinking; and process or pedagogy. There tends to be agreement among the frameworks and standards on the need for specific skills such as problem solving, but there is less agreement on specific content and teaching methods (Safford-Ramus, 2006, in press). In particular, there is agreement on the need for critical-thinking and problem-solving skills within mathematics instruction.
National Council of Teachers of Mathematics Standards
Although developed for teaching primary and secondary school mathematics, the framework developed by the National Council of Teachers of Mathematics (NCTM) has been among the most influential in adult mathematics teaching. Three documents present the NCTM approach: Curriculum and Evaluation Standards for School Mathematics (1989), Professional Standards for Teaching Mathematics (1991), and Assessment Standards for School Mathematics (1995). The intent of the Standards series was the identification of the best way to teach mathematics the first time. A revised edition of the book, Principles and Standards for School Mathematics, was published in 2000.
The NCTM principles provide a valuable model for instructors, balancing content and methodology in light of the needs of the workforce in the twenty-first century and the technology available to workers. Six principles are stipulated for school mathematics:
Equity. Excellence in mathematics education requires equityhigh expectations and strong support for all students.
Curriculum. A curriculum is more than a collection of activities: it must be coherent, focused on important mathematics, and well articulated across the grades.
Teaching. Effective mathematics teaching requires understanding what students know and need to learn and then challenging and supporting them to learn it well.
Learning. Students must learn mathematics with understanding, actively building new knowledge from experience and prior knowledge.
Assessment. Assessment should support the learning of important mathematics and furnish useful information to both teachers and students.
Technology. Technology is essential in teaching and learning mathematics; it influences the mathematics that is taught and enhances students learning (National Council of Teachers of Mathematics, 2000).
The six principles undergird the ten standards that are split between content and process. The content standards, which focus on what students should learn, include number and operation, algebra, geometry, measurement, and data analysis and probability. The process standards define the ways of acquiring and using that content knowledge and include problem solving, reasoning and proof, communications, connections, and representation. Appendix B presents the NCTM standards.
Crossroads
In September 1995, the American Mathematical Association of Two-Year Colleges (AMATYC) released Crossroads in Mathematics: Standards for Introductory College Mathematics before Calculus. This document defines the mathematics that students need to be able to pursue collegiate mathematics courses. Such core mathematics may have to be retaught to students who did not meet or master it during their youth. Crossroads also suggests methods for implementing instructional improvements. Because virtually all students in U.S. postsecondary studies are adults, Crossroads can be looked at as an andragogical document. Also, two-year colleges in the United States are a primary vehicle for adults to reenter further education, so a substantial percentage of that population consists of adults under the most restrictive definition of that term. A revision of Crossroads, titled Crossroads Revisited, is scheduled for release as this literature review is being written.
AMATYC based its standards development in Crossroads on six principles, including that mathematics should be meaningful and relevant and that the use of technology in instruction is essential. Crossroads offers three categories of standards: intellectual development, content, and pedagogy. The recommendations for intellectual development emphasize modeling and problem solving, reasoning and communicating mathematically, and the judicious use of technology to accomplish those tasks. The standards for content include number and operation sense, pattern, symbolism and algebra, geometry, functions, discrete mathematics, probability and statistics, and deductive proof. The third focus, pedagogy, examines the place of technology, interactive and collaborative learning, connection to other disciplines and real-world applications, and multiple approaches that embrace all learning styles. Appendix B includes a summary of the recommendations from the 1995 AMATYC document. The revision includes and expands on the earlier lists with substantial attention to math anxiety, learning and teaching styles, assessment, and professionalism.
Other Frameworks
In ABE, state- and practitioner-led projects have developed standards. The Adult Numeracy Networks (ANN) mathematics standards framework (Curry et al., 1996) and the National Institute for Literacys (2000) Equipped for the Future (EFF) Math Content Standard have been most directly influential. We have already discussed these in Section 2. Eight states have developed, or are in the process of developing, content standards for mathematics for their ABE programs. The conceptual bases for these efforts vary widely, with some states following an NCTM or EFF model, some employing competency-based approaches, and others defining only basic mathematics content knowledge.
The Mathematical Association of America (MAA) has for many years supported inquiry into the content and teaching methods used in undergraduate mathematics courses. Its Committee on the Undergraduate Program in Mathematics (CUPM) offered recommendations on the mathematics curriculum for all undergraduates in a report published in 2004, CUPM Curriculum Guide 2004. The report recommends standards for content selection as well as the conduct of university mathematics courses from the introductory level through the mathematics major. These standards have had influence in adult education, though to a lesser extent that the NCTM and Crossroads standards.
Research Evaluating Instructional Approaches
Although there has been significant work toward developing instructional frameworks and content standards, there has not been a similarly intense focus on evaluating the effectiveness of these frameworks or the theories underlying them for adult mathematics instruction. Unfortunately, very few research studies have used ABE students to study the effects of adult numeracy instruction, and the research that does exist is neither theory-driven nor guided by any systematic approach. There has been no agenda or systematic model guiding research in adult numeracy (Coben, 2000a). Two previous reviews of this research (Tout & Schmitt, 2002; Coben, 2003) identified fewer than 20 studies on instructional impact on adults; only 9 of the studies identified in these reviews were conducted in the United States. Reviews focusing on identifying methodologically rigorous studies of instructional effects on ABE and ESL students concur. For example, Torgerson, Brooks, and colleagues (2003) identified only one well-conducted randomized control study that used mathematics as an outcome measure, a finding verified by Condelli and Wrigley (2004), who also found that only 9 studies using ABE and ESL students demonstrated a statistically significant impact of instruction.
A central goal of the Adult Numeracy Initiative is to identify instructional practices worthy of replication (Research Question 3). Although the body of field-based, research-practitioner studies on numeracy instruction within ABE classes has been growing, this research uses primarily introspective and qualitative methods. Although the knowledge of experienced practitioners is an important resource in any effort to improve the effectiveness of adult numeracy teaching and professional development, this knowledge must be verified through more-rigorous methodologies. Consequently, our approach is to identify whether rigorous research has found instructional approaches or interventions that have been effective in enhancing the mathematics and numeracy skills of adult learners.
Methodology for Identifying Research
Our review examined all studies conducted between 1985 and 2005 that tested an instructional approach or intervention, used an outcome measure that assessed skills in mathematics or numeracy, and employed some type of comparison group. This review includes only intervention studies with adult students that addressed mathematics skill levels that would be considered basic elementary level through the secondary level. These skills are the basic mathematics skills typically taught to students enrolled in ABE programs and adult students in developmental mathematics programs (typically taught in community college settings).
While we recognize the differences between ABE programs other instructional settings, ABE classes are often taught within community colleges. In addition, developmental mathematics students are in many ways similar to ABE students, and the content of instruction to these students is also similar to what is taught in ABE. We believe that including these studies increases the potential knowledge base and thus can inform ABE instruction and address the goals of the Adult Numeracy Initiative. However, we excluded studies of developmental mathematics interventions that examined instruction of higher level (postsecondary) mathematics skills to adults, such as college algebra, because these skills are beyond what is taught in ABE.
To identify studies, we searched research databases (Proquest, ERIC, EBSCO, MATHS4Life, Dissertation Abstracts International, NALD, and Reference Manager) and generic Web sites (Google.com, Yahoo.com, MSN.com, Askjeeves.com, Webcrawler.com, Altavista.com, Excite.com, and AOL.com), using different permutations of the keywords adult numeracy basic education and adult mathematics basic education. We also visited numerous national and international mathematics- and numeracy-related Web sites to try to identify additional adult numeracy and mathematicsrelated sources. Appendix C lists all Web sites and publications included in our search. Reference Manager connected us to catalogs from more than 500 universities, and several features from the software enabled us to categorize all articles by title, author, link to PDF, URL, abstract, link to full text, and keywords, among several other categorization parameters.
This search identified 223 studies, of which only 91 related specifically to adult numeracy. We next deleted studies that
were not empirical research on instructional interventions,
did not include adults in ABE classes,
were conducted prior to 1985,
did not have outcomes related to learning mathematics,
did not have a comparison group, and
included fewer than five students per group.
At the end of this process, our search criteria left only 15 studies for the review, all of which included ABE students.
To identify additional studies, we included doctoral dissertations, identified through the mathematics dissertation database developed by Safford (2000) and updated to 2002. Safford developed this database by searching Dissertation Abstracts International (DAI) using a Boolean search for Adult AND Mathematics AND Education in the descriptor field. After we applied our selection criteria, this search resulted in an additional 9 studies, which gave us a total of 24 studies for our review. Appendix D includes a matrix that summarizes all the intervention research studies we have included in this review.
Quality of Research
The Institute for Education Sciences (IES) has established evidence standards for research, embodied in the procedures of the What Works Clearinghouse (WWC; www.whatworks.ed.gov). Applying the evidence standards allows reviewers to rate studies as (1) meeting the standards, (2) meeting the standards with reservations, or (3) failing to meet standards. Studies that meet standards have the strongest evidence of effects. These studies are primarily well-conducted randomized controlled trials and regression discontinuity studies, or quasi-experimental studies of especially strong design.
Although we were not required to follow the WWC procedures or use the evidence standards in this review, we applied them to the adult numeracy studies we identified to evaluate the overall quality. Unfortunately, the methodologies were generally of such low quality that very few could meet the evidence standards. Fewer than 5 studies used random assignment; among these, many had severe differential attrition, which made the resulting groups nonequivalent. Other studies used nonequivalent comparison groups, used very small sample sizes (fewer than 10), used nonstandardized or qualitative outcome measures, did not adequately explain the intervention, or did not adequately explain statistical analyses.
Rather than eliminate all studies that did not meet evidence standards (which would have left us with very little to report), we included all intervention studies that used some type of comparison group and had at least five students per group. In the discussion of all studies, we describe the methodology to allow an assessment of overall study quality.
Effective Instruction for Adult Learners: Research Findings
Of greatest interest to adult educators is whether the research can identify the characteristics of effective instruction. The instructional frameworks, constructivist theories, and integrative conceptions of numeracy are compelling and beg the question of whether they are effective on adult learners. Unfortunately, our review revealed that very little research examining these approaches met our criteria for rigorous methodology.
The 15 studies of the effects of instruction on mathematics learning for ABE students investigated a remarkably narrow range of topics: 13 compared the effects of technology or computer-assisted instruction (CAI). Of the 2 remaining studies, 1 examined inference training (a contextual learning method for literacy development) and 1 examined participation in family literacy. Among the 9 studies on interventions with developmental mathematics students, 5 studies evaluated cooperative learning methods, which are based on constructivist views of learning, and 4 studies evaluated the use of technology and learning.
Use of Technology in Instruction
Most of the research (13 ABE studies and 4 development mathematics studies) on the effect of instruction of adult mathematics in ABE has examined the effects of the use of instructional technology. The content standards of the three major mathematics education organizations endorse using technology for mathematics instruction. In its Principles and Standards for School Mathematics, the NCTM (2000) states:
Electronic technologiescalculators and computersare essential tools for teaching, learning, and doing mathematics. They furnish visual images of mathematical idea, they facilitate organizing and analyzing data, and they compute efficiently and accurately. They can support investigations by students in every area of mathematics, including geometry, statistics, algebra, measurement, and number. (pp. 2425)
In Beyond Crossroads, the AMATYC authors (2005) enumerate the technologies used in the community college environment: graphing calculators, mathematics software, spreadsheets, multimedia, computers, and the Internet. They also introduce distance learning, a concept with powerful potential for adult educators. The MAA (2004), although endorsing the use of technology, warns that such use must be judicious:
The use of technology can help students develop mathematical skills and understanding. However, the use of technology must be focused on students needs rather than on the capabilities of the technology itself. Instructors must first decide what mathematics is to be learned and how students are to learn it. The answers to these questions will determine whether and how students should use technology. (p. 22)
The research on the use of technology in instruction has not demonstrated that it improves the learning of mathematics by adults over instruction that does not use technology. Among the 13 studies of ABE students and the 4 developmental mathematics studies, only B. S. Burton (1995) and Wilson (1987) found statistically significant increases in achievement for CAI. Lavery, Townsend, & Wilton (1998) reported an impact of CAI, but used a sample of only six students per group. Nurss (1989) and the Indiana Opportunities Industrialization Centers (O.I.C.) of America State Council (1990) also showed a positive effect for CAI, but these studies have severe methodological problems that cloud interpretation. In addition, the instructional approaches represented within the technology and the type of technology studied vary so widely that it is difficult to draw conclusions about instruction from this research.
Although there seems to be no clear advantage or disadvantage to the incorporation of CAI into ABE, GED, or developmental mathematics courses from this research, technology, particularly the Internet, has evolved rapidly and is quite different from what it was when most of the studies were conducted. The studies span 20 years, years that have seen a great change in the quality and sophistication of educational software. The degree of change in technology has been so profound that is questionable whether the early studies have much applicability to the more technologically knowledgeable adults of the 21st century. The adult student cohort in 2005 is more knowledgeable about and dependent on technology than those of 10 years ago. We summarize each study below, beginning with the ABE studies, which include correctional education and GED courses, and concluding with studies of students in developmental mathematics courses.
Computer Use in ABE Instruction
Among the earliest studies in ABE, Barnett (1985) and Reid (1986) reported studies that used Program Logic for Automated Training Operations (PLATO) computer-assisted instruction. The Barnett study had a pretest-posttest design comparing students in two juvenile correctional facilities. There was no indication of the number of participants or the instruments used to measure the variables. Achievement and attitude of students in the PLATO group were not significantly different from those of students who received only traditional instruction.
Reid (1986) compared three teaching methods: CAI using PLATO, tutoring using Laubach materials, and traditional teaching. Subjects (n = 30) were members of existing ABE/GED classes. The TABE M and D levels served as pre- and posttest instruments. There was no significant difference in mathematical achievement among the three groups, although the CAI group gained 1.9 grade levels while the traditional group gained 1.1. Robichaud (1985) also compared students in traditional settings with those whose regular instruction was supplemented by CAI. No details concerning instrument or evaluation were supplied, but she reported that statistical analysis revealed no significant difference in skills gained. However, there was a significantly positive change in attitudes toward computers and the instructional use of CAI by the CAI users. In a later study, B. S. Burton (1995) revisited the same question of traditional versus CAI instruction with more definitive results. He compared CAI and traditional instruction with 200 adults at a vocational technical adult education center, using the TABE M and D as its assessment measure with a nonequivalent control group design. Students using CAI were found to do significantly better in mathematics than those in the control group. Age and gender had no effect, but student ethnicity and extent of prior formal education did affect results.
Nurss (1989) assessed the effectiveness of the Principle of Alphabet Literacy Systems (PALS) CAI program on the literacy skills of adult nonreaders, compared with traditional adult basic education. This trial showed a significant, positive effect for the traditional adult basic education classes (i.e., the control group). Attrition, however, was extremely high in both groups. Of the 74 students assigned to the control group, 15 percent (n = 11) remained at the posttest; 32 percent of the 135 students in the experimental group (n = 43) completed the program. One could conjecture that the cream of the control remained and therefore performed well on the test. In addition there was differential attrition, with more of the control group staying to completion, which clouded the results. However, it might be argued that the greater retention of the experimental group indicated higher student satisfaction with the instruction.
The Indiana O. I. C. of America State Council (1990) reported the results of a study that examined the effectiveness of traditional classroom instruction versus computer-assisted instruction in raising the competency levels of adults one grade level for every 80 hours of instruction. Of the 149 individuals who were pretested, only 50 attended more than 30 hours and remained at the time of posttesting. Evaluation was done using a randomized methodology with the ABLE test as the pre- and posttest instrument. No information was supplied concerning the statistical analysis used to evaluate the results. However, the report indicates that the overall average grade change for CAI students was 2.6 grades compared with an average of 1.84 grades for non-CAI students. The high attrition again makes these findings difficult to interpret.
The objective of a study by Nicol and Anderson (2000) was to evaluate an experiment that compared CAI and teacher-implemented instruction in numeracy. It is unclear whether the same two teachers taught the two intervention groups. The researchers randomized the adult students into three groups of eight students. The method of random allocation was not described, but stratification by gender was implied. The researchers reported no difference in improvement between the teacher-led intervention and the CAI, but given the very small numbers in each group, there is a high possibility of a Type II error in this study.
Lavery, Townsend and Wilton (1998) conducted a randomized control trial of 12 students in New Zealand to compare the learning outcomes associated with basic literacy education programs conducted through traditional instruction with computer-assisted instruction. The students and instructional approaches in these programs are similar to those in U.S. adult education courses. The study measured the gains in reading and numeracy skills in two training opportunities classes. Six students received traditional teaching, and another six used Readers Workshop, Math Concepts and Skills, and Computer Curriculum Corporations Computer Assisted Learning (CAL) software packages. Participants reading and numeracy skills were measured by the Burt Word Reading Test, the Neale Analysis of Reading Ability, and the KeyMath Revised Test. The results show that significantly greater achievements were made in reading (word recognition, word accuracy, and comprehension) and numeracy (mathematical concepts, operations, and applications) under CAI than under traditional instruction. The students who used the CAL made a 3-year gain on the Burt, over 1 year on the Neale, and 16 months on the mathematics test in less than 2 months of instruction. During the same time, the students who received traditional teaching made no gains in reading skills and showed a slight decline in mathematics performance.
Computer Use in Correctional Education
In addition to Barnett (1985), three studies focused specifically on incarcerated populations. Winters, Teslow and Taylor (1993) studied the effect of a CAI-supplemented program on ABE/pre-GED and GED students in an adult correctional facility. Five students were assigned to either the experimental or the control group (n = 10), and the researchers pre- and posttested with the TABE. The statistical methods used to analyze the data were unclear, but the results favored the CAI intervention: 86 percent of the students in the pilot study advanced in level in mathematics in contrast to a 50 percent gain in the control group. A comparison of students advancing one year or more showed 43 percent for the pilot study versus 14.5 percent for the control group. Once again, the small sample size limits the utility of this study.
Batchelder and Rachel (2000) studied the effect of skill-and-drill tutorial software to enhance mathematics and language skills. They randomly assigned 71 male inmates in the prisons GED program to receive either the regular classroom instruction offered or classroom instruction supplemented by CAI using the tutorial software. The classroom instruction consisted of four hours per day in English, mathematics, history, and science. Students in the experimental group received three hours of instruction per day but spent the fourth hour using the CAI software for mathematics and reading. Inmates were posttested with the CASAS reading and mathematics tests after receiving 80 hours (four weeks) of instruction. There was no significant difference between the groups on these tests.
Burnham (1985) examined the effect of a televised curriculum on an incarcerated ASE population. Subjects in a nonequivalent control group research design (n = 40) were pretested using the General Educational Performance Index (Form AA) and posttested using Form BB of the same test. The experimental group used an instructional televisions series, Adult Math, as a reinforcement resource, viewing the telelessons under supervision and then completing workbook exercises tied to the program. The control group completed self-paced workbooks and used other instructional materials but did not view the television series. The researcher found no difference in achievement between the groups, although he cautions that Adult Math is more effective when the subjects have grade-level scores of at least 5.8 in arithmetic and reading and that the literacy levels of incarcerated populations are noticeably lower than those of the general population.
Computer Use in GED Instruction
Two other studies evaluating computer-based instruction (CBI) were GED specific. Wilder (1994) compared the effects of a CBI simulation-test treatment, a CBI drill-and-practice program, and a traditional workbook drill-and-practice class on retention, completion time, and elevation of test scores on the mathematics section of the GED. The research design was a three-group, posttest-only design with unequal sample sizes, where a total of 564 students self-selected into the classes. Wilder followed the students for five years, with 308 students retained long enough to get a GED diploma. In addition, 94 percent of the simulation group was retained compared with 65 percent in the CBI drill group and 36 percent in the workbook-only group. Completion time was also considerably less in both CBI groups. Scores on the test were not significantly different.
Wardlaw (1997) studied the effect of CAI on achievement and attitude for a group of pre-GED and GED adults. The study was conducted in established classes with 60 students each in the treatment and control groups. Pre- and posttesting was done using the TABE and the Semantic Differential Attitudinal Questionnaire. Wardlaw found no significant difference on either achievement or attitude. He does offer an important caveat for developers planning to incorporate CAI into a program. Wardlaw surveyed ABE facilities and found that although some were well equipped, many others had few or outdated workstations. One facility had banned student use of the equipment because the director believed that the students were using it to arrange dates rather than study. Wardlaw suggests that these environmental issues may have contributed to the failure of CAI to effect positive attitudinal change.
Computer Use in Developmental Mathematics Instruction
Four studies examined the use of CAI with developmental mathematics students in tertiary institutions. In an early study, Wilson (1987) described a diagnostic and tutorial program that was conducted at a vocational school in Kentucky. The researchers designed a diagnostic test for pre- and posttesting, which they normed against the TABE at the 8.75 grade equivalent. The results of the experiment showed a significant effect in favor of the experimental group. Toet (1991) studied a randomly selected sample of students who had been placed into remedial reading, English, or mathematics at a community college. Using the TABE, she compared achievement between students who completed assignments based on textbook use and students who worked in a CAI laboratory. The group taking basic mathematics showed a statistically significant cognitive gain. Those studying beginning algebra were retained longer at a statistically significant level of .05. There was no significant retention difference for the basic mathematics group.
Hsieh (1992) examined the effect of two specific features of CAI, animation and manipulation, on 54 students participating in two computer-based laboratory (CBL) sections of a developmental mathematics course. The students were randomly assigned to receive instruction with or without animation and with or without manipulation. The outcome measures were overall achievement, retention of content, and motivation, measured through a questionnaire. The researcher listed five findings:
Animation enhanced retention when the tasks required high-level cognitive processes such as analysis or synthesis.
Animation did not help learning or retention when the tasks were comprehension of mathematical concepts.
Animation increased continuing motivation.
Manipulation helped the transference of mathematical concepts learned through a computer to paper-and-pencil tests.
Manipulation did not promote intrinsic motivation.
In the most recent and most thoroughly defined study, Costner (2002) examined the effectiveness of a computer algebra system (CAS) on achievement and attitudes of students in a college remedial algebra course. Students in the treatment group (n = 26) used the CAS to discover algorithms, explore algebraic manipulation, and identify misconceptions, while students in the control group (n = 25) did not have access to the CAS. Several instruments were used in the study: a researcher-designed pretest and periodic section tests; a departmental final exam; the Fennema-Sherman Attitude Toward Success in Mathematics Scale, the Confidence in Learning Mathematics Scale, and the Mathematics Usefulness Scale; a researcher-designed questionnaire and semistructured interview (n = 5); and periodic writing assignments. There was no statistically significant effect on achievement or surveyed attitudes. However, the qualitative data gathered through the questionnaire revealed significant differences in attitudes and in classroom culture issues. Students in the treatment group cited the helpfulness of group work and classroom discussions more often than students in the control group. With respect to the use of CAS, the treatment group welcomed the ability to check their work and get immediate feedback. They felt that the CAS helped them see mathematics differently, yet they attributed little of their new mathematical understanding to technology. One criticism was the unavailability of the computer in testing situations. The researcher suggests that assessment needs to be altered if CAS is an integral part of the course.
Research Evaluating Instruction Based on Constructivist Theories
As discussed in Section 2, constructivist theories of instruction and learningthe hypothesis that all knowledge is constructed by individuals acting upon external stimulihas had a great influence on recent work in adult mathematics and numeracy instruction. Although no studies have directly examined the effect of constructivist approaches of mathematics learning on ABE students, researchers have studied two constructivist models, cooperative learning and discovery learning, on students enrolled in developmental mathematics courses.
We identified three studies on cooperative learning and two studies on discovery learning. Only one of the studies on cooperative learning found a positive effect, but none indicated that cooperative learning has a detrimental effect on achievement for adult students. Findings also suggested that cooperative learning may contribute positively to student attitude while decreasing math anxiety. The studies on discovery learning were more positive, with all showing positive effects on either affective measures or measures of mathematics achievement or understanding.
Cooperative Learning
Cooperative learning embraces a number of classroom organization styles, all of which group students in learning teams for some or all of the instructional time. Several models, according to Neil Davidson, share the following characteristics (Davidson in Slavin, 1985):
The class is divided into small groups composed of two to six members.
Each group has its own working space, which may include a section of the blackboard.
The group is involved in discussing mathematical concepts and principles, practicing mathematical techniques, and solving problems.
The teacher moves from group to group, checks the students work, and provides assistance in varying degrees.
The groups sometimes gather outside of class to work on projects.
Within in each group, certain leadership and management functions must be performed.
Students in Costners study (2002) cited above found the use of group work and classroom discussion helpful. Two other studies have reported findings from research investigating the use of cooperative learning with adult populations.
Peer tutoring is cooperative learning between two individuals where each learns from and with the other. Berry (1996) studied the effect of peer tutoring in dyads on adult students in a remedial algebra class. Two studies were conducted: a 6-week program and a 12-week semester. Students self-selected the classes but had no knowledge of the planned intervention. Instructors were randomly assigned and trained in the intervention after assignment. In each case, three peer-tutoring sections were contrasted with three traditional lecture sections. Pre-and posttests were given using the Suinn Mathematics Anxiety Rating Scale, the Fennema-Sherman Mathematics Attitude Scales, a profile questionnaire, and an abbreviated version of the institutional Freshman Skills Assessment Program test. An open-ended survey was also used. Sections had an average of 35 students (n = approximately 210). Of the variables measured, only attitude increased significantly during the 6-week study. For students in the 12-week semester, the intervention group showed significant improvement in mathematics achievement and attitude as well as reduced anxiety.
In a study by Ellis (1992), each of seven instructors at a community college taught one developmental algebra section that incorporated the use of in-class study groups and one section that did not use groups. She compared the achievement and completion rates and found no significant difference between the experimental and control groups for the group neither as a whole nor on the basis of age or gender.
Discovery Learning
One method of using cooperative learning in the adult mathematics classroom is termed small-group discovery (Safford, 1998; Davidson, 1985). The NCTM standards include the use of discovery learning for mathematics. Davidson (1985) described the method in the following way:
The instructor introduced new material with brief lectures at the beginning of class, during which he posed problems and questions for investigation. For most of the class time, the students worked together cooperatively at the blackboard in four-member groups. The students discussed mathematical concepts, proved theorems, made conjectures, constructed examples and counter-examples, and developed techniques for problem solving. The instructor provided guidance and support for the small groups.
Although discovery learning has not been studied in ABE research, three studies using developmental mathematics students have examined approaches that allowed students to use discovery learning to construct their own knowledge.
Bartlett (1993) used a guided discovery approach to teaching mathematics in one section of a developmental mathematics course at a university. She defined the methodology in the following way: Under the guidance of the teacher, students find and use their own rules and generalizations to solve other problems. The experimental group (n = 27) was a class taught with this approach and was compared with the same class taught in a previous quarter (n = 52) without the approach. Outcome measures were mathematics performance measured by a researcher-developed test and mathematics anxiety measured by the Math Anxiety Rating Scale (MARS). Students in the experimental class performed better on the outcomes, and Bartlett reported that the experimental method was effective in improving the mathematics performance of adult students.
Ramus (1997) reported similar findings from a course with 13 developmental mathematics students that used discovery learning methods. She supplemented quantitative measures (course tests) with qualitative interviews of 8 students. Students reported a sense of ownership of the rules of algebra because they had discovered them from classroom exercises and also self-reported a positive change in attitude toward mathematics and increased confidence that transferred to other activities outside the classroom. Quantitative measures, derived from the course examination, were less conclusive. Examination results were scored using two rubrics, one to measure correctness and one to measure the use of problem-solving strategies. An ANOVA showed that the experimental section performed as well as the evening section but less well than the daytime class. The author suggested that the different demographic composition of the evening and experimental classes may have affected the outcome as much as the intervention.
Pace (1989) explored the applicability of constructivist methods to the teaching of geometry concepts in a remedial mathematics class at an urban community college. Students (n = 67) were pretested using the Applied Geometry Test, the Van Hiele Geometry Test, and the New Jersey College Basic Skill Placement Test. They were randomly assigned to four sections of the course, two experimental and two control, all taught by the same instructor. The treatment class instruction consisted of five 80-minute sessions during which students explored concepts of area and perimeter using activities embedded in applied problem-solving settings. Students were posttested and delayed posttested. The data were assessed using single and multivariate linear regression models. Those in the treatment program performed significantly better than their counterparts.
ABE Instructional Interventions
Our review identified only two additional studies of instructional interventions on the ABE student population that included a measure of mathematics as an outcome measure. In neither of these studies was numeracy or mathematics instruction the main focus of the intervention. One study examined inference training, a meaning-making strategy used for reading development. This study may be relevant to adult mathematics instruction to the extent that numeracy may be considered an integrative skill that includes literacy abilities. The second study examined the general impact of family literacy instruction compared with general ABE instruction and therefore provides an indication of the overall effect of the programmatic model, although not mathematics instruction specifically.
Farr (1987) investigated the effects of inference training in learning vocabulary on verbal abilities and mathematics problem solving among 40 ABE students. Half the students had inference training, a predictive reading strategy where they were taught vocabulary skills and reasoning training, and the other students received traditional ABE instruction without the training. Although the main focus of the study was literacy development, mathematics problem solving was included as a dependent variable to ascertain whether training in inferencing in language acquisition would be reflected in other areas. The results showed a correlation between mathematics performance and reading performance. The results also showed that verbal ability correlated with the ability to solve analogies and neologisms.
Irby et al. (1992) conducted a randomized control trial of approximately 25 predominantly black and Hispanic students in an ABE setting; 15 students were in a family literacy project (intervention group), and 10 were enrolled in GED classes only (control). The objective of the study was to evaluate the effectiveness of a family literacy project on the numeracy and literacy levels of adults. The intervention was conducted in a family literacy project comprising several components, and ABE classes were offered twice a week for 12 weeks. Instructors developed individualized educational plans for each student to work at his or her own pace. The results indicated that students in the family literacy project showed a higher average gain in reading and mathematics compared with the GED class.
Summary
In a recent review of the research on the effects of instruction of ABE and ESL students, Condelli and Wrigley (2004) concluded that the research:
reflect[s] a haphazard and unorganized approach toward studying adult literacy and [is] not guided by any theory, approach or school of thought about good pedagogy. They do not provide a comprehensive body of knowledge on the impacts or literacy interventions in ABE. (p. 22)
The same can be said of the research studying the effects of instruction in mathematics and numeracy on ABE students. With only 15 studies examining mathematics interventions, almost all of them dealing with the use of technology in instruction, we cannot consider this research a meaningful guide toward directing future efforts in practice or research. The additional 9 research studies on developmental mathematics students suggest promising directions, particularly in studying constructivist approaches toward teaching and learning. However, this research is also limited and indicates a very early state of inquiry.
Besides the need for theory- and standards-driven research, our review has identified a lack of research on instruction for adults that addresses individual learning differences. There are descriptive and theoretical studies on how adults learn, including cognitive influences, learning disabilities, gender differences and motivation, but we found no studies of how learning and instruction interact with these differences to influence the development of numeracy. In addition, we found no research of any type examining instruction to adult ESL learners. There exists no research base at all on how numeracy is taught in ESL classes, let alone studies that examine instructional approaches and their impact on these learners.
4. Assessment Issues in Adult Numeracy
As we have just demonstrated, the research on the impact of instruction on adult mathematics learning has not kept pace with the strides in theory development or conceptualizations in numeracy. The same situation exists in regard to the assessment of adults mathematics and numeracy skills. There exist very few assessments of mathematics skills for ABE students, and those that do exist do not address critical-thinking and problem-solving skills or other aspects of numeracy. Assessment in adult literacy programs is driven more by the GED tests and the need to meet reporting needs of the National Reporting System (NRS), the adult education programs accountability system.
In this section, we not only review the scant research base and practice on adult numeracy assessment but also reflect on key issues related to the development of effective assessments in adult numeracy education. We bring to the surface issues and dilemmas that can inform the goals of the Adult Numeracy Initiative and a future research and development agenda in this area. We begin with an overview of the purposes of assessments in adult numeracy and the limited related research and take a closer look at the most widely used instruments. We then analyze the nature of an assessment and what can be improved about it and summarize principles and ideas to inform the conceptualization of good assessments in numeracy education.
The Purposes of Assessments in Adult Numeracy Education and Related Research
A discussion of the assessment of numeracy in adult education is a complex and at times daunting matter. The very heterogeneous array of adult education programs encompasses varying sizes and formats; the programs operate in different contexts with different degrees of independence; and many of their operations are undocumented. Numeracy education is often subsumed as part of basic education, literacy education, workplace education, prison education, and other titles, unlike mathematics education, which normally occupies a prominent and separate space in K12 school contexts. Hence, numeracy is often not discussed as a stand-alone topic.
Purposes of Assessment
Assessments related to the learning of numeracy (and literacy) are undertaken for many purposes associated with needs that learners, teachers, and program administrators face in various stages of activity (Sticht, 1990; Ananda, 2000), as follows:
Entrance stage. Assessments are conducted as part of an initial diagnosis of incoming students skills, capacities, and work habits. Assessments inform decisions about placement, help set learning goals, and influence the choice of curricula and teaching/learning methods. In the context of numeracy learning, entrance-stage assessments may also have to detect whether learners have informal mathematical knowledge, math anxiety, or other characteristics that may contribute to or affect further learning. Additional assessments may be needed for new students who show evidence of having special learning needs; the results of these assessments may have an impact on decisions about special accommodations or adaptations so that these learners can demonstrate their full range of capacities (Sacks & Cebula, 2000).
Teaching / learning stage. After a period of learning, assessments may be used for formative evaluation or feedback, that is, to monitor learners progress, examine areas of strength and weakness, and help with designing a further course of study or deciding on needed interventions. When learning is undertaken with the goal of eventually passing a formal, external test (e.g., GED, workplace certification), selected exams or practice tests may be used intermittently to determine whether learners are ready to engage in the real test. At the entrance stage and at this stage, assessments may involve a mix of teacher-developed or locally designed methods, both written and oral, as well as formal instruments from external sources (e.g., commercial standardized assessments, GED practice tests). The assessments should enable both the teacher and the learners to understand the interpretation of the results of the assessment and the logic underlying decisions.
End/exit stage. When learners end their prescribed program of studies, or when programs need to measure learners progress because of accountability reporting demands by sponsors (e.g., the NRS) or as part of accreditation schemes, assessments may be used to document overall learning gains and thus serve a summative function or help with program evaluation. At times, the end stage may be a stepping stone into a new cycle of learning numeracy or mathematics at a higher level; hence, summative assessments may also serve a diagnostic function.
Other assessments. Although partially overlapping with the uses of assessment above, sometimes assessments are conducted as part of research projects by academic researchers or by state and federal agencies. They may be part of program evaluation initiatives that emerge from the needs of stakeholders other than the learners, teachers, or administrators linked with a specific program. These external demands may affect the time allotted to assessments or the level of motivation of those being assessed.
Related Research
Our review of the literature shows that it is difficult to locate publications relating to adult numeracy assessment because this topic is usually intertwined with assessments pertaining to literacy and other skill areas. Hence, the knowledge base we uncovered is quite limited and patchy. Below are examples illustrating the dearth of attention to assessment related directly to numeracy learning.
Of a total of 88 presentations, workshops, and discussion groups in the proceedings of the last three conferences of ALM (Adult Learning Mathematics, an international forum of researchers and practitioners) held in 2002, 2003, and 2004, only 4 discussed assessment; only 2 of these presented empirical data (both on the same single study).
In the 2002, 2003, and 2004 issues of Adventures in Assessment, a widely read online magazine published once a year by SABES/World Education in Massachusetts for adult education practitioners nationwide, only 1 of a total of 20 articles examined mathematics assessment.
In the Research Companion to Principles and Standards for School Mathematics, published by NCTM in 2003 (Kilpatrick, Martin, & Schifter, 2003), out of 23 core chapters, only 1 directly examined assessment issues.
Few authors in the sources above directly addressed numeracy assessment, yet quite a few touched on more general issues with relevance to numeracy assessment, such as problem-solving processes, task authenticity, the role of context demands on performance, the impact of dispositions such as beliefs or attitudes, the nature of mathematical practices in workplace versus school contexts, and many more. This finding should not be too surprising because assessment serves many useful functions in any learning and teaching context and affects what learners, teachers, and programs do, yet it is a support activity in the service of larger goals.
Indeed, in the literature search we conducted for this review, numerous articles presented opinions and models for assessment or reported on studies that used assessments of numeracy skills (e.g., to evaluate learning gains). Few sources, however, focused on the characteristics of adult numeracy assessments or on their improvement as an object of scholarly inquiry and empirical research. Most of those that did were not related to adult education per se, but to the development of policy-driven research projects and large-scale assessments (Kirsch, Jungeblut, Jenkins, & Kolstad, 1993; Gillespie, 2004; Gal et al., 2005; Ginsburg, Cooke, Leinwand, Noell, & Pollock, 2005; Brooks, Heath, & Pollard, 2005). On a broad systemic level, assessments of adults have been examined in connection with the development of the NRS and the search for performance measures that can satisfy accountability requirements or in the context of planning for the National Assessment of Adult Literacy (Stites, 2000). These efforts looked across a range of skills, with literacy (i.e., language-related skills) at the forefront and numeracy in the background, sometime in a restricted sense, such the notion of Quantitative literacy as defined in the NALS (Kirsch et al., 1993) and subsequently in the NAAL.
Over the last 10 to 15 years, several monographs have examined aspects of assessments in adult education programs. Some looked at assessment issues in general (Rose & Leahy, 1998) or reviewed a broad range of assessment tools (Sticht, 1990), whereas others focused on methods that can be used for assessment in specific contexts, such as teaching workplace literacy (Taylor, 1997; Sticht, 1999) or English language learners (Van Duzer & Berdan, 1999). Some projects focused less on instruments and more on ways the assessments can be incorporated to support teachers ongoing work (Ananda, 2000) or on considerations affecting the decisions of administrators or teachers to adopt one assessment scheme over another (Nagel, 1999).
Only two reports were found on issues in numeracy assessment in the context of adult education programs, yet they are of partial use in the present context. Cumming, Gal, and Ginsburg (1998), as part of the Numeracy Project at the National Center on Adult Literacy (NCAL) and with the goal of informing practitioners and program personnel more than policymakers, examined principles for good assessment and limitations of methods in field use, especially of the TABE test. Although informative, this review referred to TABE versions 5-6, which have since been replaced by versions 7-8 and, starting in 2003, by TABE 9-10 whose content has changed somewhat.
As part of a project of the National Literacy Secretariat in Canada, Hagedorn and colleagues (2003) examined several numeracy frameworks and provided a detailed description of the features of the math component of three tests in widest use (GED, TABE, and CASAS). They also examined the EFF mathematics standards, already introduced in Section 2, and the numeracy aspects of the NRS. Both the EFF and the NRS descriptors are not tests; rather, they outline the content areas or skills that adults are expected to develop during instruction and thus have to be eventually assessed. Although useful as a summary of existing frameworks with implications for assessment, the Hagedorn report was meant to be descriptive and does not provide a critical analysis of tests or frameworks.
Overview of Widely Used Instruments for ABE Numeracy Assessment
With the implementation of the NRS, all states must now use for reporting purposes a standardized test to measure educational gain in ABE and ESL students. NRS Implementation Guidelines require that tests must be reliable and valid; allowable tests include the Tests of Adult Basic Education (TABE), the Comprehensive Adult Student Assessment System (CASAS), the Basic English Skills test (BEST), and the Adult Basic Learning Examination (ABLE )(OVAE, 2005). Although these requirements do not preclude the use of other types of assessments (e.g., nonstandardized) for other needs of teachers and programs as long as they are reliable and valid, most programs have adopted the mathematics sections of the TABE and CASAS (Gal & Schuh, 1994) to measure the mathematics abilities of ABE learners.
We now take a look at these two assessment instruments and critically evaluate them in light of the numeracy definitions and instructional frameworks presented earlier. In addition, we also note the GED test, which for many adult education programs serves as an important target or milestone as they prepare ABE learners.
TABE
The TABE is a family of widely used standardized tests published by CTB/McGraw-Hill. The most recent revision of the tests, version 9-10, was put into use in 2003 (TABE 2003a, 2003b, 2003c). In addition to various subtests related to literacy, two of the TABE subtests assess mathematics computation and applied mathematics, each with 40 to 50 multiple-choice questions presented with four response options. The test is norm-referenced (i.e., scores are computed in reference to a norming group of adults), and results are usually reported as grade-level equivalents divided into five levels, from preliteracy (level L equal to grades 01.9) to Advanced (level A equal to grades 8.612.9). The TABE system includes a Locator test and a short Survey form that according to the publisher can be used to help place students at different levels. At each level, two equivalent forms exist, presently 9 and 10, which according to the publisher can be used to measure growth and educational gains as pre- and posttest measures. It is important to note that the TABE is a skill-based test; that is, most items measure specific skills (e.g., whole number operations, conversion of fractions), often with relatively little contextual information. The publisher claims that the revised forms 9-10 offer better coverage of key areas of mathematics. The training materials for teachers, also from the publisher, indicate that the higher skill levels use not only items in arithmetic but also some that touch on measurement, geometry, statistics, and algebra. That said, no systematic analysis has been published of the actual content by which we could evaluate these claims, and presently it appears that teachers and programs are expected to use the test on the assumption that it is better but without having access to much detailed information regarding its alignment with instructional goals.
CASAS
The CASAS is widely used for assessing adult basic reading, mathematics, listening, writing, and speaking skills. The CASAS (2004) manual claims to measure some 300 separate competencies by using items couched in realistic functional contexts. CASAS was designed from the outset to offer an integrated system for assessment, training, and evaluation. It offers more than 100 separate tests, which measure groups of competencies at different levels of ability by using items that examine proficiency in performing specific tasks involving solving life-skill problems or applying general reading and mathematics skills. Like the TABE, the CASAS uses only multiple-choice items, although the emphasis is on dealing with such real-world situations as workplace, community, or family tasks, all of which may involve the activation of multiple skills. The CASAS manual does not explain exactly how the tests are scored, but it appears that test scores are based on the difficulty estimates for each item, which were derived through IRT (Rasch) scaling. Scores are usually reported on a numerical scale ranging between 150 and 250, commonly divided into five levels from A (beginning literacy) to E (advanced adult secondary).
Both the TABE and the CASAS claim that using the tests enables programs to establish measurable goals, conduct diagnostic assessments, document learning outcomes, and report program gains to students, staff, and external stakeholders. Practitioners who want to use the tests in educational contexts are offered various suggestions for accommodations in test administration for students with special learning needs. The test publishers claim that these accommodations do not change what the tests intend to measure. Further, numerous tables of correlations are available from the publishers, which link specific test items or scale scores to various chapters or units in a diverse range of instructional materials, thus enabling teachers to assign learning resources to students who fail specific test items. In particular, the CASAS (2004) technical manual provides numerous tables showing linkages or mapping between CASAS levels and levels or score ranges for other systems or measures, such as NRS, EFF, GED, and NALS.
The developers of both assessment systems claim that their instruments undergo rigorous test development and validation procedures and meet common psychometric standards. The CASAS technical manual lists various studies that have provided information about the tests validity, such as content validation through various procedures involving different stakeholders, positive association between CASAS scores and learning gains, or positive correlations in linking studies between CASAS scores and GED mathematics scores. The TABE Web site presents more modest information (undated report) on the same topics, such as positive gains in programs that used CTB/McGraw-Hill instructional materials and measured performance by TABE forms and positive correlations between TABE scores and grade levels in school.
It is important to emphasize that the recent GED 2002 versions have been developed on the basis of the NAEP Mathematics Framework, and item types and their proportions in the GED are designed to reflect NAEP content guidelines. Both the TABE and the CASAS are not expressly designed to reflect NAEP, yet both show as part of their validation data that scores on the TABE or the CASAS are highly predictive of scores on the GED. Given that the CASAS is competency-based and the TABE is skill-based, further analysis is warranted to understand what cognitive and other processes are at work when learners work on all these different instruments yet achieve highly correlated results.
A Critical Look
The TABE and the CASAS are widely employed, yet they have differences in logic, construction, and use. Nevertheless, they both appear to be reliable and valid as far as standard techniques and published sources allow us to evaluate. Various concerns and questions can be raised that touch on validity, construct coverage, permissible interpretations of the meaning of test scores, and alignment between assessments and instruction. These concerns are presented below regarding the TABE and the CASAS, but they are just as relevant when evaluating the quality of other assessments:
Both tests claim to have good content validity, in part ascertained by interviewing diverse educators and learners, yet little information is published regarding their underlying construct. Both, and especially the TABE, purport to cover key mathematical subdomains such as number, measurement, algebra, and statistics (in the higher skill levels). These appear to be linked to well-known frameworks of mathematical knowledge, such as the one from NCTM. However, little published information exists (from test publishers or other sources) about the extent to which the tests cover broader facets of numeracy and acknowledge the nature of the everyday mathematical tasks and problems that adults have to solve. We are not claiming here that they do not do soonly that there is no published materials on the underlying construct and its coverage. For example, test manuals do not report the percentage of items representing the key subdomains or having high versus low literacy demands. The Massachusetts Department of Education (2005) compared the content of TABE, AMES, CASAS and ABLE to thestate curriculum frameworks. The analysis found that the TABE is best aligned to the states ELA and Math standards, but the TABE items aligned with only 50% of the framework and the test has limited ability to provide information deemed essential for teachers and students.
How is it possible to reconcile the fact that one test (TABE) is skill based and one is competency based (CASAS)? On the one hand, a test that emphasizes assessing skills in a decontextualized context may not provide much information on learners capacities and competencies in a functional context. This is a serious issue from the point of view of skill transfer because there is no valid evidence that skill transfer can happen. Indeed, research on everyday mathematics, noted above, suggests that the transfer of school or decontextualized learning is difficult. On the other hand, the demonstration of competencies in a functional context may be domain-specific (Strasser, 2003) and learners may not have more generalizable skills. A related research question asks to what extent are the practitioners who are using the tests aware of the tests different logic and the implications for the interpretations about skills that can be drawn from each test?
There is a long-standing awareness of the limitations of forced-choice items, used by both the TABE and the CASAS, to reflect reasoning, problem-solving, or communication skills (Sticht, 1990). A further limitation of the TABE results from its extensive reliance on problems that involve little or no text, ignoring the inherent links between numeracy and literacy skills in everyday functional contexts. The CASAS involves more stimuli with text in them, but test-takers do not have to produce any text. Thus, the tests do not address the expectation that learners of mathematics are also able to communicate mathematically (NCTM, 2000; U.S. Department of Labor, SCANS, 1991).
Both tests expect accurate results that can be clearly classified as right or wrong. However, many real-life situations call for approximate answers, estimates, or opinions or judgments rather than for accurate results. Further, both tests score performance in terms of correct/incorrect responses. They ignore the possibility of a partially correct response. Yet, recent large-scale assessments of mathematical skills employ scoring systems that give credit to partial responses (such as TIMSS and PISA; Ginsburg et al., 2005) or accommodate various levels of accuracy in responses (such as ALL; Gal et al., 2005). The upshot is that because of their forced-choice format, both the TABE and the CASAS lose information about the skills of the adults being testedit is technically feasible to accommodate more levels of performance in ways that better reflect the nature of adults numeracy.
There are claims that a hierarchy of skill levels cannot be ascertained in all areas of mathematics (Kilpatrick, 2001; Gal et al., 2005). It is unclear how the developers of the two tests have coped with this issue in their suggestions for interpretation of test results. This is especially true of the TABE, where raw scores are converted into grade-level equivalents, given known limitations of grade-level equivalents (Spruck Wrigley, 1998; Sireci & Zenisky, 2003).
Overall, the TABE and the CASAS assessments reflect definitions of adult numeracy within the formative and mathematical phases in the Maguire and ODonohue framework discussed in Section 2. Neither assessment is adequate for conceptions of numeracy in the integrative phase, such as those embodied in many of the standards and frameworks influencing the field. The NRS descriptors for numeracy also reflect the formative and mathematics phases of numeracy, with most skills related to arithmetic or number. Additional numeracy-related skills are listed under functional and workplace skills and mainly call for the ability to read graphs and charts, deal with forms, and solve multi-step problems. This last statement implies an integration of skills, giving the NRS descriptors a split personality when it comes to numeracy issues.
Tests that assess skills such as problem solving and critical thinking are difficult to develop. They are also hard to implement in the ABE system because they are time-consuming to administer and difficult to score. However, assessments reflecting the integrative approach to numeracy are used in the Netherlands as part of its national Realistic Mathematics Education (RME) curriculum for adult students and in Australia for its Certificates in General Education for Adults (CGEA) framework (Tout & Schmitt, 2002). The implication is that it is possible to create assessments that align with overall instructional models. Assessment should focus educators on broader instructional goals and be designed in a way that will not encourage teachers to teach to the test. Many programs already use the GED test as a de facto definition of what students need to know. This situation may become more ingrained with the increased use of both the TABE and the CASAS, which are sanctioned for use by many states and programs to meet NRS demands.
Defining the Meaning of Assessment and How It Can Be Improved
The absence of recent studies and critical analyses related to assessment in adult numeracy programs justifies the current review of assessment issues in the Adult Numeracy Initiative. As a starting point, we briefly reflect in this subsection on the nature of assessments and how they can be improved before discussing in the next subsection the criteria for good assessments in the area of adult numeracy education.
Definitions of assessments may vary, yet in one way or another, they all discuss a systematic process for collecting and evaluating information about people (Anastasi, 1997). In the context of learning mathematical topics, Wilson and Kenney (2003), using the NCTM Assessment Standards (1995), explain that the information collected pertains to a students knowledge of, ability to use, and disposition towards, mathematics. As Wiggins (1992) has aptly warned, however, all too often there is a tendency to equate assessment with testing and to assume that testing is a simple, complication-free process that yields clear-cut results. Instead, it should be acknowledged at the outset that assessment is far from being a simple or unitary construct and that the complex nature of an assessment has implications for how it can be improved.
An analysis of various claims and ideas raised in the testing and assessment literature over several decades (e.g., Cronbach & Gleser, 1965; Messick, 2000), coupled with an analysis of the multiplicity of purposes of assessments in adult numeracy education and of the diverse ways in which they can be used by learners, teachers, and programs, shows that an assessment as used in adult education has at least four key components.
First, all assessments use an information collection component, a process or sequences of steps and activities for gathering data in diverse ways (Anastasi, 1997). The collection may involve a tool, a person, or both (e.g., a written GED test, a teacher using a scoring rubric to grade a learners written solution to a word problem or the contents of a cumulative portfolio of a learners class work over a period of time, an observation form for rating an employees performance on a work sample used for workplace licensing, a computer-based questionnaire). The tools themselves may have different degrees of formality or complexity. Artifacts or technical aids, such as a hand-held calculator, a ruler, or computer software, may be involved and used by the learner.
Second, an assessment usually activates a judgment/evaluation component in which information from different sources (e.g., subtest scores, performance on functional tasks, teacher observations) may first be integrated or combined and then evaluated. In this component, meaning is given to collected data, and interpretations are reached on the basis of expectations, performance standards, norms, and the like, whether implicit (e.g., a teachers informal evaluation) or explicit (e.g., number of correct responses compared with national norms, IRT scaling of responses of groups of test-takers). When the data collection involves a simple test (e.g., a single TABE booklet), the evaluation of a persons performance may appear simple and free of human judgment, as in the calculation of a total raw score on a test and its conversion to a grade level equivalent. However, when we examine the overall range of occasions when information about a specific learner is collected and evaluated, we clearly see that many judgments and interpretations are involved. As the psychological testing literature has repeatedly emphasized (e.g., Goldberg, 1986), peoples judgments and interpretations in such situations may be affected by multiple factors and may have different degrees of accuracy or validity. Consider the challenge of evaluating whether a learner knows or does not know a certain mathematical construct or procedure when the learner is not a native speaker of English or is anxious. In these and similar circumstances, the teacher/assessor has to decide what (if any) allowance to make for language or learning style differences, and the judgment process may be affected by a host of seemingly external but at times very relevant factors.
Third, the assessment will often involve a decision component that may be interwoven with the first two components (Cronbach & Gleser, 1965). For example, a student may be routed to different test forms on the TABE on the basis of the TABE Locator test or may receive different items as part of a computerized adaptive test system that is based on preliminary item parameters chosen by researchers. Another example occurs when a teacher has to decide whether to introduce an accommodation (e.g., add testing time, rephrase a question) because of personal characteristics of a specific learner, and if so, how far to go in this regard.
Fourth, assessments in educational contexts always involve a human component because people may be the object of the assessment, conduct the assessment, or examine its results and reach some conclusions. Teachers or administrators involved in the assessment pathway have beliefs, attitudes, emotions, and both personal and organizational motivations. The performance of learners undergoing assessment may be influenced by dispositional factors discussed in earlier sections (e.g., math anxiety or test anxiety, beliefs about testing) or by motivational factors (the perceived importance of the learning of mathematical topics, or of the assessment event itself, in the context of the learners life course and goals). The learner-related factors and teacher-related factors may operate separately or interact with each other, and they may have an impact on the behavior, attention level, or degree of effort invested by those being assessed or doing the assessment.
Finally, value judgments affect all the other assessment components in explicit or implicit ways. Test planners, administrators, teachers, and learners may operate in different social or organizational ecologies and may have different perspectives on issues such as what is important to measure or what is a fair assessment. Adult education staff who administer the assessments, often lack sufficient training on the purpose, administration and scoring of tests and impose their own judgment and values on assessment. This lack of training and understanding may lead to improper administration, invalid scores and incorrect interpretation of outcome measures. Experts interested in psychological and educational testing have time and again argued and demonstrated that values affect choice of data collection methods, interpretations, judgments, and behaviors of stakeholders involved in an assessment (Messick, 2000). The recognition of the centrality of values and the influence of the human component on all aspects of the assessment pathway (e.g., test planning, selection of a data collection method, interpretations of performance, decisions) has led to a strong emphasis on the need for thorough training for those involved in assessing human capabilities (Alfonso & Pratt, 1997). This recognition has also led various professional associations to establish professional standards in this area.
It follows that assessment is a complex, dynamic system. It can combine technical elements and supporting artifacts. It involves human interaction as well as internal and external actions and choices of individuals. It unfolds over time and is conducted in a social context that presents diverse demands on those being assessed. Both the inner workings (setup, artifacts used or not, evaluation of performance, time limits imposed) and the outcomes of an assessment (scores, written evaluation, pass/fail decisions) are influenced by criteria or values imposed by the assessment designer, by the beliefs and training of the humans conducting the assessment, by the motivation and reactions of those being assessed, and by organizational stakeholders.
These considerations lead us to two important realizations:
Judgments regarding the goodness or quality of an assessment can refer to any of its componentsnot only to the collection methods (e.g., content and format of test items or tasks used) but also to the stages involving integration and interpretation, or decisions. Further, given that assessments always involve humans and are embedded in a certain context, the quality of an assessment may also be related to such associated factors as the demand characteristics of the testing context, time limits, artifacts used, training of the people involved, and the values or criteria applied when interpreting performance.
Improvements in an assessment can relate to any of its components or associated factors described above, either alone or in combination. Further, assessments in adult numeracy education usually occur in an organizational context (e.g., ABE program, community college, workplace); hence, broader institutional or systemic issues can also be examined and improved, such as how assessment processes are scheduled or budgeted, the criteria that program directors use for choosing one assessment method over another, or the professional qualifications and preparation of those administering assessments or interpreting their results.
Considerations for Developing Improved Assessments for Adult Numeracy
If we have concerns about current assessments, we should first consider what criteria to use to make value judgments about them and then reflect on whether these criteria are the right ones. Discussions of what counts as a good assessment often begin with a description of a set of general principles or criteria. The literature on psychological and educational assessment invariably states that assessments should be examined in terms of their reliability and validity and specifies multiple types or forms of each of these two key psychometric properties (Anastasi, 1997). The testing literature has also referred, using somewhat different terminology, to various other important aspects of assessments, such as degree of standardization of an instrument (seen as reflecting its degree of subjectivity or objectivity); cost of testing; ease of scoring and usage; fairness or discrimination; and utility, a concept related to the contribution of a test to an organization making acceptance/rejection or placement decisions (Cronbach & Gleser, 1965). These attributes of assessment are typically required of all high-stakes educational assessments. Indeed, the NRS Guidelines require all adult education programs to use standardized, reliable, and valid assessments.
However, in discussing the assessment of numeracy, we must note additional principles or ideas, especially by those involved in teaching the relevant subject matter. The U.S. mathematics education community has articulated several principles for assessment as part of the reform movement, which started two decades ago. The Mathematical Sciences Education Board (1994) enumerated three principles of good assessment in mathematics: the Content Principle (assessment should reflect the mathematics that is most important for students to learn), the Learning Principle (assessment should enhance mathematics learning and support good instructional practices), and the Equity Principle (assessment should support every students opportunity to learn important mathematics). NCTMs (1995) Assessment Standards raised additional points, later reiterated in its Principles and Standards for School Mathematics (2000), regarding openness, valid inferences, and coherence of an assessment with the curriculum and instruction.
The mathematics-oriented assessment principles echo several key ideas found in the general testing literature, especially regarding validity, fairness, and utility. Yet, these principles go above and beyond technical demands for validity and reliability and bring new elements into the discourse about good assessments of numeracy skills. They also encompass the nature of mathematical knowledge, the intended goals of the learning process, and the varied uses of an assessment or its influence regarding the learner, the teacher, and the program. Messick (1989) has argued that validity should be viewed broadly as the extent to which empirical findings and theoretical rationales support the appropriateness of inferences and actions based on test scores or performance on the assessment instrument. However, the actions taken by a teacher and a learner after they know an assessments results are seldom based on the assessment itself. Rather, such actions are influenced by the broader ecology or context within which the assessment takes place and within which the teacher and the learner operate. In particular, an assessment becomes meaningful when there is a linkage among the assessment, the curriculum, and the learners own goals (which are especially important for adult learners; Donovan, 2002), as well as when both teachers and learners believe that an assessment measures valued skills or capacities. It thus becomes paramount to examine what content areas related to mathematics and numeracy are important for adults to know.
What Should Be Assessed in Adult Numeracy Instruction?
The answer to this question will affect our judgment of the quality of different assessments in numeracy education. However, the answer is complex because it depends on the breadth of the analysis being attempted and on the extent to which we go beyond traditional ideas that equate mathematical knowledge with mastery of basic arithmetic concepts and accurate execution of computational procedures.
Section 2 has already reviewed several perspectives on the concept of numeracy, clustered into three groups or phases: formative, mathematical, and integrative. Below we summarize these and also consider additional key influences that help us sketch the nature of the skills, knowledge, and dispositions that are part of the constructs that could be the focus of teaching/learning, and hence also of assessments, in the field of adult numeracy education:
Curriculum frameworks and statements regarding the goals of mathematics learning for adults and young adults (e.g., Australian Committee for Training Curriculum, 1994; Curry et al., 1996; NCTM, 2000; Stein, 2000) present a broad set of areas for learning: knowledge and understanding of numbers and operations that involve computation, as well as number sense and estimation; algebra and modeling; geometry and shape; measurement; and data and chance. In addition, knowing mathematics involves problem solving, reasoning, and communicative capacities as well as an understanding of connections and representations of various mathematical ideas.
Conceptions of workplace skills (e.g., U.S. Department of Labor, SCANS, 1991; Mayer, 1992; Packer, 1997; Forman & Steen, 1999) further show that effective functioning on the job involves not only a diverse set of mathematical and arithmetical skills but also broader knowledge and skills related, for example, to the ability to allocate resources, handle scheduling, understand the role of quantitative information in the operation of systems, and use technological tools to quantify or display quantitative information.
Teachers and the professional literature have mostly addressed the acquisition, teaching, and learning of language skills and mathematical skills as two separate areas of inquiry and practice with little crossover (Gal, 2000). However, many real-world functional tasks require adults to integrate numeracy and literacy skills. Examples are interpretive tasks that do not require the manipulation of numbers but do require the expression of opinions (e.g., statistical literacy tasks, decision tasks involving notions of chance) and other tasks that involve numbers or quantitative statements embedded in text (e.g., forms, schedules, manuals, technical and financial documents, statistics in the media). Kirsch and Mosenthal (1993) integrate numeracy and literacy in their literacy model, which comprises prose, document, and quantitative components. The integration of these skill areas underlies the design of various large-scale surveys of numeracy and mathematical skills, such as the NALS, the International Adult Literacy Survey (IALS), NAAL, and the recent ALL survey (Gal et al., 2005), and the design of some quantitative literacy tasks in the Trends in International Mathematics and Science Study (TIMSS).
The research on ethnomathematics, briefly discussed in Section 2, has focused on the nature of everyday mathematics. One of the conclusions in this literature is that differential performance can occur when assessments are divorced from, as opposed to contextualized in, realistic settings (Lave, Murtagh, & de la Rocha, 1984). Problem solving in contextualized real-life and work activities may be complex and different from solving school-like problems (Resnick, 1987; Greeno, 2003). Many researchers (e.g., Strasser, 2003; Wedege, 2003) discuss the hidden aspects of workplace mathematics and point to the fact that workers often have trouble identifying the math in what they do or encounter. These and other observations suggest that good assessments need to enable learners, teachers, and programs to identify the ability to transfer and apply learned numeracy skills in real, functional contexts, but not only those where the mathematics is explicit and obvious. At the same time, the ability to handle functional numeracy tasks will not necessarily imply that a learner has acquired generalizable skills or the ability to handle abstract or more formal mathematical concepts and ideas. Thus, a mix of both functional and more abstract tasks may be needed.
Finally, many studies and curricular frameworks emphasize the critical role that positive attitudes, beliefs, and habits of mind (collectively called here dispositions) can play in supporting effective and confident numerate behavior and in enhancing adults ability to manage the mathematical demands of different situations (MacLeod, 1996; Gal, 2002b). Indeed, items measuring selected dispositions associated with mathematical activities and feelings about mathematics are part of major large-scale assessments in the mathematical areas (e.g., NAEP, PISA), and more elaborate scales have been included in the ALL survey and shown (Gal et al., 2005) to correlate positively with performance.
Overall, the picture emerging from this review is that it is presently unclear how well statements about target skills or curricular standards (as reflected, for example, in the EFF and ANN standards, or in the NRS skill specification) are linked to the structure and content of key assessment schemes and how all these are aligned with instructional practices. Too little is known about the types and uses of assessments developed locally by teachers or programs, and no systematic analyses have been conducted of the mathematics or numeracy components of the most recent versions of the TABE, the CASAS, and other assessment systems. Clearly there is room for states and programs to develop or experiment with assessment systems that offer better or at least more explicit alignment with curricular goals and teaching methods, and some states (such as Massachusetts or Ohio) have recently begun to take steps in this direction.
Conclusions
Multiple factors converge to challenge anyone trying to summarize the state of the field in numeracy assessment or to offer any quick fixes. The scope of the objects to be assessed (e.g., skills, capacities, dispositions) and the deficiencies of current assessment methods depend to some extent on the eye of the beholder. Definitions of numeracy vary, from relatively narrow notions that focus on computational or procedural skills to broader, integrative views that go well beyond computations and encompass a broad set of mathematical skills, communicative and interpretive abilities, supporting dispositions, and skill transfer. The teaching/learning contexts in which adults engage with numeracy-related topics vary greatly given the diversity of backgrounds, ages, or purposes for learning that adults bring. Conceptions regarding the goals of adult numeracy education and of the competencies to be developed evolve over time and are affected by multiple stakeholders (Curry et al., 1996; Forman & Steen, 1999; Kilpatrick, 2001).
Policymakers pressures for accountability have an impact on what skills are deemed important and assessed (or not) by programs and teachers (Stites, 2000). Funding and program accountability requirements, and especially the NRS, place different emphases on preferred forms of assessment, although they allow the concurrent use of tests that provide different types of information of varying qualities. This approach may provide information that learners have made some progress, but what exactly that progress is and whether the progress is toward valued goals cannot be fully ascertained from the reported information provided by current instruments. The reliance on only standardized tests for reporting, tests that presently rely on multiple-choice format, runs the danger of pushing more teachers to teach for the test and stay away from using assessment approaches that rely more on performance in diverse forms, such as oral presentations, portfolios, or various locally developed assessments (which are quite popular with adult educators in countries such as United Kingdom and Australia).
Given NRS demands, there is likely to be a tendency to use only one or two styles of information gathering for multiple purposes (Nagel, 1999; Messemer & Valentine, 2004). The use of only one or two instruments may in fact have some advantages; for example, it can enable learners, teachers, and funders to use a consistent system of concepts and interpretations when reporting and discussing assessment results. Yet, the use of only one or two instruments may be problematic, given on the one hand the different needs of learners, teachers, and programs and on the other hand the breadth of the numeracy construct (at least according to the integrative views presented in Section 2) that is at present only partially addressed by existing large-scale assessments. The multiplicity of purposes or contexts of the uses of assessment outlined earlier implies a need for different forms and types of information gathering and interpretation at different points during a learners progress in a program. The reports covered in our review do not contain much information on the extent to which the assessment m e t h o d s i n u s e i n p r o g r a m s a r e s u f f i c i e n t l y d i f f e r e n t i a t e d i n t e r m s o f t h e s e m u l t i p l e p u r p o s e s t h e y m a y b e , b u t t h i s i s s u e r e q u i r e s f u r t h e r i n v e s t i g a t i o n , g i v e n t h a t i n e a c h a s s e s s m e n t o n l y a l i m i t e d n u m b e r o f i t e m s a r e u s e d .
A s C u m m i n g &