Papers and Presentations, Mathematics and Science Initiative
This is the third education summit in the last year and a half at which I've had the role of summarizing scientific research. I'm honored to be here, but I have to tell you, this isn't getting any easier.
The present topic, mathematics, has been particularly challenging because the research base is relatively thin, there is a lot of controversy about what math children should learn and how they should be taught it, and there is a dearth of good work that directly addresses the needs of educators and policy makers.
As witness to the latter point, a superintendent of a large school district contacted me not long ago to ask if there was research that would help him select an elementary school math curriculum that would be effective for the types of children served by his district. He had just taken over as superintendent, and there was a lot riding on his curriculum decisions, both for him and for the children in the district. I had to tell him that there was no rigorous research on the efficacy of widely available elementary mathematics curricula, and that about all I could offer him was my opinion. He said thanks, but he had plenty of opinions already.
I am not saying that there is no quality research on the learning or teaching of mathematics or that the existing research is irrelevant to instructional decisions. Were that the case there would be nothing left for me to do now except sit down. However, I am saying that research on math is in its infancy compared, for example, to research on reading, and that what it provides for policy and practice is more in the way of educated guesses than strong direction.
Achievement and equity
Our strongest data are from statistical studies that plot changes in achievement levels over time and that allow comparisons of achievement across different types of learners and different education systems. These statistical reports can't tell us how to make things better, but they do tell us how we're doing and whether we're making progress. Other speakers today will address in some detail results from such assessments as the National Assessment of Educational Progress, known as NAEP, and the Third International Math and Science Study, known as TIMSS.
I will summarize the NAEP results by saying that although the trend for achievement is generally up over the last 30 years, we have a long way to go. Large numbers of U.S. students show mastery of only rudimentary mathematics and only a small proportion achieve at high levels.
In the most recent NAEP, only 26% of grade 4 students, 27% of grade 8 students, and 17% of grade 12 students were judged "proficient" in mathematics. At the same time 31% of grade 4 students, 34% of grade 8 students, and 35% of grade 12 students scored below the "basic" level.
Students who score below the basic level do not demonstrate even partial mastery of the material that is appropriate for their age group. For example, in 1996, few fourth graders below the basic level could answer the question: How many fourths make a whole? Overall, half of the nation's fourth graders could not answer this question.
Low levels of achievement are more likely among minority groups and children from lowincome backgrounds than among children from advantaged circumstances. For instance, in the year 2000 NAEP, 68% of African American 8th graders scored below basic in math compared to only 23% of white students. For the past decade, the gap in NAEP mathematics scores between white and black students and between white and Hispanic students has remained large and relatively unchanged.
On the TIMSS, our performance relative to other nations depends dramatically on children's grade and age. Our children are above average in elementary school, average in middle school, and are nearly at the bottom of the pack at exit from high school.
While levels of achievement in mathematics among U.S. students are low, the demand for a mathematically proficient workforce is increasing. The United States cannot fill all the jobs in mathematically intensive fields with qualified U.S. citizens. As a result, Congress has been forced in recent years to provide an expanded pool of visas for foreign nationals with hightech skills. At the same time, the number of college degrees awarded in technical areas has dropped sharply for United States citizens.
What do we know about the effectiveness of practices and policies in mathematics education?
Tracking
Many schools in the U.S. separate children into different classes based on class grades or standardized test scores in mathematics. Typically tracking or grouping of students for instruction begins in earnest in middle school. Proponents of grouping argue that teachers are better able to meet student needs when the students in the same class are relatively homogeneous in ability level. Opponents of grouping argue that it has pernicious effects, particularly for low achieving students for whom standards are lowered. The large research base on the effects of grouping indicates that, in fact, lower performing students do less well when they are grouped together rather than taught in heterogeneous classes, while ability grouping provides little advantage to higher performing students (as long as the same curriculum is being delivered in all classes). Given the requirement of the No Child Left Behind Act that all children in grades 3 through 8 become mathematically proficient, and given the research indicating that ability tracking negatively affects children in the lower performing groups, there are good reasons to abandon ability grouping, particularly in grades K through 8.
Assessment
A number of studies have examined the value of feeding back into the classroom the results from frequent assessment of students. One approach that has been studied provided teachers with weekly performance graphs on individual students Children in classrooms in which teachers received this feedback performed at significantly higher levels than students in classrooms in which the performance graphs were not available. Other studies have shown that student performance is enhanced still further when instructional materials tailored to each student's progress are provided to teachers along with student performance graphs.
The existing research on the positive effects of frequent classroom assessment suggests the potential of a system that would link statewide content standards, to endofyear statewide performance assessments, to frequent classroombased assessment of individual student progress towards yearend goals, to instructional materials individualized to each student's needs. Elements of such a system are in place or are being built here and there, but no state or district is anywhere close to having a seamless, coherent assessment system of the type I've described.
Structured peer feedback
It is difficult for teachers to provide individualized feedback to students learning math in the typical classroom of 20 to 25 students. Structured peer feedback, or peer tutoring, can provide students with more immediate feedback than would be available from a teacher, can enhance motivation for learning through the powerful influence of peers, and may heighten understanding by putting children in the position of having to explain or teach concepts to other children.
Several high quality studies have addressed the effects of structured peer feedback in mathematics education. As it turns out, not all forms of peer tutoring are created equal. In one study, 4th and 5th graders were assigned to work together in pairs to solve math problems in class. Children who were required to switch back and forth between the role of teacher and student, and children who were rewarded for the performance of both members of their team had much better math scores after 5 months than children whose rewards were based on their individual performance and who weren't required to switch systematically between the roles of teacher and student. Thus peer tutoring can be effective, but simply putting children together and asking them to help each other solve problems is not the way to go about it. The peer relationship has to be structured so that children reciprocate between teacher and learner roles, and so that children are rewarded for the performance of all the members of the team.
The Math Wars
For those who may be unaware, we are in the middle of a philosophical war about what math children should be taught and how they should be taught it. During the 1990s in the United States, a reform movement in mathematics and science education gained ascendance based on the constructivist theories of the famous Swiss psychologist, Jean Piaget. A number of views characterize constructivism as applied to mathematics education, including: 1) that children construct their own understanding of mathematical principles, rather than being passive receptacles of knowledge imparted by others; 2) that the goal of instruction is to aid children in developing their own understanding, rather than to teach mathematical facts and procedures or to impart conventional conceptions of mathematics; 3) that children construct an understanding of mathematics by working with concrete, real life referents, not by learning facts and procedures in isolation; and 4) that children learn through curiosity and a desire to understand their world, not through the imposition of external rewards.
In keeping with these views, 1990s constructivist reforms in math involved problems and examples drawn from reallife situations and designed so that children could discover underlying mathematical principles in the process of working out their own solutions. These studentdirected activities, sometimes called discovery learning, took precedence over direct instruction by teachers. Professional development for teachers emphasized the teacher's own conceptual understanding of mathematics as well as the teacher's ability to figure out what children are thinking in real time in order to guide children's efforts to make sense of math.
In addition to these pedagogical shifts, constructivist reforms redefined standards for mathematics content and performance. The definition of mathematical proficiency was broadened from the ability to perform the computations necessary to solve math problems, to the ability to understand mathematical concepts, to apply mathematics to novel problems, and to reason mathematically. Broadly speaking, reformers focused on the development of conceptual understanding of mathematics and lessened the emphasis on fluent mastery of mathematical facts and procedures. Reports, recommendations, and standards from such respected sources as the National Council of Teachers of Mathematics and the National Research Council indicated that instruction should address all strands of proficiency simultaneously, and rejected the "old ways" of teaching math, such as mastering computational procedures followed by problem solving.
Critics of the 1990s math reforms charged that in shifting the focus from computation to understanding, from teacherdirected to studentdirected learning, and from sequenced instruction along a hierarchy of skills to simultaneous instruction in all strands of mathematical competence, reformers generated a recipe for poor preparation of students for mathematically challenging content. The critics argue that the 1990s math reforms ignore the importance of fundamental building blocks, are not sufficiently rigorous, do not cover aspects of math content that are necessary for proficiency, and overgeneralize the role of curiosity and discovery as core principles of mathematics learning.
It would be very satisfying if I could tell you that the math wars have been resolved based on high quality research. Unfortunately, we are far from that. However, there is research that suggests where some of practices and assumptions of both the constructivists and their critics may require more nuanced implementation.
Conceptual understanding
Let's start with an area in which there is a clear victory for constructivism: the importance of conceptual understanding. There is a large literature demonstrating the limits on generalization of math skills that can occur when instruction focuses exclusively on learning facts and procedures.
Consider decimal fractions. Learning how to add and subtract decimal fractions is often a struggle for children because of preconceptions they bring to the task from integer arithmetic. For instance, with integers, numbers with more digits represent larger quantities than numbers with fewer digits, e.g., 509 is more 7. However, this relationship breaks down for decimal fractions, where .7 is more than .509. Research has shown that children who receive sufficient practice doing arithmetic with decimal fractions may be able to add and subtract correctly while having a very narrow understanding of the meaning of the numbers they are computing. For example, they may be able to add .5 and .4, but may be unable to provide a number between .5 and .4, or to identify the pizza that has .5 of its slices remaining.
Studies have shown that providing children with practice on visual representations of decimal fractions can help children transfer their knowledge to problems on which they have not been trained. For example, one set of researchers had children practice locating decimal fractions on a visual number line. Zero and one were marked on the scale.. Given a fraction such as 0.509, children would try to locate it on the number line, and would receive feedback on whether they were correct.
Children who received practice on this visualization task were subsequently better able to solve other decimal tasks such as choosing the decimal fraction that is nearest to, greater than, or less than a target. Thus practice with the number line visualization gave children a way of representing the meaning of decimals that allowed them to transfer their learning to problems that were different in form but similar in the underlying mathematics to the problems on which they were trained.
Thus, conceptual understanding is a good thing because it can tie together mathematical tasks that might otherwise seem disconnected to a child.
After acknowledging the importance of conceptual understanding for mathematical proficiency, the story gets more complex. In the basic constructivist view, conceptual understanding is best arrived at through discovery learning with reallife problems, is sufficient in itself for mathematical proficiency, and can be generated through teaching methods that appeal to children's natural curiosity.
However, a number of studies have demonstrated that conceptual understanding can be produced through a variety of pedagogical techniques, including carefully sequenced direct instruction on the underlying principles, practice on a wide variety of problem types, and exposure to worked examples. In other words, the type of knowledge that allows a learner to solve many types problems doesn't need to be discovered by the learner to be effective. For example, children who practiced placing decimal fractions on a number line, in the study I've just described, still made mistakes after considerable practice. A frequent error was to ignore a zero in the tenths position, thus treating .03 as .30. A subsequent study showed that simply telling children to "notice the first digit" before they solved problems substantially enhanced their performance compared to basic discovery learning with the number line problems. In other words, providing direct instruction on what to attend to created conceptual understanding.
A second finding that complicates the basic constructivist view is that discovery activities may substantially compromise learning unless the child already has mastered the background knowledge that is relevant to the problem to be explored.
Cognitive psychologists have discovered that humans have fixed limits on the attention and memory that can be used to solve problems. One way around these limits is have certain components of a task become so routine and overlearned that they become automatic. For instance, children who are first learning to read have great difficulty paying attention to the meaning of the text because they are working so hard on sounding out the words. However, once decoding has become fluent, the child no longer needs to think about it, and the brain resources that the child has available to solve realtime problems can be directed towards understanding the meaning of print.
Although teaching children to understand the principles of multiplication by having them double a cookie recipe may seem like a good idea, if the child doesn't know the times table, the cognitive requirements of working with cookie dough and the cognitive requirements of multiplication will be too much to handle and will detract from learning. A clear instructional implication is that discovery activities should come later in a sequence of instruction, after children have acquired the requisite background knowledge to handle openended, real life problems. I am not saying that discovery activities should wait until graduate school. They can occur at any grade. However, the child should be prepared for the activity so that he can focus on what is important to the instructional goal. This is a basic principle of instructional design that is often ignored in approaches that rely on discovery activities.
A third finding that complicates the basic constructivist view concerns the inefficiency of discovery methods. Per the previous discussion of the number line, it may take a very long time for some children to discover that they have to pay attention to the first digit in solving decimal fractions. Why not tell them? As the famous psychologist Jerome Bruner said about discovery learning back in 1966, "it is the most inefficient technique possible for regaining what has been gathered over a long period of time." The algorithms, procedures, and facts of mathematics are powerful cultural inventions that have accumulated over thousands of years of human history. We simple cannot expect every child to discover the Pythagorean theorem. As with the previous point, this argues for discovery methods being used judiciously. There is a time and a place in an instructional sequence for children to apply their skills and to learn how to solve authentic openended problems. It is not everyday or everyway.
The final tenet of constructivism that requires a more nuanced view is that learning should be intrinsically motivated; it should be fun. Part of the rationale for discovery learning with real life problems is that drill and practice on math facts and procedures are not fun. If math isn't fun, children won't learn it, or so the thinking goes.
This slide includes data from 37 countries in which 8th grade students took the 1995 TIMSS assessment. The X axis represents how much the children said they liked math. The Y axis represents their math scores. The correlation is  .63 between math scores and how much kids like math. This is a very strong negative relationship. Children in countries that were at the top on math achievement, such as Japan, don't like math. Children in countries at the bottom of international achievement, such as South Africa, like math more than children from any other country in the world. Perhaps you find that as interesting as I do.
Don't get me wrong. I am not arguing for the pedagogical value of drudgery. Whatever we can do to make any academic subject more intrinsically interesting is worthwhile, as long as we don't compromise the content along the way. In mathematics, we can use computerbased animation, realtime feedback, appropriately placed real life problems, and social processes to make learning more fun. The younger the child, the more important these motivators will be. However, the type of practice that results in skills becoming automatic typically takes considerable repetition and timeontask. This is true for hitting a tennis ball or playing the violin or decoding written text or doing mathematical calculations. Doing something over and over again until you don't have to think about it may rarely be great fun, particular in the context of other ways that children can spend their time. By failing to acknowledge that mathematical learning involves work, the United States may be placing a ceiling on the levels of proficiency that it can expect its students to achieve.
Curriculum
Curriculum is an area in which the competing claims of math reformers and math traditionalists should be testable. Curriculum involves both content and pedagogy (or methods of teaching). At the broadest level we would expect a constructivist curriculum to focus on content such as realworld problems and teaching methods that involve discovery learning, while a skillsbased curriculum would focus on computation through teaching approaches that involve considerable practice.
Unfortunately, there are no studies that have pitted carefully implemented constructivist curricula against carefully implemented skillsbased approaches. The typical study in the literature compares the effects of a particular curriculum against businessasusual, and compares classrooms or districts that have volunteered to use the target curriculum against those that have not. These are weak designs because the very act of volunteering to use a new curriculum typically carries with it extra motivation to succeed, and thus biases the results towards the new curriculum. Nevertheless, such studies can be informative given certain outcomes. For instance, large and educationally meaningful differences favoring the target curriculum or the businessasusual approach would be difficult to ignore if the schools being compared were roughly similar prior to the introduction of the target curriculum. This is because large effects are difficult to obtain in education and are unlikely to be generated just by the bias for more motivated schools to volunteer for a new curriculum. Very small differences or parity between the target curriculum and businessasusual could also be informative. For instance, if achievement were low in both conditions, it might suggest that neither businessasusual nor the target curriculum should be continued.
A recent study examined achievement test data from three states in schools using elementary school reform mathematics curricula that were developed since the late 1980s with funding from the National Science Foundation. These curricula generally followed the constructivist principles I've articulated. Constructivist math students' test results were compared to those of students in businessasusual schools that were matched by reading score and poverty level. Over 100,000 student test scores were analyzed. While the results favored students using the constructivist curricula, the absolute size of the differences between the constructivist students and the businessasusual students was small. Averaging across the grades in the three states, constructivist students answered correctly 66% the total possible questions on endofyear math tests, and businessasusual students answered 65% of the questions correctly.
With respect to the math wars, these findings suggest that constructivist math curricula are not doing any harm to children compared to businessasusual approaches, at least with respect to state assessments. Given the concerns that have been raised in some quarters, this is reassuring. However, some would argue that the organizational, material, and staff development costs necessary to shift to a constructivist curricula cannot be rationalized on the basis of such small differences in outcomes.
Professional Development
Proponents of constructivist reforms might argue that the small differences in outcomes are a reflection of failures to properly implement the reforms. Indeed, there are several studies demonstrating that the effects of math curricula on students outcomes are mediated by the quality of implementation of a curriculum
Three schools in Pittsburgh that were weak implementers of a standardsbased math curriculum were compared with three schools with similar demographics that were strong implementers. Note that racial differences were eliminated in the strong implementation schools, and that performance soared.
It would be inappropriate, however, to prefer a constructivist math curriculum over another approach based on demonstrations that the constructivist curriculum works better when it is well implemented that when it is not. Everything works better when it is well implemented than when it is not. We need, but do not have, research that compares the costs and effectiveness of contrasting programs that are well implemented.
Alignment
While I have focused on mathematics education at the classroom level, there are both logical and empirical reasons to believe that substantial and sustained improvement in math achievement requires attention to school and district level variables that provide the support for effective classroom practice in mathematics. At the broadest level this raises questions about how to achieve alignment among mathematics content standards, performance standards, accountability systems, curriculum, teaching materials, assessment, classroom practice, and the preparation and professional development of mathematics teachers.
Differences in mathematics achievement among schools and districts serving students of similar economic and racial/ethnic backgrounds are likely to reflect, in part, differences in the alignment of components of policy and practice. When these differences occur within states where every school is operating under the same state standards and accountability system, they point to the important role of organizational and management variables at the local level in enhancing student learning. There are thousands of schools across the nation serving large numbers of lowincome and minority children that manage to generate high mathematics achievement. We need to learn how to clone these schools. When the differences occur across states, they point to the importance of statelevel policies in supporting mathematics achievement for all students within a state.
North Carolina and Texas managed to lead the nation in math gains in the 1990s. They did so despite high minority enrollments, lower than average per pupil expenditures, and average class sizes. What North Carolina and Texas had in common was an aligned series of policies, including state standards for math content in each grade, an assessment system designed to measure achievement of those standards, and accountability at the school level for performance.
Putting It Together
What do we know from the research I've reviewed?
We know that we need to do better in math education in the United States, particularly in middle and high school. We know that at the classroom level, frequent assessment is useful, particularly when teachers are given help on what they should do for children who aren't performing well. We know that children can help each other learn math if peer tutoring is appropriately structured. We know that tracking children by ability level is likely to widen achievement gaps rather than diminish them. We know that how children think about math problems is important, not just whether they get the right answer. We know that direct instruction can help students learn computational skills and understand math principles. As a corollary, we know that children don't have to discover math principles on their own or work with authentic openended problems in order to understanding mathematical concepts. At the same time, there are situations in which discovery activities can be useful. We know that we don't have to make math fun for children to learn it, though a little sugar to help the medicine go down is a good thing. Most importantly, we know that classrooms are not islands and that broad achievement gains require coordinated systems from the state level down to the district and schoollevel that align standards, assessment, accountability, instructional leadership, effective management, teacher professional development, and curricula.
These are not trivial or unimportant research findings. There is a lot that we don't know, but what we know already, if put into practice, could be the basis for substantial progress towards the goal of mathematical proficiency for all children.
Sources Used
Anderson, J.R., Reder, L.M., & Simon, H.A. Applications and misapplications of cognitive psychology to mathematics education. Pittsburgh: Carnegie Mellon University (http://actr.psy.cmu.edu/papers/misapplied.html)
COMAP (2002). The ARC Center tristate student achievement study. Lexington, MA: The Consortium for Mathematics and Its Applications.
Darch, C., Carnine, D., & Gersten, R. (1984). Explicit instruction in mathematics problem solving. The Journal of Educational Research, 77(6), 351359.
Dixon, R.C., Carnine, D.W., Lee, D., Wallin, J., & Chard, D. (1998). Review of high quality experimental mathematics research. Report to the California State Board of Education.
Fantuzzo, J.W., King, J.A., & Heller, L.R. (1992). Effects of reciprocal peer tutoring on mathematics and school adjustment: A component analysis. Journal of Educational Psychology, 84(3), 331339.
Fantuzzo, J.W., Davis, G.Y., & Ginsburg, M.D. (1995). Effects of parent involvement in isolation or in combination with peer tutoring on student selfconcept and mathematics achievement. Journal of Educational Psychology, 87(2), 272281.
Fuson, K.C., Smith, S.T., & LoCicero, A.M. (1997). Supporting Latino first graders' tenstructured thinking in urban classrooms. Journal of Research in Mathematics Education, 28(6), 738766.
Gersten, R., & Baker, S. K. (1998). Real world use of scientific concepts: Integrating situated cognition with explicit instruction. Exceptional Children, 65(1), 2335.
Griffin, S.A., Case, R., & Siegler, R.S. (1994). Rightstart: Providing the central conceptual prerequisites for first formal learning of arithmetic to students at risk for school failure. In K McGilly (Ed.), Classroom lessons: Integrating cognitive theory and classroom practice, pp. 2549. Cambridge, MA: The MIT Press.
McDaniel, M.A., & Schlager, M.S. (1990). Discovery learning and transfer of problemsolving skills. Cognition and Instruction, 7(2), 129159.
Moore, L. J., & Carnine, D. W. (1989). A comparison of two approaches to teaching ratio and proportions to remedial and learning disabled students: Active teaching with either basal or empirically validated curriculum design material. Remedial and Special Education, 10(4), 2837.
National Center for Education Statistics (2001). The Nation's Report Card: Mathematics 2000. Washington, D.C.: U.S. Government Printing Office (NCES 2001517).
National Center for Education Statistics (2000a). The condition of education 2000. Washington, D.C.: U.S. Government Printing Office (NCES 2000062).
RittleJohnson, B., & Siegler, R.S. (1998). The relation between conceptual and procedural knowledge in learning mathematics: A review. In C. Donlan (Ed.), The development of mathematical skills. Psychology Press.
RittleJohnson, B. R., Siegler, R. S., & Alibali, M. W. (2001). Developing conceptual understanding and procedural skill in mathematics: An iterative process. Journal of Educational Psychology, 93(2), 346362.
Slavin, R. (1983). When does cooperative learning increase student achievement? Psychological Bulletin, 94(3), 429445.
Tuovinen, J.E.; Sweeler, J. (1999). A comparison of cognitive load associated with discovery learning and worked examples. Journal of Educational Psychology, 91(2), 334341.
U.S. Bureau of Labor Statistics (1999). Occupational projections and training data, 19981999 edition. Washington, D.C.: U.S. Bureau of Labor Statistics
Woodward, J., & Baxter, J. (1997). The effects of an innovative approach to mathematics on academically lowachieving students in mainstreamed settings. Exceptional Children, 63(3), 373388.
Woodward, J., Baxter, J., & Robinson, R. (1999). Rules and reasons: Decimal instruction for academically lowachieving students. Learning Disabilities Research and Practice, 14, 1524.


