 
 
Education Development Center, Inc.
Principal Investigator: Ellen Mandinach
Efficacy Project: An Examination of the Impact of Big Math for Little Kids on
PreK and Kindergarten Students' Learning of Math
American children do not perform as well in mathematics as children from other countries, and American children from low income families do not do as well as children from middle and upper income families. Many policymakers and education officials are calling for a greater emphasis on mathematics education in early childhood education programs, but such an emphasis raises the question of how to provide effective mathematics education to young children. The purpose of this project is to evaluate the efficacy of a math curriculum that has been developed for prekindergarten and kindergarten students, Big Math for Little Kids (BMLK). This program offers a structured sequence of activities designed to promote challenging mathematical learning and related verbal expression. BMLK is designed for use at least 20 to 30 minutes each day throughout a typical 32week academic year. Lessons, which can be implemented in large groups, small groups, and with individual children, take the form of games, activities with manipulatives, explorations, stories, a very small amount of work with writing and reading mathematics, and various other activities. Lessons are organized into six major content strands: number, shape, measurement, operations on numbers, patterns and logic, and space. The strands cover similar topics at each age level, but offer more advanced material at the kindergarten level. Teachers are provided with suggestions on how to assess children's mathematical learning and thinking in the context of instruction.
In the first year of the project, 16 large, public childcare centers will be randomly assigned to either the treatment or the control condition. In the eight treatment centers, 16 prekindergarten teachers (two per center) will attend a twoday BMLK summer training program, with six followup sessions during the academic year. An equal number of teachers from the eight control group centers will serve as the comparison group. The comparison group will continue to use Creative Curriculum, the standard program used in these centers. In the second year the teachers will implement the two curricula in their classrooms, serving 320 prekindergarten children in each curriculum group. In the third year the researchers will follow students who remain in the same centers for kindergarten. A total of eight kindergarten classrooms, with approximately 160 children, will be included for each curriculum group. The researchers are using a nationally normed test of math learning to compare the effects of the two curricula on student learning. In addition, data obtained from classroom observations and teacher interviews will be used to examine whether children's learning is related to the fidelity and intensity with which the teachers implement the BMLK curriculum. The goal is to develop and evaluate a program that can be used to improve student mathematics achievement in early childhood education.
Educational Testing Service
Principal Investigator: Dylan Wiliam
Development Project: Developing and Using Diagnostic Items in Mathematics
and Science
Competence in mathematics and science is a requirement for many jobs in a technologically advanced society. International comparisons of the academic achievement of U.S. students in mathematics and science show that their performance leaves plenty of room for improvement. The purpose of this project is to develop a set of diagnostic questions, closely tied to common state content standards, that teachers can use to better understand student misconceptions that impede their academic progress in science and mathematics, and to carry out an evaluation of the effects on student learning of having teachers integrate the use of these questions into their teaching.
In the first phase of this project, the researchers are examining mathematics and science education materials currently in use, and developing questions teachers can use to identify student misconceptions that may occur in the process of mastering that material. In the second phase the researchers are working with 48 elementary and middle school teachers to have them use those questions in their instruction, using this as a basis for refining the questions and the way the researchers help the teachers learn how to make use of them. In the third phase, the researchers are randomly assigning 96 teachers either to learn how use the refined questions in their teaching or to continue as before, and subsequently evaluating the effects of this approach on the academic achievement of their students. The researchers are also using classroom observation and teacher journals to study how much teachers make use of these questions and how that use alters the teachers' classroom practices. The goal of the project is to improve students' math and science achievement by developing an effective strategy for improving mathematics and science instruction at the elementary and middle school levels.
Quantum Solutions, Inc.
Principal Investigator: Benny Johnson
Development Project: Integrated Software for Artificial Intelligence Tutoring
and Assessment in Science
American students do comparatively poorly in international studies of academic achievement at the high school level in chemistry, which is a core subject in the physical sciences course sequence in American high schools. To date the capacity of computerassisted instruction in science to strengthen student learning has been considered quite limited. The purposes of this project are to complete the development of a computerbased tutoring and assessment system for a firstyear chemistry course and to accumulate evidence of the potential of the system to improve student learning and achievement. The system is being designed for use by individual students on demand at school or at home.
The researchers are first developing the procedures for grading student work in an intelligent tutoring system that they have already created. Actual student work samples will be used to test and validate the system's procedures and to conduct an initial external review of the system with a small group of high school chemistry teachers. The system is being designed to provide helpful detailed comments to students if they make mistakes while trying to solve chemistry problems and to generate reports for teachers and students that analyze student learning as reflected by their performance on a given set of chemistry problems.
When the system is fully developed, it will be field tested with chemistry teachers in several high schools. In this quasiexperimental study, 10 to 12 teachers will implement the tutoring system with the assessment component in one chemistry class. In a second class, each teacher will continue normal instruction, not using any functions of the tutoring system. With a third class, each teacher will use the assessment system, but not the tutorial, and with a fourth class, each teacher will use only the tutoring program. Using a prepost design, student achievement will be compared across the four groups. Because it is not possible to have a randomly selected group in this test, statistical analyses will be used to determine preexisting similarities and differences between the groups. The end result of this project will be a fully developed intelligent tutor and assessment system for high school chemistry with evidence of the potential of this system to improve high school students' chemistry achievement.
University of Oregon
Principal Investigator: David Chard
Development Project: Early Learning in Mathematics: A Prevention Approach
Children's mathematics achievement in the early grades in U.S. schools has been low for decades, and the gap between the achievement levels of children from lowincome families and minorities and the achievement levels of their more advantaged peers is large. At the same time, expectations for mathematics achievement have risen for students, particularly in the early grades, including kindergarten. Unfortunately, insufficient information is available regarding how best to meet these rising expectations for students as they are just beginning school, especially for students experiencing significant difficulties in mathematics, and there is very little empirical evidence regarding the effects of mathematics reforms on learning. The purpose of the current study is to address the need to identify practices that promote mathematics learning in young children by developing and evaluating the efficacy of a mathematics instructional intervention and progress monitoring system designed to address the instructional needs of a wide range of kindergarten learners.
In the first year of the project, mathematics instruction strategies, curriculum materials, and student activities are being developed, along with tests and observation procedures designed to measure student learning and the quality of the educational approach. This work is being done in collaboration with four kindergarten teachers recruited based on their expertise in kindergarten curriculum and instruction who will serve as teacherresearchers and who will pilot test intervention materials with their students. In the second year of the project, a study will be conducted in which 20 kindergarten teachers will be randomly assigned either to use the new mathematics instruction approach in their classrooms or to continue as before. In both conditions, mathematics instruction will occur for approximately 20 minutes per day, 5 days per week for the entire school year. A series of mathematics assessments will be administered to determine the efficacy of the new mathematics curriculum compared with the standard curriculum. These will include standardized mathematics assessments at pretest and posttest (one of the standardized assessments will also be administered at midyear). In addition, a sample of eight randomlyselected students (four atrisk, four average achieving) in each classroom will be administered performance assessments at three points during the year to determine ongoing student learning and progress. Observational and interview measures will be used to assess the fidelity and intensity of implementation, as well as the participation and engagement of the 8 students selected for more indepth assessment.
In the third year, the teachers who participated in the second year study will use the same instructional approach they used in the previous year with a new class of kindergarten students, and the same assessment procedures will be used. In addition to examinations of the differential efficacy of the new instructional approach compared with standard mathematics instruction, an additional focus will be on whether teachers assigned to use the new mathematics instructional intervention become more adept at using the new approach during the second year, and whether this results in improved student outcomes, particularly for atrisk students. In addition, during year 3, students who participated during Year 2 will be reassessed at the end of their first grade year to determine the longerterm impact of the kindergarten intervention on student mathematics learning. The overall goal of this project is to develop a usable and effective approach to mathematics instruction in kindergarten, including instructional strategies, curriculum materials, a computer software program for student practice, and assessment strategies for identifying students who are having trouble learning mathematics.
WestEd
Principal Investigator: Steve Schneider
Development Project: Algebraic Interventions for Measured Achievement
Student proficiency in algebra in the United States is quite low, and the proficiency of Black and Hispanic students is substantially lower than that of White students. These low rates are becoming even more significant as workplace demands increasingly require the kind of problem solving skills learned as part of algebra. The purpose of this project is to develop and evaluate a curriculum designed to target typical trouble spots in school algebra, in order to strengthen students' mastery of key algebra skills.
The researchers are designing lesson units of software, print materials, and teacher professional development aimed at improving student understanding in six topic areas, namely, patterns, basic numeracy, variables and equations, ratio and proportion, coordinate plane, and inequalities. These lesson units are being integrated with the Cognitive Tutors educational software program, a program that has been found in prior research to improve student algebra performance. The researchers are evaluating the quality of these units in a study in which 15 teachers are randomly assigned to use the units in their high school algebra classes, while 15 teachers in the same schools continue to teach as before, with a total expected student sample of 1500 students. The researchers are then revising the lesson units to improve them further, and then carrying out the same kind of study in the following year. The researchers are testing the students to see if the units are more effective compared to the standard curriculum, and to see if the revised units lead to better student performance compared to the initial versions. The goal of the project is to improve students' algebra performance by producing new computer and teacherdirected lessons targeting specific topics that commonly cause problems for students.


