Name of Institution: RAND Corporation
Principal Investigator: Dr. John Pane
Title: Experimental Field Study of Cognitive Tutor Geometry Curriculum
Purpose: The need for effective mathematics curricula and instruction at the high school level has been well documented. Data from the National Assessment of Educational Progress continue to show very low mathematics proficiency among high school students, with large gaps in the performance of students of different racial, ethnic and income groups. Many educators recognize the need to improve high school mathematics achievement, but do not always know how to address the problem because few curricula are supported by rigorous scientific research. The purpose of this study is to obtain evidence on the effectiveness of Cognitive Tutor Geometry, a high school geometry curriculum that has shown promise in earlier quasi-experimental studies. The program combines individualized tutorial software with teacher-guided group work and problem solving. At the end of the project, the evidence will show whether implementation of Cognitive Tutor Geometry in one large district improves student performance on the state's end-of-course geometry assessment.
Setting: The evaluation is being conducted in 8 high schools in the Baltimore County Public School District, an urban fringe district that serves students from a wide range of racial/ethnic and socioeconomic backgrounds.
Population: The participants are approximately 2,000 high school students and 16 math teachers.
Intervention: Cognitive Tutor Geometry is designed to provide individualized instruction to address students’ specific needs. The curriculum includes challenging problems that reflect real-world situations and provide opportunities for students to progress from concrete to abstract thinking. Students spend 40 percent of their class time using individualized tutorial software built on a detailed computational model of student thinking; 60 percent of class time will be devoted to teacher-guided group work and problem solving. As part of implementation, teachers receive four days of teacher training. The curriculum has shown evidence of promise from quasi-experimental studies.
Research Design and Methods: The study is a randomized controlled trial in which two teachers in each participating high school will teach Geometry concurrently during two periods per day. One teacher is randomly assigned to teach Cognitive Tutor Geometry during the earlier period, and the other teacher uses the school's existing Geometry curriculum. During the later period, the teachers switch curricula. For both periods, students enrolled in Geometry are randomly assigned to experimental or control classes. The study will run for two years, using the same teachers to the extent possible, so as to capture any improvements in implementation after a year of experience with the curriculum.
Control Condition: Students in the control group receive their high school's usual geometry instruction.
Key Measures: Pre- and post-test data will be collected on student performance on the Maryland State Department of Education Geometry Assessment. In addition, student survey data will be collected on other outcomes that may influence long-term student achievement, such as mathematics confidence and attitudes, and career plans. Data will be collected on student course-taking.
The fidelity of implementation will be monitored using a combination of classroom observations, teacher interviews, and a collection of classroom artifacts including lesson plans, logs, and teacher-developed assessments.
Data Analytic Strategy: The analysis strategy uses hierarchical linear models to estimate the impact of Cognitive Tutor Geometry on test scores in geometry and generalized mixed models to estimate the impact on dichotomous variables including course taking, attitudes, and career plans.