More Resources Complete Report  PDF (463K) | MS Word (600K)

The Comprehensive School Reform (CSR) program provides both a framework and the funding to enable schools to change their organization and practices so all students can achieve high standards. In 1998, Congress appropriated $145 million for the Comprehensive School Reform Demonstration (CSRD) program as a way to encourage schools to integrate local, state, and federal resources into a comprehensive effort that would better meet student learning needs. Like Title I schoolwide programs, CSRD was intended to help schools use multiple sources of funds and integrate programs while allowing flexibility and enhancing accountability for student learning. Its unique aspect was the expectation that schools would collaborate with expert partners to implement whole-school reform models that had a strong research base and a successful replication record. With the passage of the No Child Left Behind (NCLB) Act in 2002, CSR became a fully authorized program and is no longer a demonstration program. Further, NCLB describes 11 components of comprehensive school reform and, some argue, focuses on the underlying processes that facilitate the kinds of changes needed in order for schools to ensure that all students learn rather than specific whole-school reform models.

Evaluation Questions

The Longitudinal Assessment of Comprehensive School Reform Implementation and Outcomes (LACIO) responds to the NCLB Act's requirement for an evaluation of the federal Comprehensive School Reform (CSR) program. The legislation sets two broad goals for the evaluation: first, to evaluate the implementation and outcomes achieved by schools after three years of implementing comprehensive school reforms and, second, to assess the effectiveness of comprehensive school reform in schools with diverse characteristics. In order to address these requirements, the study focuses on the targeting of CSR program funds; the implementation of CSR in schools, districts and states; the conditions at the state and district level that influence implementation of CSR in schools; and the relationship between CSR implementation and student achievement outcomes.

This first year report focuses on the implementation of school reform activities and the targeting of CSR program funds. The report presents data collected from a random sample of 400 CSR schools that received funding in 2002 and 400 non-CSR schools with similar demographic and achievement characteristics. It draws from three data sources—school-level surveys of principals and teachers, the National School Level State Assessment Score Database, and the National Center for Educational Statistics (NCES) Common Core of Data (CCD). The data were collected in spring 2003, at the end of the first year of CSR implementation (2002-03) in the sample schools.

Key Study Findings

Two key findings emerged from the first year of this evaluation.

First, although both CSR and non CSR schools are engaged in reform, reform in CSR schools is more likely to include adoption of models and other activities closely associated with research-based models. CSR schools, as compared with non-CSR schools, were more likely to implement the following components:

• Adopt externally developed strategies that have been replicated. CSR schools were more likely to identify a specific reform model (85 percent of CSR schools vs. 49 percent of non-CSR schools) and to report using evidence from research that the reform model chosen improves student achievement (42 percent vs. 26 percent).
• Provide more continuous professional development opportunities. CSR schools were more likely to offer professional development activities for all teachers (90 percent vs. 73 percent) and allocate over 10 days to teacher professional development (56 percent vs. 39 percent).
• Include measurable goals for student performance associated with the reform model (57 percent vs. 41 percent).
• Reflect support from staff by including a formal vote by teachers for the reform model (82 percent vs. 55 percent).
• Provide support for staff by receiving on-site consulting relevant to the reform (85 percent vs. 57 percent).
• Evaluate the reform. CSR schools were more likely to include the requirements of the reform model in the scope and content of evaluation (66 percent vs. 42 percent) and assess the utility of external assistance (41 percent vs. 30 percent).

The second key finding is that CSR funds are strongly targeted to high-poverty schools and low-performing schools, and schools receiving CSR funds are lower performing than are other schools with similar demographic characteristics at the time they receive awards.

• CSR funds were strongly targeted to high-poverty schools and to those with high concentrations of minority students. Almost half (45 percent) of CSR schools had poverty rates of at least 75 percent, nearly three times greater than the percentage of all schools in this highest-poverty group (16 percent) and close to double the percentage of Title I schools (26 percent)
• At the time of funding, CSR schools were significantly more likely to report that they were identified as a low-performing school according to the criteria used in their state (46 percent) than were the non CSR schools (28 percent).
• When CSR schools received their funding they were also more likely to have received state sanctions due to low performance (11 percent vs. 3 percent).
• CSR schools had lower baseline achievement scores than did Title I schoolwides in reading and mathematics at most grade levels (elementary, middle, and high school) at the time the awards were made.

Conclusion

Both CSR schools and non-CSR schools report that they are implementing many components of comprehensive school reform. However, CSR schools appear more likely to include adoption of models and other activities more closely associated with research-based models. With CSR funds, the schools were more likely to adopt models, focus professional development, and track student performance than were non CSR schools. Furthermore, states and districts seem to have targeted CSR funds to those schools that have the greatest need to change practices in order to support high achievement for all students.

This report did not measure the quality of implementation, and subsequent reports from this evaluation will examine this issue. One such question is whether the use of CSR funds accelerates reform in the lowest performing schools. In subsequent years, the evaluation will provide information about whether CSR schools implement reform components more thoroughly than do non CSR schools.