Data & Research EVALUATION OF PROGRAMS
Reading First Implementation Evaluation Final Report
Report Highlights


Background

The No Child Left Behind Act of 2001 (PL 107-110) established the Reading First (RF) Program (Title I, Part B, Subpart 1), a major Federal initiative designed to help ensure that all children can read at or above grade level by the end of third-grade. The legislation (Section 1205 of the Elementary and Secondary Education Act) requires an evaluation of the implementation of the program. To meet this requirement, the Department contracted with Abt Associates to conduct an evaluation that addresses the following questions:

  1. How does reading instruction differ in RF schools and non-RF Title I schools as RF schools' implementation efforts mature over time?
  2. Does student achievement improve more quickly in schools with Reading First funds than in non-RF Title I schools not receiving RF funds?
  3. Is there any relationship between how schools implement Reading First-aligned practices and changes in reading achievement?

This final study report provides findings from surveys completed in spring 2005 and 2007 by K-3 teachers, principals, and reading coaches in nationally representative samples of RF schools and non-RF Title I schools; and state and national databases that provided measures of school-level reading scores on state assessments across all school districts nationwide.

Report Findings

Reading programs implemented in RF schools differ from those in non-RF Title I schools in several ways.

Instructional Time. RF schools dedicated more time to reading instruction in their K-3 classrooms than did non-RF Title I schools. Most RF and non-RF Title I schools scheduled designated reading blocks of at least 90 minutes for their K-3 classrooms; (98 percent versus 77 percent). K-3 teachers in RF schools reported spending an average of 103 minutes per day on reading activities, compared to 81 minutes reported by teachers in non-RF Title I schools. This translates to approximately 110 additional minutes per week of reading instruction for K-3 students in RF schools.

Instructional Materials. After making substantial changes to their reading programs during their first year of implementation in 2004-05, mature RF schools were less likely to make many additional changes in 2006-07 than were non-RF Title I schools. Only 3 percent of RF schools adopted a new core reading program in 2006-07 and 14 percent added new materials for English language learners (ELLs). However, some RF schools continued to add new intervention programs for struggling readers (40 percent) and new supplementary materials (42 percent) in 2006-07.

Scientifically based Reading Research (SBRR) Alignment. Reading coaches were more likely to report that their K-3 teachers are knowledgeable about SBRR (79 percent vs. 58 percent), that core reading programs in RF schools are more likely to be aligned with SBRR than are the programs in non-RF Title I schools (93 percent vs. 76 percent), and that reading intervention materials in their schools are aligned with SBRR (94 percent vs. 79 percent) than were their counterparts in non-RF Title I schools.

Assistance to Struggling Readers. In 2006-07, RF teachers were more likely than teachers in non-RF Title I schools to report placing struggling readers into intervention services (80 percent vs. 63 percent), and to report using diagnostic assessments to determine struggling readers' core deficits (84 percent vs. 67 percent).

Assessment. Teachers in RF schools were more likely than teachers in non-RF Title I schools to rate the use of assessment results as central to their instruction for such purposes as grouping students (91 percent vs. 78 percent), identifying students in need of interventions (83 percent vs. 69 percent), and measuring student progress (88 percent vs. 80 percent). In 2006-07, RF teachers were significantly more likely than teachers in non-RF Title I schools to identify standardized tests as the assessment used most often and to administer such tests more often than teachers in non-RF Title I schools.

Reading Coaches. RF schools were significantly more likely to have a reading coach than were non-RF Title I schools (99 percent vs. 57 percent). Reading coaches in RF schools reported spending larger proportions of their time as in the role of reading coach than did coaches in non-RF Title I schools; specifically, 75 percent of RF coaches and 19 percent of coaches in non-RF Title I schools reported that they spend all of their time in this role.

Professional Development. In 2006-07, a greater percentage of RF teachers reported participating in reading-related professional development activities than did teachers in non-RF Title I schools (90 percent vs. 73 percent). On average, teachers in RF schools reported participating in approximately twice the number of workshops (5.5 vs. 2.8) and spending twice the number of hours in reading-related professional development activities (30.6 hours vs. 15.5 hours) than did teachers in non-RF Title I schools.

Non-RF Title I schools increasingly report activities aligned with the principles of Reading First.

Instructional Materials. Four of the five reading programs cited most frequently by RF schools were also among the five most popular programs in the non-RF Title I schools. At least 10 percent of both RF and non-RF Title I schools reported using Harcourt Trophies, McGraw-Hill Open Court, Scott Foresman Reading and Houghton Mifflin Reading.

Assistance to Struggling Readers. Materials and activities specifically aimed at helping struggling readers were available in most non-RF Title I schools (85 percent) and RF schools (91 percent).

Professional Development. In 2006-07, teachers in non-RF Title I schools reported participating in professional development on the five essential components at high levels: reading comprehension (90 percent), fluency (86 percent), phonics (76 percent), phonemic awareness (75 percent), and vocabulary (72 percent).

Based on analyses of states' reading assessment scores, there is limited but statistically significant evidence that successive cohorts of third- and fourth-grade students in RF schools improved their reading performance over time more quickly than did their counterparts in non-RF Title I schools.

Third-Grade Reading Performance. Average effect sizes across 24 states indicate that RF schools gained between 2 and 3 percentage points more, on average, from pre-to post-RF implementation than non-RF Title I schools on the proportion of students meeting standards on states' third-grade reading assessments, a statistically significant yet small difference (p < .001). In 12 of 24 states, the improvement in third-grade reading scores among RF schools was statistically significantly larger than in non-RF Title I schools for at least one of the four methods used to define pre- and post-RF implementation years. In the other 12 states, there were no statistically significant differences between the two groups of schools.

Fourth-Grade Reading Performance. Average effect sizes across 17 states indicate that RF schools gained between 2 and 3 percentage points more, on average, from pre- to post-RF implementation than non-RF Title I schools on the proportion of students meeting standards on states' fourth-grade reading assessments, a statistically significant yet small difference (p < .001). In six of 17 states, the improvement in fourth-grade reading scores among RF schools was statistically significantly larger than in non-RF Title I schools for at least one of methods (described earlier) used in the analysis. In 11 states there were no significant differences between the two types of schools.

There is a positive and statistically significant relationship between only one of four measures of RF and non-RF Title I schools' implementation of RF-aligned activities, as measured through surveys, and their levels of third-grade reading achievement.

The study team analyzed the relationship between schools' third-grade reading scores on state assessments and four composite measures constructed from survey data that characterize teachers' RF-aligned activities: classroom reading instruction; strategies to help struggling readers; participation in professional development; and uses of assessment to inform instruction. [  1  ] For every increase of one standard deviation unit in the struggling readers implementation composite score, the probability of being in the top quartile increased by 15.6 percentage points, for the average school (p <.001).

This report is available online at www.ed.gov/about/offices/list/opepd/ppss/reports.html.

Notes

  1. This analysis includes both RF and non-RF Title I schools, since the implementation of activities aligned with Reading First can potentially occur in either type of school.


 
Print this page Printable view Send this page Share this page
Last Modified: 10/09/2008