Elementary and Secondary Education
Implementing Accountability and Supports under ESEA Flexibility (2016) presents the results of a study of the early implementation of the ESEA flexibility accountability and support provisions. The findings are based on telephone interviews conducted with a sample of 12 states, 22 districts, and 25 school-level officials in late 2013 and early 2014 and a review of relevant policy documents. All of the selected states were approved for ESEA flexibility during the first two submission windows (November 2011 and February 2012).
Early Implementation of State Differentiated Accountability Plans Under the No Child Left Behind Act (2012) describes how nine states implemented their differentiated accountability plans through January 2010, based on interviews and reviews of state documents. The differentiated accountability pilot was established by the Department to allow states to waive certain accountability requirements by varying the intensity and types of interventions provided in Title I schools identified for improvement. The Department approved differentiated accountability waivers for Florida, Georgia, Illinois, Indiana, Maryland, and Ohio in July 2008 and for Arkansas, Louisiana, and New York in January 2009.
Key findings include: Eight of the nine participating states determined school improvement status based on both the percentage of AYP indicators the school had met and the number of years the school had been in school improvement. Although most changes enacted were allowable prior to the waiver, state respondents reported that the differentiated accountability pilot provided an impetus for their states to implement strategies intended to better coordinate, target, and expand their technical assistance services.
Comparability of State and Local Expenditures Among Schools Within Districts: A Report From the Study of School-Level Expenditures (2011). This report presents findings from the first-ever national data collection on school-level expenditures, collected in response to a requirement in the American Recovery and Reinvestment Act of 2009 (ARRA). The report examines the distribution of state and local education expenditures at the school level, including comparisons between Title I and non-Title I schools and between higher-poverty and lower-poverty schools.
Key findings include: Within districts that had both Title I and non-Title I schools, more than 40 percent of Title I schools had lower personnel expenditures per pupil than did non-Title I schools at the same school grade level. Similarly, more than one-third of higher-poverty schools had lower per-pupil personnel expenditures than lower-poverty schools in their districts. In addition, between 39 to 47 percent of Title I districts had lower per-pupil expenditures in their Title I schools than in their non-Title I schools at the same grade level.
The Potential Impact of Revising the Title I Comparability Requirement to Focus on School-Level Expenditures (2011). This policy brief uses the school-level expenditure data from the Study of School-Level Expenditures to examine the potential impact of revising the Title I comparability requirement to focus on school-level expenditures. The federal Title I program requires that school districts provide services in higher-poverty, Title I schools from state and local funds that are at least comparable to those in lower-poverty, non-Title I schools. The current Title I comparability requirement allows school districts to demonstrate compliance in various ways and does not require comparability of actual school-level expenditures.
Key findings include: An estimated 18 to 28 percent of Title I districts would not be in compliance with an expenditures-based comparability requirement, depending on the specifications of the requirement. However, the estimated cost of complying with an expenditures-based comparability requirement amounts to just 1 to 4 percent of school-level expenditures in affected districts, on average. Low-spending Title I schools and higher-poverty schools would see their per-pupil expenditures rise by an average of 4 to 15 percent.
- Policy Brief: PDF (272KB)
Supplemental Educational Services and Student Achievement in Five Waiver Districts (2011) presents final implementation and outcome findings from the five districts that received waivers to serve as Supplemental Educational Service (SES) providers, despite being identified for improvement, corrective action or restructuring. Federal regulations prohibit school districts identified for improvement or corrective action from serving as SES providers. The SES waiver pilot program allowed five identified districts to serve as SES providers beginning in 2005–06 (Boston and Chicago), 2006–07 (Hillsborough County, Florida and Anchorage, Alaska), and 2008–09 (Charlotte-Mecklenburg, North Carolina). In 2009–10, the pilot was replaced with a more expansive waiver opportunity that allows states to request a waiver from the U.S. Department of Education to approve identified districts or schools as SES providers.
- In the three districts that did not serve as SES providers before the waiver (Anchorage, Charlotte Mecklenburg, and Hillsborough), SES participation rates increased in the first year of the waiver. (Boston and Chicago served as providers prior to receipt of the waiver.)
- There were few demographic or academic differences between students served by district providers and students served by non-district providers.
- Students in three of the five districts demonstrated statistically significantly larger mathematics achievement gains during periods of SES participation than during periods of nonparticipation. In addition, in two districts, SES participation was associated with statistically significant reading gains.
- Averaged across the five districts, the overall association between SES participation and achievement gains was statistically significant in both mathematics and reading, relative to nonparticipation.
- Across the five districts, the achievement gains associated with SES participation relative to nonparticipation did not differ for district and non-district providers for either mathematics or reading.
- All five districts reported using multiple communication strategies to reach eligible families, provided balanced information about SES providers, translated information into at least one language other than English, and provided extended enrollment periods.
Final Report on the Evaluation of the Growth Model Pilot Project (2011) documents the Growth Model Pilot Project (GMPP). GMPP was initiated to allow states to experiment with adjustments to the No Child Left Behind Act (NCLB) status accountability system, in order to improve the validity of AYP determinations by giving schools credit for students who are making significant growth. The pilot allowed states, districts, and schools to count students who were "on track" to being proficient but not yet there. Under NCLB, such students were not counted as proficient for the purpose of AYP determinations. The pilot was initiated in November 2005 with the goal of approving up to ten states to incorporate growth models in school AYP determinations. The project was written into regulation in late 2008; now any state may apply to use a growth model meeting certain core principles. Currently, 15 states are implementing growth models under this authority: Alaska, Arizona, Arkansas, Colorado, Delaware, Florida, Iowa, Michigan, Minnesota, Missouri, North Carolina, Ohio, Pennsylvania, Tennessee, and Texas.
Key findings include:
- Growth models enabled additional schools to make AYP compared to status and safe-harbor rules alone, but the percentages of schools that made AYP because of the growth models were generally not large.
- The impact of growth models varied widely across states.
- Most (but not all) schools that made AYP by status would also have met their reading and math AMOs under a hypothetical "growth-only" model (i.e., one using neither status nor safe harbor but only growth).
- Controlled simulations comparing the impacts of different types of growth models on student and school growth results show that the "projection model" functions in stark contrast with "transition" and "trajectory" models.
- Simulations comparing the results of different growth models using the same data show that projection models have the highest correct classification rates for future proficiency: over 80 percent. These rates are 5 to 20 percentage points higher than trajectory and transition models, depending on the grade level and proximity to the growth model time limit. While the projection model is more accurate, it is theoretically more difficult to implement and to explain to practitioners and parents than the other models.
- Although not an option under the Growth Model Pilot guidelines, growth models not tied directly to proficiency standards could identify a broader contingent of students as making adequate growth than current models. One alternative to the GMPP-permissible growth-to-proficiency models that could be used with vertical test score scales is the difference between proficiency cut scores in successive grade levels.
State and Local Implementation of the No Child Left Behind Act, Volume IX-Accountability Under NCLB: Final Report (2010) provides updated information on state, district, and school implementation of NCLB provisions concerning accountability and school improvement. The report is based on the second round of data collection from the National Longitudinal Study of NCLB and the Study of State Implementation of Accountability and Teacher Quality Under NCLB. The report presents findings from interviews with state education officials in all states and surveys of nationally-representative samples of districts, principals, and teachers conducted in 2004-05 and 2006-07.
Key findings include: In 2006-07, 20 percent of Title I schools (10,781 schools) were identified for improvement, and 53 percent of these schools were located in 1 percent of the nation's Title I districts (177 districts). Among schools that did not make adequate yearly progress (AYP) in 2005-06, 55 percent missed targets for either the school as a whole or for multiple student subgroups. States reported performance results for 2005-06 testing more quickly than for 2003-04 testing, but roughly one-third of states were still finalizing calculations and processing appeals well into the school year. All states reported having a system of support for schools identified for improvement, and most states reported providing some level of support to all identified schools. In 2006-07, required interventions occurred in most, but not all, Title I schools in improvement or corrective action. However, most Title I schools in restructuring status did not experience any of the specific interventions named in the law.
Interim Report on the Evaluation of the Growth Model Pilot Project (2010) documents the Growth Model Pilot Project (GMPP). GMPP was initiated to allow states to experiment with adjustments to the No Child Left Behind Act (NCLB) status accountability system in order to improve the validity of AYP determinations, by giving schools credit for students who are making significant growth. The pilot allowed states, districts, and schools to count students who were on track to being proficient (but not yet there). Under NCLB, such students are not counted as proficient for the purpose of AYP determinations. The pilot was initiated in November 2005 with the goal of approving up to ten states to incorporate growth models in school AYP determinations under NCLB. No longer a pilot, the project was written into regulation in late 2008; now any state may apply to use a growth model meeting certain core principles. Currently 15 states are implementing growth models under this authority: Alaska, Arizona, Arkansas, Colorado, Delaware, Florida, Iowa, Michigan, Minnesota, Missouri, North Carolina, Ohio, Pennsylvania, Tennessee, and Texas.
Key findings include: Growth models enabled additional schools to make AYP, but overall percentage increases were not large. The growth models in the first eight pilot states resulted in an additional 371 schools making AYP in 2006-07, a 6 percent increase over the number that made AYP under the NCLB status and safe harbor criteria. High-poverty schools were more likely to benefit from the status-plus-growth framework (an 8 percent increase in the number making AYP) than were schools with low poverty rates (a 3 percent increase). The impact of growth models varied widely across states. The percentage increase in the number of schools making AYP due to growth models ranged from 0 to 2 percent in four states to a high of 12 to 14 percent in three states (Florida, Arkansas, and Iowa). Three of the eight states (Arkansas, Florida, and Iowa) accounted for 90 percent of the schools that made AYP solely through growth, and two states (Florida and Iowa) accounted for 71 percent of such schools. In Iowa, 69 percent of the schools that missed AYP under the NLCB status and safe harbor rules were able to make AYP under their growth model. However, in the other seven states, the growth model had a much smaller impact, allowing 0 to 17 percent of schools that would not have otherwise made AYP to do so using growth. Most (but not all) schools that made AYP by status would also have made AYP under the growth model alone. Eighty-five percent of the schools that made AYP under the NCLB status criteria also would have made AYP strictly by using the growth criteria.
Title I Implementation: Update on Recent Evaluation Findings (2009) provides a summary of findings from Title I evaluation studies that have become available after the publication of the National Assessment of Title I final report in 2007. The report presents data collected in 2006-07 through the National Longitudinal Study of NCLB and the Study of State Implementation of Accountability and Teacher Quality Under NCLB. The report includes findings from interviews with state education officials in all states, surveys of nationally representative samples of districts, principals, and teachers, data from consolidated state performance reports, and analyses of student achievement trends on state assessments and NAEP.
Key findings include: In states with consistent achievement trend data from 2004-05 to 2006-07, the percent of students reaching the state's proficient level rose for most student groups, but most states would not meet NCLB's goal of 100 percent proficiency by 2013-14 unless student achievement increases at a faster rate. Nearly 11,000 Title I schools were identified for improvement in 2006-07, and almost half were in the more advanced stages of corrective action and restructuring. Most elementary teachers reported no change from 2004-05 to 2006-07 in the amount of instructional time that they spent on specific subjects. Student participation in Title I school choice and supplemental educational services (SES) continues to rise, and district expenditures on these choice options doubled from 2003-04 to 2005-06. In a subsample of seven districts, student SES participants showed greater achievement gains than non-participating students, but no statistically significant relationship was found between school choice participation and student achievement. Most teachers have been designated as highly qualified under NCLB, but teachers in high-poverty schools had less experience and were less likely to have a degree in the subject that they teach.
An Exploratory Analysis of Adequate Yearly Progress, Identification for Improvement, and Student Achievement in Two States and Three Cities (2009). This report presents the results of exploratory quasi-experimental analyses that use a Regression Discontinuity (RD) design to examine the relationships between certain features of NCLB accountability and subsequent student achievement in Title I schools in two states and three school districts. Specifically, the report examines the effects of not making AYP or of being identified for the first year of school improvement status (after missing AYP for two consecutive years).
Key findings include: The study found some positive achievement impacts for schools that missed AYP, but not for schools that were identified for the first year of school improvement; effects were not consistent across years and outcomes. Findings from two states and three cities cannot be generalized to produce a national estimate of program effects on student achievement. In addition, the report discusses several study limitations, including technical features of the RD method requiring that the analysis focus on schools that had missed AYP or had been identified for improvement for the first time, which may be relatively weak interventions relative to the full set of progressively more intensive interventions prescribed under Title I.
State and Local Implementation of the No Child Left Behind Act, Volume VIITitle I School Choice and Supplemental Educational Services: Final Report (2009) provides updated information on the implementation and usage of choice options that are offered to students in Title I schools that have been identified for improvement. The report is based on the second round of data collection from the National Longitudinal Study of NCLB and the Study of State Implementation of Accountability and Teacher Quality Under NCLB. The report presents findings from interviews with state education officials in all states and surveys of nationally-representative samples of districts, principals, and teachers conducted in 2004-05 and 2006-07, as well as surveys of parents in eight large urban school districts in those same years.
Key findings include: Numbers of students eligible for and participating in choice and supplemental educational services (SES) have increased substantially, although participation rates remained stable at about 1 and 17 percent, respectively. Nearly all districts required to offer these options reported that they notified parents, and the timeliness of parent notifications has improved. However, eligible parents who were surveyed in the eight districts were often unaware of the choice options, even though all eight districts provided evidence that they had sent notification letters to parents about the options. Among parents surveyed who took advantage of the Title I choice options, over 80 percent said they were satisfied.
State and Local Implementation of the No Child Left Behind Act, Volume VITargeting and Uses of Federal Education Funds (2009) examines how well federal funds are targeted to districts and schools serving economically disadvantaged students, how Title I targeting has changed over the past seven years, how districts have spent federal funds, and the base of state and local resources to which federal funds are added. The report covers six federal programs: Title I, Part A; Reading First; Comprehensive School Reform (CSR); Title II, Part A; Title III, Part A; and Perkins Vocational Education State Grants. The report uses data on federal program allocations from all states, as well as data from a nationally representative sample of 300 school districts on federal program allocations and expenditure data for the 2004-05 school year.
Key findings include: Federal education funds were more strongly targeted to high-poverty districts than were state and local funds. However, the higher level of federal funding in high-poverty districts was not sufficient to close the funding gap between high- and low-poverty districts. The overall share of Title I funds going to the highest-poverty districts and schools changed little between 1997-98 and 2004-05, and the highest-poverty schools continued to receive smaller Title I allocations per low-income student than did the lowest-poverty schools. Schools that were identified for improvement were more likely to receive Title I funds than non-identified schools, but they received smaller allocations per low-income pupil. Most funds for the six federal programs in this study were used for instruction and instructional support.
State and Local Implementation of the No Child Left Behind Act, Volume VImplementation of the 1 Percent Rule and 2 Percent Interim Policy Options (2009) presents findings about the implementation of regulations and guidelines issued under the No Child Left Behind Act that provide flexibility for the treatment of certain students with disabilities in state assessment and accountability systems. These findings are from the Study of State Implementation of Accountability and Teacher Quality Under NCLB, based on surveys of state officials in 2004-05 and 2006-07 and analysis of extant data about state implementation of NCLB assessment and accountability requirements.
Key findings include: Most states with accurate data reported that less than 10 percent of tested students with disabilities participated in an alternate assessment based on alternate achievement standards. Twenty-two states granted exceptions to districts to exceed the 1 percent cap on the inclusion of proficient and advanced scores from alternate assessments based on alternate achievement standards for AYP calculations for 2005-06 testing, and 21 states used the 2 percent proxy option for AYP calculations. Data from a subset of states suggested that the effects of using the 2 percent proxy varied greatly by state. For example, in one state, use of the 2 percent proxy did not enable any schools to make AYP, whereas in another state, 159 schools made AYP in 2005-06 through the 2 percent proxy.
Supplemental Educational Services and Student Achievement in Waiver Districts: Anchorage and Hillsborough (2009). This report has been removed from this website because an error was discovered in the report's analyses. The report should not be cited. An updated report is in progress that includes three additional districts, as well as additional years of longitudinal student achievement data, and is expected to be released later this year.
State and Local Implementation of the No Child Left Behind Act, Volume IV—Title I School Choice and Supplemental Educational Services: Interim Report (2008) provides information on the implementation and usage of choice options that are offered to students in Title I schools that have been identified for improvement. The report is based on the first round of data collection from the National Longitudinal Study of NCLB and the Study of State Implementation of Accountability and Teacher Quality Under NCLB. The report presents findings from interviews with state education officials in all states and surveys of nationally-representative samples of districts, principals, and teachers conducted in 2003-04 and 2004-05, as well as surveys of parents in eight large urban school districts in those same years.
National Assessment of Title I: Final Report to Congress (2007) presents findings from the congressionally mandated National Assessment of Title I on the implementation and impact of the program. Volume I contains key findings on the implementation of the program under No Child Left Behind, and Volume II presents a report on follow-up findings from Closing the Reading Gap, an evaluation of the impact of supplemental remedial reading programs on achievement of 3rd and 5th grade students.
- Complete Report website
State and Local Implementation of the No Child Left Behind Act, Volume III-Accountability Under NCLB: Interim Report (2007) draws on data from the 2004-05 data collection cycles of two federally funded studies -- the Study of State Implementation of Accountability and Teacher Quality Under NCLB (SSI-NCLB) and the National Longitudinal Study of NCLB (NLS-NCLB) -- to describe patterns in state, district, and school implementation of NCLB provisions concerning accountability and school improvement.
State and Local Implementation of the No Child Left Behind Act, Volume I-Title I School Choice, Supplemental Educational Services, and Student Achievement (2007) examines the impact of participation in Title I school choice and supplemental educational services (SES) on student achievement, as well as the characteristics of participating students. The quasi-experimental analysis used student-level participation and state assessment data from nine large urban districts for 2000-01 through 2004-05.
Key findings include: On average, across seven districts that could be included in the achievement effects analysis, participation in SES had a statistically significant, positive effect on students' achievement in reading and math. Students participating for multiple years experienced larger gains. In contrast, across six districts, no statistically significant effect on achievement, positive or negative, was found for students participating in Title I school choice, but sample sizes for school choice were much smaller than were those for supplemental services. Across the nine districts, SES participants had lower prior achievement than eligible students who did not participate, while Title I school choice participants had similar prior achievement levels to eligible nonparticipants.
- Highlights: PDF (109 KB) | MS Word (87 KB)
- Full Report: PDF (1.42 MB) | MS Word (3.31 MB)
- Interpretation of Effect Sizes: PDF (110 KB) | MS Word (42 KB)
National Assessment of Title I: Interim Report to Congress (2006) includes two parts: Volume I, "Implementation of Title I," and Volume II, "Closing the Reading Gap: First Year Findings from a Randomized Trial of Four Reading Interventions for Striving Readers."
- Complete Report website
Title I Accountability and School Improvement Efforts From 2001 to 2004 (2006) examines the implementation of accountability and school improvement under Title I of the No Child Left Behind Act (NCLB) from 2001-02, the year before NCLB went into effect, through 2003-04, the second year of implementation of NCLB. The report includes a special focus on 2003-04, with findings on identification of schools for improvement, interventions implemented at schools identified for improvement, and public school choice and supplemental educational services under Title I.
Evaluation of Title I Accountability Systems and School Improvement Efforts: Findings From 2002-03 (2005) examines the implementation of Title I accountability provisions during the 2002-03 school year, the first full year NCLB was in effect. The report provides information on the identification of schools for improvement and interventions implemented in these schools, including public school choice and supplemental educational services under Title I.
Case Studies of Supplemental Services Under the No Child Left Behind Act: Findings from 2003-04 (2005) examines implementation of supplemental educational services provisions of Title I of the No Child Left Behind Act (NCLB) during the 2003-04 school year, the second year the requirements were in effect, through case studies of nine districts in six states. Building on findings reported in the Year One Report for the case studies, this report details how supplemental services were implemented at all levels, considers continuing challenges to implementation, and provides additional examples of promising approaches.
State ESEA Title I Participation Information for 2003-04 (2007) summarizes Title I information reported by states in their performance report for school year 2003-04 as well as comparisons to 2002-03 and previous years. The report covers the Title I, Part A, Grants to Local Educational Agencies program and provides information on the numbers of districts, schools, and students served; the numbers meeting adequate yearly progress (AYP) criteria and those identified for school improvement; instructional and other services supported under Title I; and staffing.
State ESEA Title I Participation Information for 2002-03 (2006) summarizes Title I information reported by states in their performance report for school year 2002-03 as well as comparisons to previous years. The report covers the Title I, Part A, Grants to Local Educational Agencies program and provides information on the numbers of districts, schools, and students served; the numbers meeting adequate yearly progress (AYP) criteria and those identified for school improvement; instructional and other services supported under Title I; and staffing.
State ESEA Title I Participation Information for 2001-02 (2005) summarizes Title I information reported by states in their performance report for school year 2001-02 as well as comparisons to previous years. The report covers the Title I, Part A, Grants to Local Educational Agencies program and provides information on the numbers of districts, schools, and students served; the numbers meeting adequate yearly progress (AYP) criteria and those identified for school improvement; instructional and other services supported under Title I; and staffing.
State Education Indicators with a Focus on Title I: 2003-04 (2007) provides information on key indicators of the condition and progress of K-12 public education in the 2003-04 school year, including indicators of state progress in implementing state accountability systems. It includes two-page state profiles as well as summary tables of several accountability-related indicators. State profiles, a national summary, and the complete report are available.
State Education Indicators with a Focus on Title I: 2002-03 (2007) provides information on key indicators of the condition and progress of K-12 public education in the 2002-03 school year, including indicators of state progress in implementing state accountability systems. It includes two-page state profiles as well as summary tables of several accountability-related indicators. State profiles, a national summary, and the complete report are available.
State Education Indicators with a Focus on Title I: 2001-02 (2005) provides information on key indicators of the condition and progress of K-12 public education in the 2001-02 school year, including indicators of state progress in implementing state accountability systems. It includes two-page state profiles as well as summary tables of several accountability-related indicators. State profiles, a national summary, and the complete report are available.
Early Implementation of the Supplemental Educational Services Provisions of the No Child Left Behind Act: Year One Report (2004) examines the implementation of the supplemental educational services provisions of Title I of the No Child Left Behind Act (NCLB) during the 2002-03 school year, the first school year that the requirements were in effect. Through case studies of nine districts in six states, the report describes how states, districts, schools and providers implemented supplemental services. The report also looks at how challenges experienced during the first year of implementation can be overcome in future years and provides some examples of promising approaches.
Evaluation of Title I Accountability and School Improvement Efforts (TASSIE): First Year Findings (2004) examines the implementation of Title I accountability provisions during the 2001-02 school year, the year before NCLB went into effect, and provides a baseline for tracking the implementation of NCLB accountability provisions over time. The report provides information on the identification of schools and districts in need of improvement. It also describes support and interventions implemented for low-performing schools and schools identified for improvement.
State ESEA Title I Participation Information for 2000-2001 (2004) summarizes Title I information reported by states in their performance report for school year 2000-2001, as well as comparisons to 1999-2000 and previous years. The report covers the Title I, Part A, Grants to Local Educational Agencies program and provides information on the numbers of districts, schools, and students served; the numbers meeting adequate yearly progress (AYP) criteria and those identified for school improvement; instructional and other services supported under Title I; and staffing.
High Standards for All Students: A Report from the National Assessment of Title I on Progress and Challenges Since the 1994 Reauthorization (2001) provides a comprehensive summary of the most recent data available from the National Assessment of Title I on the implementation of the Title I program and the academic performance of children in high-poverty schools.
National Longitudinal Survey of Schools examines the implementation of the Title I program from 1998-99 to 2000-01 based on surveys of principals and teachers in a nationally representative sample of Title I schools. Two reports and three evaluation briefs are available:
A Snapshot of Title I Schools, 2000-01 (2003) examines the implementation of the Title I program in 1998-99 through 2000-01 based on surveys of principals and teachers in a nationally representative sample of Title I schools.
PDF (424K) | MS Word (2.5MB)
A Snapshot of Title I Schools Serving Migrant Students, 2000-01 (2003) provides information on implementation of the Title I program in schools serving migrant students. It compares Title I schools with migrant students to all Title I schools in the areas of social, demographic, and organizational characteristics and the implementation of Title I provisions between 1998-99 and 2000-01.
PDF (237K) | MS Word (1.3MB)
Evaluation Brief: Schools Identified as in Need of Improvement Under Title I (2002) examines levels of understanding of school improvement status by school principals; technical assistance received by these schools; whether schools have been subjected to corrective actions; schools' progress in meeting adequate yearly progress targets and moving out of school improvement status.
PDF (347K) | MS Word (186K)
Evaluation Brief: Provision of Title I Services (2002) examines the extent to which changes in Title I legislation have helped promote school improvement activities, as well as the provision of instructional services including extended time, use of pullout and in-class instruction, use of teacher aides; and coordination of services for special population students.
PDF (339K) | MS Word (222K)
Longitudinal Evaluation of School Change and Performance: Final Report (2001) examines changes in student performance in a sample of 71 Title I schools, based on a longitudinal sample of students as they progressed from 3rd to 5th grade between 1997 and 1999. The study analyzes student outcomes associated with specific aspects of curriculum and instruction and identified policy conditions-especially regarding standards-based reform-under which effective classroom practices were likely to flourish.
- Report Highlights
- Executive Summary: PDF (473K) | MS Word (152K)
- Technical Report: PDF (1.22MB) | MS Word (.98MB)
Promising Results, Continuing Challenges: National Assessment of Title I (1999) summarizes findings from a variety of studies conducted for the National Assessment of Title I, examining the implementation and impact of the program. The report examines progress in the performance of students in high-poverty schools, the development of state standards and assessment systems, accountability systems and school improvement efforts, the targeting of Title I funds, Title I services at the school level, support for family involvement, services for students in private schools, and services provided under the Even Start, Migrant Education, and Neglected and Delinquent programs.
Language Learners (Title III)
Educational Experiences of English Learners: Analyses of Extant Data (2016). Three briefs describe the educational experiences and performance of English learner (EL) students based on data from the 2011–12 Civil Rights Data Collection (CRDC) and other data from EDFacts. The instructional staff brief explores ELs’ exposure to novice, uncertified, or frequently absent teachers. The college preparatory courses and programs brief explores the extent to which ELs have access to and participate in advanced coursework and other college preparatory activities. The grade retention, high school graduation, and GED attainment brief explores the educational success of ELs with respect to grade retention, high school graduation, and GED preparation program participation and credential attainment.
- Instructional Staff: PDF (305K)
- College Preparatory Courses and Programs: PDF (428K)
- Grade Retention, High School Graduation, and GED Attainment: PDF (674K)
National Evaluation of Title III Implementation: A Survey of States' English Language Proficiency Standards (2012). This report summarizes findings from a survey of the English Language Proficiency (ELP) standards used by states during the 2008-09 school year. The survey examined the basic structure of the standards, as well as basic content features such as references to, and support for, the acquisition of academic English and knowledge in the content areas.
- Across all of the states and the District of Columbia, there were 32 different sets of ELP standards used during the 2008-09 school year.
- Twenty-eight of the 32 sets of ELP standards were structured to address grade-level or grade-cluster expectations from kindergarten through grade 12 in each of the reading, writing, speaking and listening domains.
- Nineteen of the 32 sets of ELP standards included explicit references to the state's academic content areas or standards in at least one content area.
- Twenty-nine sets of ELP standards made at least occasional reference (two or more times) to at least one of the four essential linguistic components of academic language identified for this studythe phonological, lexical, grammatical, and functional component.
- Six sets of standards (representing 25 states) provided support to aid educators in the form of specific instructional suggestions.
National Evaluation of Title III Implementation: Report on State and Local Implementation (2012). This report answers a range of questions about the implementation of the Title III program. It draws on data collected during the 2009-10 school year through telephone interviews with all state Title III directors, a survey of a nationally representative sample of 1,528 Title III subgrantees, and case studies of a purposive sample of 12 districts nested within five states. The study also analyzed extant data such as data from the Consolidated State Performance Reports.
- Title III districts vary in the criteria they use to determine which students are considered English Learners (ELs), so a student who is identified as an EL according to one district's practices may or may not be identified as such in another district.
- Due to variation in how states defined and measured their AMAOs, AMAOs in one state were not comparable to AMAOs in another state. Fifty-five percent of Title III districts nationwide reported meeting all three of their AMAOs in 2008-09.
- In 2009-10, English as a Second Language (ESL) was the most common type of EL service among Title III districts and instruction in the native language was the least common type of EL service.
- In 2009-10, officials in more than half of Title III districts reported difficulty recruiting some categories of teachers for ELs.
The Language Instruction Educational Programs (LIEPs): A Review of the Foundational Literature (2012) describes LIEP characteristics that may influence the quality of programs delivered to ELs in grades K through 12. The research reviewed for this study suggests that ELs who receive some kind of language support or specialized instruction show better outcomes on various academic measures than those who receive no special support. While multiple meta-analyses and large-scale research studies have found that models following the bilingual approach can produce better outcomes than ESL models, as measured by general academic content assessments or measures of reading comprehension or skills, other studies indicate that the quality of instructional practices matter as well as the language of instruction. Researchers also found examples of high-quality programs that come from both bilingual and ESL approaches which suggests that no single approach (e.g., ESL or bilingual) is effective at all times and under all circumstances.
National Evaluation of Title III Implementation Supplemental Report: Exploring Approaches to Setting English Language Proficiency (ELP) Performance Criteria and Monitoring English Learner Progress (2012). This report offers several empirical methods that state policy-making authorities can use as part of a larger deliberative process for setting English Language Proficiency (ELP) performance standards and operationalizing ELP assessment and accountability criteria. The approaches presented in the report are intended to stimulate discussion and further exploration of additional methods among state data analysts, technical assistance providers, and researchers.
Key findings: Potential methods for analyzing empirical data in order to assist policymakers in determining an empirically-based ELP standard for English learners (ELs) include: (1) decision consistency analysis; (2) logistic regression analysis; and (3) descriptive box plot analysis. Taken together, these approaches provide multiple sources of evidence for investigating and corroborating the point at which an ELP performance standard might be set. Approaches for establishing a target time frame for ELs to attain a pre-identified ELP performance standard include: (1) descriptive longitudinal analysis that follows EL students who start at a pre-specified date at varying English-proficiency levels; and (2) event history analysis that estimates the time required for a typical EL student to attain a given ELP performance standard. Approaches for taking a student's ELP level into account when setting academic progress and proficiency expectations include: (1) progressive benchmarking method; (2) indexed progress method; and (3) status and growth accountability matrix (SGAM) method.
Title III Policy: State of the States (2010) discusses state implementation of the Title III accountability requirements based on phone interviews with six state Title III Directors in the spring of 2009, interviews with six experts and university-based researchers who work on education for English Learners, and based on earlier data collected in 2004 through 2007 under the Study of State Implementation of Accountability and Teacher Quality under No Child Left Behind and under the National Longitudinal Study of No Child Left Behind.
Key findings discussed in the brief are as follows:
- All states had established ELP standards by the 2006-07 school year, and all states had established ELP assessments by the 2007-08 school year. While many states missed the earlier deadlines established by ESEA and the U.S. Department of Education, this may be attributable to the difficulty and/or newness of the tasks involved. Very few states had English language proficiency standards or assessments before the 2001 ESEA amendments required them.
- Many states reported difficulties developing a cut score for AMAO 1 and AMAO 2 because of their lack of experience with this type of testing. For example, establishing an English language proficiency growth target, which is required for AMAO 1, is difficult without historical growth data. In 2006-07, only 12 state directors reported that their states had finalized AMAO targets. More than half were in the process of revising their AMAOs, a process that is still continuing in some states, based on the ELP testing data they were beginning to collect through the annual assessments.
- In 2006-07, 30 states were applying accountability actions to districts that had not met their AMAO targets for consecutive years. Eighteen of these 30 states were using preliminary AMAO targets, since only 12 states had finalized their AMAOs by 2006-07. In addition, due to delays in the development of ELP standards, assessments, and AMAOs, some states refrained from imposing consequences.
Title III Accountability: Behind the Numbers (2010) summarizes data reported by states in their Consolidated State Performance Reports (CSPRs) for 2004-05 through 2007-08. The CSPRs are annual reports required under ESEA that states use to submit information to the U.S. Department of Education about their activities and outcomes related to specific ESEA programs. The CSPR data reflect states' direct reports as of March 2009 and have not been validated by the U.S. Department of Education or other external parties.
Key findings from the brief are as follows:
- Because states have used a wide variety of definitions and other features to set their AMAO targets, it is very difficult to compare districts across states. Additionally, these definitional variations, along with differences in the ELP assessments themselves, make the targets easier to reach in some states than in others.
- AMAO 2: States also use a variety of approaches in determining whether ELs are "attaining English language proficiency." For example, some states include all EL students in applying their AMAO 2 target. Others include only the subset of EL students who were identified as likely to achieve proficiency within the reporting period, an approach ED now prohibits beginning with the 2009-10 reporting year. Additionally, even states using the same ELP assessment establish different cut scores for determining proficiency.
- Analysis of districts in three states selected for this brief indicates that half the districts that were designated for Title III improvement in 2007-08 also were designated for improvement under Title I in that year. However, this analysis also indicates that many districts that missed their AMAOs under Title III were still able to make their Title I AYP targets. These districts tended to be disproportionately small and rural.
Title III Accountability and District Improvement Efforts: A Closer Look (2010) summarizes finding s from interviews with six Title III Directors and nine Title III district-level directors in the spring of 2009. States and districts were selected in order to collect information from some entities with a long history of serving English Learners (ELs) as well as from entities that have experienced a rapid increase in EL enrollments in recent years. Findings from this small sample cannot be generalized to all states or districts.
Key findings from the brief are as follows:
- Each of the five states interviewed for the study has one or more districts that have not met their AMAOs for 2 years or more. These states require these districts to develop an improvement plan, as required by Title III. These states also are requiring districts that miss their AMAOs for even 1 year to submit a notice of the failure to their EL parents. The states are providing sample parent letters to districts for this purpose.
- The six states vary in the technical assistance systems they have put in place to help districts achieve their AMAO targets. California and New York have established regional centers that regularly provide supports in the nature of improvement plan suggestions and professional development.
- In all six states interviewed, the majority of the EL population is enrolled in Title I schools. All of the state officials surveyed indicated that coordination of services between the Title I and Title III offices has increased in recent years.
- Challenges district officials reported include the shortage of teachers with special EL training, the difficulty in making schools feel accountable for their EL students' achievement when the AMAOs only apply at the district level, and inadequate funding.
Highly Qualified Teachers Enrolled in Programs Providing Alternative Routes to Teacher Certification or Licensure (2015) summarizes state- and district-level data on the numbers of full-time equivalent (FTE) highly qualified teachers who were enrolled in alternative route programs for three groups of teachers(1) all teachers, (2) special education teachers, and (3) teachers in language instruction educational programs for English learners (ELs) under Title III of the Elementary and Secondary Education Act of 1965 (ESEA)as well as for teachers in high-poverty and rural school districts.
- Key findings include: Highly qualified teachers who were enrolled in alternative route teacher preparation programs made up a small proportion of total HQTs, both overall as well as for the four subgroups of HQTs examined in this report. Overall, across 48 states and the District of Columbia, 1.5 percent of HQTs were enrolled in alternative route programs. The average percentage of HQTs enrolled in alternative route programs was 1.9 percent for special education teachers, 1.3 percent for Title III teachers, 2.3 percent for high‑poverty school districts, and 1.3 percent for rural school districts. High-poverty school districts had higher percentages of HQTs in alternative route programs, on average, than districts with low poverty rates. Rural school districts had a lower percentage of HQTs in alternative route programs than urban districts and a proportion that was close to the proportions in town and suburban districts.
- Report: PDF (897KB)
- Results in Brief: PDF (143KB)
- Dataset on Highly Qualified Teachers Enrolled in Alternative Route Programs: Zipped (1.11MB)
Providing Effective Teachers for All Students: Examples from Five Districts (2012). This report is based on case studies of five school districts that use data on student achievement growth to identify effective teachers, implement performance pay initiatives or other human resource policies, and seek to ensure an equitable distribution of effective teachers, particularly in high-need schools. Study methods included on-site interviews with district-level staff, teachers' association or union representatives, and principals, as well as analysis of district documents and materials.
- All five districts used student achievement growth as one measure of teacher effectiveness for some or all teachers. In addition, four districts used new or revised observation‑based assessments in conjunction with achievement growth, or were in the process of developing them.
- All five districts used their measures of teacher effectiveness in some human resource policies. For example, four districts used effectiveness information in performance pay initiatives.
- Three of the five districts had policies that targeted high‑need schools, drawing on effectiveness information.
- All three offered financial incentives to teachers to move to or stay in high‑need schools.
- One district had hiring and transfer policies designed to provide principals in high‑need schools additional opportunities to hire effective teachers.
- The five districts' efforts suggest a number of key challenges that other districts and states may need to address as they consider using measures of teacher effectiveness. For example, interviewees noted challenges in implementing classroom observation systems that were both rigorous and manageable in terms of scheduling complexity and time required.
Teacher Incentive Fund: First Implementation Report: 2006 and 2007 Grantees (2012). This study describes several aspects of the implementation of the first two cohorts of Teacher Incentive Fund (TIF) grantees. It highlights the main characteristics or components of TIF performance pay plans in terms of strategies, targets and size of award. It also discusses system supports (planning, communication, program and funding stability, data systems, and relationship to other human resource policies) and broader contextual factors (e.g., stakeholder satisfaction) that impede or enhance implementation of performance pay systems. Finally it indicates what prospects are for sustainability of the programs beyond the life of the grant.
Key findings include: Eligible teachers and administrators participated in the TIF projects at high rates; only four teacher projects had participation rates below 90 percent. Two-thirds of grantees provided awards for student performance at the group or individual teacher level. Educators reported that TIF promoted collaboration more often than it encouraged negative competition. In the majority of projects, almost all teachers and administrators received awards: during the 2008-09 school year, grantees paid approximately $70 million in incentive awards to more than 20,000 educators. Eleven principal projects and seven teacher projects paid out less than two percent of an average salary, while 11 principal projects and 16 teacher projects paid six percent of salary on average. Grantees developed a complementary blend of activities, supports and rewards designed to improve educator practice. Project implementation across TIF grantees was affected by several factors. Educators in TIF projects had concerns about fairness of their performance pay systems. For many grantees, communication about the program was a very difficult part of implementation. Project leaders asserted that educators paid little attention to attempts to communicate about the project before they received their first incentive award payments; at that point, if educators received a lower amount than they had anticipated, they sought communication from the grantee but only after they questioned the fairness of their performance award. Performance pay projects had inherent financial challenges. Grantees exhibited a range of models of sustainability plans. None included immediate pay cuts for educators (i.e., cuts in basic pay so that more money is left over for compensation-based supplements), and all offered increased financial compensation for award recipients.
Recent Trends in Mean Scores and Characteristics of Test-Takers on Praxis II Licensure Tests (2010). This study examines changes in teacher licensure scores from 1999 to 2006. The study focuses specifically on nine tests in the Praxis II series because these tests are among the most widely used assessments across multiple states for purposes of measuring content knowledge for initial teacher licensure. The purpose of this study is to identify trends in Praxis scores on a select number of tests across recent years and across as many states as possible. The study focuses on trends in mean scores for those who pass the Praxis II tests, as these are individuals who are eligible to enter teaching. Analyses are disaggregated by passing status, gender, race, and whether or not the test candidate has prior teaching experience.Key findings are as follows:
- For the Praxis tests examined in this study, there is little change in mean test scores observed over time. The study identifies many significant trend effects, but the magnitude of these effects is relatively small. Because the samples include thousands of individuals, even small differences can be statistically significant, though substantively unimportant.
- Those who pass Praxis tests have scores substantially higher than those who do not. Individuals who pass these Praxis II tests have mean and median scores that are approximately two standard deviations higher than those who fail. Standard deviations are based on all test-takers from the respective samples. This pattern is consistent across all tests examined in this study.
- There has been an increase in the number of individuals taking the Praxis II tests and the increase is seen both among individuals with teaching experience and without teaching experience. For many of the tests included in the analyses, the size of the sample of test-takers nearly doubled over the different range of years included. At the same time that a substantial increase in test-takers was occurring, there was very little movement in scores. There is no clear rationale for why this pattern of data exists. It appears that the increase in candidates has not materially affected the overall preparation of individuals to succeed on the respective tests.
- The large, positive trends in SAT scores by Praxis II candidates observed in prior research are not echoed here in similarly large positive trends in Praxis II scores. This may be due to the more limited number of years included in this study as well as several other factors. Gitomer (2007), primarily on the basis of SAT scores, concluded that the academic quality of teachers had improved a substantial amount over recent years. This begs the question of whether there are reasons to explain the current study’s more modest findings compared with the prior study. As explained in the report, there are a number of critical differences between these studies that might account for the somewhat different findings.
State and Local Implementation of the No Child Left Behind Act, Volume VIII-Teacher Quality Under NCLB: Final Report (2009) provides updated information on the progress that states, districts, and schools have made in implementing NCLB's teacher quality, professional development, and paraprofessional provisions. The report is based on the second round of data collection from the National Longitudinal Study of NCLB and the Study of State Implementation of Accountability and Teacher Quality Under NCLB. The report presents findings from interviews with state education officials in all states and surveys of nationally representative samples of districts, principals, and teachers conducted in 2004-05 and 2006-07.
Key findings include: By 2006-07, the vast majority of teachers met their states' requirements to be considered highly qualified under NCLB. However, state requirements for the demonstration of content-knowledge expertise varied greatly. Teachers in high-poverty and high-minority schools were more likely to report that they were not highly qualified. Moreover, even among teachers who were considered highly qualified, teachers in high-poverty schools had less experience and were less likely to have a degree in the subject they taught. Although nearly all teachers reported taking part in content-focused professional development related to teaching reading or mathematics during the 2005-06 school year and summer, a relatively small proportion participated in such learning opportunities for an extended period of time.
State and Local Implementation of the No Child Left Behind Act, Volume II-Teacher Quality under NCLB: Interim Report (2007) draws on data from the 2004-05 data collection cycles of two federally funded studies -- the Study of State Implementation of Accountability and Teacher Quality Under NCLB (SSI-NCLB) and the National Longitudinal Study of NCLB (NLS-NCLB) -- to describe patterns in state, district, and school implementation of Title I and Title II provisions concerning teacher quality, professional development, and paraprofessionals.
Transition to Teaching Program Evaluation: An Interim Report on the FY 2002 Grantees (2007) presents the results of the interim evaluation of the program with data from the FY 2002 grantees in the third year of five-year grants. Data were collected from November 2004-February 2006 through an online Annual Performance Report in which grantees provided project-level characteristics and outcomes, eight case studies of grantees, a participant survey of teachers of record, and interim evaluations submitted by grantees.
The study found that 59 percent of program FY 2002 participants were midcareer professionals, 27 percent were recent college graduates, and 14 percent were paraprofessionals. TTT teachers were significantly more racially and ethnically diverse than public school teachers as a whole. TTT teachers were generally placed in subjects in which schools have the greatest need for teachers, including math, science, and special education.
The Teaching American History Evaluation: Final Report (2011) provides findings on a study of the Teaching American History (TAH) program, which provides funds to local school districts for developing and operating three-year professional development projects to improve instruction in this subject. The study found that state history assessment data were too limited to be used to conduct analyses of TAH effects. In addition, the study’s analyses of TAH grantee evaluations found that the evaluations were not sufficiently rigorous to determine the impact of the TAH program on student achievement or teacher knowledge. Case studies found the following: most grantees struggled to recruit teachers most in need of improvement; TAH professional development generally balanced the delivery of content knowledge with strengthening of teachers’ pedagogical skills; and partnerships gave teachers access to organizations rich in historical resources and expertise, and were flexible enough to adapt to the needs of the teachers they served.
The Evaluation of the Teaching American History (TAH) Program (2005) examines the implementation of this professional development program and the characteristics of the activities, content, and teacher participants for projects awarded during the first two years of the program. Findings are based on: surveys of program participants and project directors; case studies of projects; analyses of program documents; and lesson plans produced by program participants.
The study found that summer institutes were the most frequently offered professional development activity. The teachers who participated in TAH activities were often not those traditionally thought of as most in need of history professional development. TAH participants were most likely to be experienced secondary teachers (70 percent) with academic backgrounds in history.
Improving Teacher Quality in U.S. School Districts (2004) provides descriptive data from a nationally representative sample of school districts showing how they used their Title II, Part A funds during the 2002-2003 school year. The brief provides data on how districts distributed their funds among the variety of allowable Title II activities, and it disaggregates the data by district size and poverty level.
Third National Even Start Evaluation: Program Impacts and Implications for Improvement (2003) provides information from the third national evaluation of the Even Start Family Literacy Program. The third national evaluation includes two complementary studies: (1) the Even Start Performance Information Reporting System (ESPIRS) and (2) the Experimental Design Study (EDS). The ESPIRS provided annual data from 1997-1998 through 2000-2001 on the universe of Even Start projects. The EDS is an impact study that used an experimental design; families in the study were randomly assigned either to participate in Even Start or to be in a control group. This report presents descriptive information on all Even Start programs and participants based on all four years of ESPIRS data collection, and discusses program impacts based on pretest and posttest data collected from the 18 EDS projects.
- Complete Report website
State Administration of the Even Start Family Literacy Program (2003) is based on a survey of state Even Start coordinators during the 2001-02 school year, as well as case studies in 12 states. The report describes Even Start administration at the state level and factors that facilitate or impede program improvement activities conducted by state coordinators. The report is intended to: 1) help federal staff better target their guidance and technical assistance to states; and 2) provide state coordinators with descriptions of program administration practices in other states as a self-assessment guide.
Migrant Education Program Annual Report: Eligibility, Participation, Services (2001-02) and Achievement (2002-03) (2006) provides information from the Consolidated State Performance Report (CSPR) about migrant children and youths who were eligible and who participated in MEP-funded services during 2001-02. and discusses the academic achievement of migrant students in 2002-03.
Title I Migrant Education Program Trends Summary Report: 1998-2001 (2004) summarizes the participation information provided by state education agencies (SEAs) on the Migrant Education Program for the school years 1998-99 through 2000-01.
State Title I Migrant Participation Information: 1999-2000 (2004) is an annual report that summarizes the participation information provided by the state education agencies (SEAs) on the Migrant Education Program for a given school year.
Volume I, Title I Schools Serving Migrant Students: Recent Evidence From The National Longitudinal Survey of Schools examines whether and how Title I schools that serve migrant students are implementing the provisions of Title I, and to describe the characteristics of and conditions in schools serving migrant children during the 1998-99 school year.
PDF (588K) | Word (1.71MB)
Volume II, Measurement of Migrant Student Educational Achievement investigates the extent to which migrant students participate in state and local assessment and accountability programs, and the types and quality of academic outcome data on migrant students collected and maintained by state and local educational agencies.
PDF (577K) | Word (1.05MB)
Volume III, Coordinating the Education of Migrant Students: Lessons Learned from the Field examines promising practices in migrant education programs. Four groups of two or three districts that share students who move back and forth between them were chosen for study.
PDF (499K) | Word (506K)
A Snapshot of Title I Schools Serving Migrant Students, 2000-01 (2003) provides information on implementation of the Title I program in schools serving migrant students. It compares Title I schools with migrant students to all Title I schools in the areas of social, demographic, and organizational characteristics and the implementation of Title I provisions between 1998-99 and 2000-01.
Education for Homeless Children
State and District Implementation of the Education for Homeless Children and Youth Program (2015). This report examines how states and districts were implementing the EHCY program during the 2010–11 school year, based on surveys of state EHCY coordinators and district homeless liaisons as well as analysis of extant data. Topics include how states allocate EHCY funds, the roles and responsibilities of the state coordinators and district liaisons, types of services provided, technical assistance, data collected by states and districts, and potential barriers to school success for homeless children and youth.
The Education for Homeless Children and Youth Program: Learning to Succeed (2002) provides evidence that states and school districts have made significant progress in revising laws, policies, regulations' and practices that have served as barriers to the enrollment, attendance, and school achievement of homeless students.
Volume I: Reducing Barriers for Homeless Children and Youth for Access and Achievement examines state and local efforts to serve the educational needs of homeless children and youth, and to overcome barriers that affect these students' enrollment, attendance, and school success.
PDF (879K) | Word (352K)
Volume II: Educating Homeless Children and Youth: A Resource Guide to Promising Practices suggests strategies and processes that states, districts, and schools can use to overcome some of the many barriers that keep homeless children and youth from getting the education to which they are entitled. It also presents approaches for helping them to achieve the same high standards expected of all children. The promising practices the guide describes all come from states and districts that have placed a strong emphasis on enrolling homeless children and youth in school and helping them to be successful students.
PDF (1.24MB) | Word (501K)
Safe and Healthy Students
Evaluation of the Carol M. White Physical Education Program: Final Report (2015) provides information on the implementation of Fiscal Year 2010 PEP grants. The report focuses on PEP grantees' experiences with self-assessments, policy efforts, programmatic activities, community partnership, and Body Mass Index data. The report uses information from two surveys (conducted in years one and three of the grant program) and five case studies (conducted in year three).
The Carol M. White Physical Education Program Project Implementation During Year 1 of the Grant (2012). The primary focus of PEP is to develop high-quality physical education programs, create an environment supportive of physical activity, and encourage healthy eating habits and good nutrition. The Department revised PEP in FY 2010 to include an absolute priority that required PEP projects to address the relevant state's physical education standards and develop, expand, or improve its physical education program by including instruction in healthy eating habits and good nutrition. PEP's new direction also included establishing two competitive priorities, awarding additional points to projects proposing one or both of the following: partnerships with community entities and the collection and use of body mass index (BMI) measurement. This brief provides information on the implementation of PEP.
Key findings include:
- All but one project formed community partnerships, indicating the success of the revised PEP's emphasis on promoting collaboration.
- All but three grantees either collected or planned to collect BMI data.
- Frequently reported PEP project implementation challenges included issues with Government Performance and Results Act (GPRA) data collection and a lack of time to prepare for the start of the PEP grant following the award notification.
- The most common reported challenges with GPRA data collection included the lack of proper reporting by youths, loss or theft of equipment, and failure to return information.
- Evaluation Brief: PDF (400K)
Analysis of State Bullying Laws and Policies (2011) reviews states' bullying laws and model bullying policies and school districts' bullying policies. The report uses the U.S. Department of Education's guidance document, "Anti-Bullying Policies: Examples of Provisions in State Laws," as an organizing framework for the review.
Key findings include: As of April 2011, 46 states had bullying laws, 45 of which directed school districts to adopt bullying policies. Forty-one states had model bullying policies. Thirty-six states included provisions in their education codes prohibiting cyberbullying or bullying using electronic media. Thirteen states specified that schools have jurisdiction over off-campus behavior if it creates a hostile school environment.
Prevalence and Implementation Fidelity of Research-Based Prevention Programs in Public Schools (2011): This report presents findings on key program implementation measures for the Safe and Drug-Free Schools and Communities Act (SDFSCA): State Grants Program. Funding for the State Grants Program was eliminated in Fiscal Year 2010. Prior to this, grants were awarded to States to support a variety of drug and violence prevention activities for school-age youths. The study examines: (1) the prevalence of research-based drug and violence prevention programs in schools and (2) the extent to which research-based drug and violence prevention programs adhere to the program features on which they are based (or the program's implementation fidelity). Findings are based on a review of the research literature and national probability sample surveys of districts, schools and research-based prevention programs.
Key findings include: The study identified 19 prevention programs with research-based evidence that suggested an improvement in student behavior. Of all prevention programs that were implemented in the nation's public schools, an estimated 8 percent of them were research-based (e.g., the 19 research-based prevention programs represented 8 percent of all prevention programs implemented in schools). Nearly half of the research-based curriculum programs (44 percent) provided in schools met minimum standards for overall fidelity of implementation; that is, these programs passed on all four program-specific measures of implementation fidelity.
- Full Report: PDF (1.5M)
Educator Sexual Misconduct: A Synthesis of Existing Literature (2004) is a report requested by Congress. It examines the incidence and prevalence of abuse, patterns of misconduct, and prevention strategies, among other items.
Wide Scope, Questionable Quality: Three Reports from the Study on School Violence and Prevention (2000) investigates the extent of problem behavior in schools nationally and several aspects of delinquency prevention efforts in schools, such as the types and quality of prevention efforts, how schools plan and use information about prevention options to improve their own efforts and school management, and sources of funding for school prevention activities.
Wide Scope, Questionable Quality: Drug and Violence Prevention Efforts in American Schools presents findings from surveys of a national sample of elementary, middle, and high schools, including surveys of school principals and prevention activity providers, and, in the middle and high schools, of teachers and students, along with surveys of district Safe and Drug-Free Schools program coordinators.
PDF (911K) | Word (910K)
A Closer Look at Drug and Violence Prevention Efforts in American Schools presents case studies of 40 schools (20 middle schools and 20 high schools) included in the national survey.
PDF (837K) | Word (436K)
School Crime Patterns: A National Profile of U.S. Public Schools Using Rates of Crime Reported to Police examines data from a previous NCES survey asking principals about the number and types of crimes they report to police. New analysis focuses on high schools, profiling schools with high and low levels of reported crime.
PDF (617K) | Word (311K)
Evaluation of Magnet Schools Assistance Program: 1998 Grantees: Year 1 Interim Report (2001) focuses on the 57 districts funded by the Magnet Schools Assistance Program (MSAP) for the school years 1998-99, 1999-2000, and 2000-01, and on the 292 magnet schools supported by MSAP. The first report of a four-year evaluation, it provides descriptive information about the MSAP-supported districts and schools: their desegregation and achievement objectives, the systemic reforms they are designed to support, and the innovative methods and practices they are implementing. The final report will present data on the extent to which the desegregation and achievement objectives have been met and will include case studies of eight of the MSAP projects.
- Report website including highlights, executive summary and the complete report.
Evaluation of the Magnet School Assistance Program (MSAP): 1998 Grantees (2003), the final report for this study examines the progress of MSAP projects in achieving four legislative purposes of the program. Particular attention is given to program outcomes in reducing minority student isolation and improving student achievement.
- Executive Summary
- Main Report: PDF (1MB) | Word (1.9MB)
- Case Studies Appendix: PDF (1.2MB) | Word (1.3MB)
Implementation of the Credit Enhancement for Charter School Facilities Program: Final Report (2008). The Credit Enhancement for Charter School Facilities Program was established in 2001 to address a critical problem faced by many charter schools - lack of suitable facilities and difficulty obtaining financing to secure suitable facilities. The program makes competitive grants to eligible public entities that provide credit enhancements to absorb some of the risk in making facilities loans to charter schools. This study examines, broadly, how the program was implemented by the nine organizations that received grants in FY 2002 - FY 2004, and specifically, whether the program is providing: (1) improved access of charter schools to capital markets, (2) better financing rates and terms than the schools otherwise could obtain, and (3) assistance to charter schools that are serving students with the greatest need for school choice. The study draws on information from grantee applications and annual performance reports; several secondary data sources; and discussions with samples of grantees, lenders, and schools assisted by grantees.
Results indicated that credit enhancements provided by the program facilitated a total of $168 million in loans to 84 schools that served over 23,000 students. Students in these schools were more likely to be low-income and minority than students enrolled in all charter schools and in all U.S. public schools. Many of the assisted schools could not have received facility loans at any price without the program because lenders believed that these schools reflected a prohibitively high level of risk. With the credit enhancements provided by grantees, assisted charter schools received loans with rates and terms that were better than otherwise would have been available to them.
Evaluation of Public Charter Schools Program: Final Evaluation Report (2004). The final report examines the operations of the Public Charter Schools Program (PCSP) in supporting continued growth and development of the charter school sector in American public education. In addition, it provides information on the characteristics of the sector as of 2001-02, with particular attention to the nature of charter school accountability.
Evaluation of Public Charter Schools Program: Year One Evaluation Report (2000) uses Year 1 data to paint a comprehensive picture of a number of issues: the development of the Public Charter Schools Program, state and charter school authorizer perspectives on charter school flexibility and accountability, and the charter school activities of states and a sample of charter school authorizers. This picture, however, is also a "snapshot" of a rapidly evolving movement during a narrow time interval (summer and fall 1999).
- Report website including the executive summary and the complete report.
The U.S.-China E-Language Project: A Study of a Gaming Approach to English Language Learning for Middle School Students (2011). In 2001, the U.S. Department of Education and the Ministry of Education in China entered into a bilateral partnership to develop a technology-driven approach to foreign language learning that integrates gaming, immersion, voice recognition, problem-based learning tasks, and other features that made it a significant research and development pilot project for study. The purpose of this report is to describe the evaluation of a key outcome of this bilateral partnership, The Forgotten World. This program was implemented as a supplementary activity in middle school classrooms in western China to teach the English language and American culture to eighth-grade students. The evaluation was conducted in five treatment schools and five comparison schools during the 2009-10 school year and included approximately 3,500 students. The evaluation showed statistically significant positive results of using The Forgotten World for the lower performing students along with the positive effects on student motivation. Almost all of the teachers in the treatment schools (95 percent) who participated in the project reported that their use of The Forgotten World changed the way they think about teaching.
Teachers' Ability to Use Data to Inform Instruction: Challenges and Supports (2011). This report describes an exploratory study on teachers' thinking about data and the implications of the study's findings for teacher preparation and support. Understanding the nature of teachers' proficiencies and difficulties in data use is important for providing appropriate training and support to teachers because they are expected to use student data as a basis for improving the effectiveness of their practice.
Key findings include:
- Data Location. Teachers in case study schools generally were adept at finding information shown explicitly in a table or graph.
- Data Comprehension. A majority of case study teachers demonstrated reasonable skill in comparing data in a table or graph to corresponding prose characterizations. Common, however, were difficulties in evaluating written statements about data that required basic math calculations, distinguishing a histogram from a bar graph, and considering the difference between cross-sectional and longitudinal data sets. This finding suggests that teachers may come away from presentations of school or district data with misconceptions about their students' performance.
- Data Interpretation. Case study teachers were more likely to examine score distributions and to think about the potential effect of extremely high or low scores on a group average when shown individual students' scores on a class roster than when looking at tables or graphs showing averages for a grade, school, or district. An implication of this finding is that teachers will need more support when they are expected to make sense of summaries of larger data sets as part of a grade-level, school, or district improvement team.
- Data Use for Instructional Decision Making. Many case study teachers expressed a desire to see assessment results at the level of subscales (groups of test items) related to specific standards and at the level of individual items in order to tailor their instruction. After years of increased emphasis on accountability, these teachers appeared quite sensitive to the fact that students will do better on a test if they have received instruction on the covered content and had their learning assessed in the same way (e.g., same item format) in the past.
- Question Posing. In order to use an electronic data system to identify areas for improvement, educators need to be able to frame questions that can be addressed by the data in the system. Most case study teachers struggled when trying to pose questions relevant to improving achievement that could be investigated using the data in a typical electronic system. They were more likely to frame questions around student demographic variables (e.g., "Did girls have higher reading achievement scores than boys?" than around school variables (e.g., "Do student achievement scores vary for different teachers?").
Use of Education Data at the Local Level: From Accountability to Instructional Improvement (2010). The national Study of Education Data Systems and Decision Making examined both the implementation of student data systems and the broader set of practices involving the use of data to improve instruction, regardless of whether or not the data were accessed through an electronic system. Earlier study reports have documented a dramatic increase in the proportion of teachers with access to a student data system between 2005 and 2007 and described school practices with respect to data use and the challenges that are part of student data system implementation. This final report builds on the picture of local practices in implementing data-driven decision making provided in the earlier reports by presenting data from the national district survey as well as from site visits conducted during 2007–08 to 36 schools in 12 districts.
Key findings include:
- Data-driven decision making is an ongoing process rather than a one-time event centered on the acquisition of a data system. Districts will get more out of their investments in electronic data systems if they think about data-driven decision making as a system-wide innovation and develop a long-term strategy for its implementation as part of a continuous improvement process.
- To influence teachers' day-to-day instruction, data systems must provide teachers with information that is both timely and relevant to their instructional decisions. To be useful to teachers, systems need to provide data from recently given assessments that provide diagnostic information on students' learning needs
- Human and organizational supports for data use are just as important as the technical quality of the data system. Professional development around data use is widespread, but only a small minority of districts and schools have made data use a regular part of teachers' practice.
- Districts can promote data-driven decision making in schools by providing time for teachers to meet with colleagues to discuss and use data, funding positions for instructional coaches who help teachers connect data to alternative instructional approaches, and by modeling data-driven decision making for continuous improvement in their own operations.
- Districts' greatest perceived area of need with respect to data-driven decision making is for models of how to connect student data to instructional practices. Among teachers, there is a need to enhance their assessment interpretation and data use skills.
Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies (2009). A systematic search of the research literature from 1996 through July 2008 identified more than a thousand empirical studies of online learning. Analysts screened these studies to find those that (a) contrasted an online to a face-to-face condition, (b) measured student learning outcomes, (c) used a rigorous research design, and (d) provided adequate information to calculate an effect size. As a result of this screening, 51 independent effects were identified that could be subjected to meta-analysis. Key findings include:
- Students who took all or part of their class online performed better, on average, than those taking the same course through traditional face-to-face instruction. Learning outcomes for students who engaged in online learning exceeded those of students receiving face-to-face instruction, with an average effect size of +0.24 favoring online conditions. The mean difference between online and face-to-face conditions across the 51 contrasts is statistically significant at the p < .01 level.
- Instruction combining online and face-to-face elements had a larger advantage relative to purely face-to-face instruction than did purely online instruction. The mean effect size in studies comparing blended with face-to-face instruction was +0.35, p < .001. This effect size is larger than that for studies comparing purely online and purely face-to-face conditions, which had an average effect size of +0.14, p < .05.
- Few rigorous research studies of the effectiveness of online learning for K–12 students have been published. The systematic search of the research literature found just five experimental or controlled quasi-experimental studies comparing the learning effects of online versus face-to-face instruction for K-12 students. As such, caution is required in generalizing to the K-12 population because the results are for the most part based on studies in other settings (e.g., medical training, higher education).
Evaluation of the Enhancing Education Through Technology Program: Final Report (2009). This study collected information about educational technology practices related to the Enhancing Education Through Technology (EETT) program. The report is structured around the EETT program objectives and specific performance measures developed by the Department to meet the requirements of the Government Performance and Results Act of 1993. The program objectives and performance measures focus on teachers' and students' access to technology, technology-related professional development, technology integration, and student technology literacy. The study collected data from nationally representative samples of states, districts and teachers between school years 2002-03 and 2006-07.
- High-speed Internet access in K-12 classrooms. Overall, 63 percent of teachers reported in 2006-07 that students had high-speed Internet access in their classrooms. There were no statistically significant differences in reported classroom access to high-speed Internet for students between teachers in high-poverty schools and those in low-poverty schools in either 2004-05 or 2006-07. However, seventy-two percent of teachers in elementary grades, compared with 55 percent in middle school grades and 49 percent in high school grades, reported having high-speed Internet access within their classrooms. Differences in subject taught and in school location (rural, suburban, urban) were not significant predictors of classroom Internet access.
- Quality of technology-related professional development. Seven practices often cited as elements of best practice for technology-related teacher professional development were identified through review of the literature. When asked to describe their most useful technology-related professional development experience in 2006-07, 20 percent of teachers indicated that this professional development did not include any of the seven research-suggested characteristics.
- Technology integration. The GPRA measure for technology integration is "the percentage of districts receiving Educational Technology State Grants funds [EETT funds] that have effectively and fully integrated technology." As reported on the 2007 state survey, most states either had not adopted a definition of effective integration of technology or did not measure the percentage of districts meeting the statewide definition.
- Assessing student technology literacy. One of the GPRA measures for the EETT program is "the percentage of students who meet state technology standards by the end of the eighth grade." Forty-four states had either stand-alone technology standards for students or technology standards that were integrated into other student academic standards. Six states reported conducting statewide assessments of student technology literacy in 2005-06, up from just two states in 2002-03. Twenty-five states reported relying on districts to assess student technology literacy.
Implementing Data-Informed Decision Making in Schools: Teacher Access, Supports and Use (2009) describes the student data systems available to school staff members, how school staff members are using the systems and other forms of student data, teachers' understanding of data displays and data interpretation issues, and the supports and challenges for school-level use of student data in planning and implementing instruction. This report draws on case study findings in nine purposively sampled districts, nominated on the basis of the strength of their data use activities. Researchers interviewed district staff members as well as principals and teachers from three schools within each district. In addition, a set of scenarios involving hypothetical student data were presented to teachers at each school to probe their understanding of student data. In addition to case study data, this report also draws on data from secondary sources (spring 2007 district and teacher surveys from the U.S. Department of Education's National Educational Technology Trends Study).
Key findings: Data from student data systems are being used in school improvement efforts but are having little effect on teachers' daily instructional decisions. Typically, the information needed for instructional decision making was spread across multiple systems without mechanisms for regular transport of information from one system to another. As a result, neither teachers nor administrators see a comprehensive record of students' educational experiences and performances that is both longitudinal and up to date. In addition, only a minority of data systems incorporate resources such as instructional materials, model lesson plans, and formative assessment results linked to frameworks and curriculum guides.
- Full Report: Word(4.3 MB)
Teachers' Use of Student Data Systems to Improve Instruction (2007) provides the first national estimates of the prevalence of K-12 teachers' access to and use of student data systems. This paper was developed through a secondary analysis of national survey data from over 6000 teachers and over 1000 district technology coordinators conducted in 2005 as part of the National Educational Technology Trends Study. Evaluation questions include: (1) How broadly are student data systems being implemented in districts and schools?, (2) Within these systems, how prevalent are tools for generating and acting on data?, and (3) How are school staff using student data systems?
Teachers' Use of Student Data Systems to Improve Instruction: 2005 to 2007 (2008). This issue brief is the second in a two-part series examining teachers' access to and use of data from student data systems. This brief was developed through a secondary analysis of data from teachers and district technology coordinators surveyed at two points in time from 2005 to 2007 as part of the National Educational Technology Trends Study.
Key findings include: The percent of teachers reporting access to an electronic student data system grew significantly between 2005 and 2007, rising from 48 percent in 2005 to 74 percent in 2007. Even though 74 percent of all teachers reported having access to student data systems in 2007, the proportion of teachers with data system access who also have tools for making instructional decisions informed by data remains below 20 percent. The majority of teachers continue to use these systems to provide information to parents (68 percent), and track individual student test scores and monitor student progress (65 percent, respectively).
National Educational Technology Trends Study State Strategies Report: Vol. 1 (2007). The Enhancing Education Through Technology program (EETT) is among the largest federal programs seeking to improve student achievement through the use of technology. This report examines the state priorities and programs that EETT supports and the relationship between state educational technology program activities and the overarching goals and purposes of the No Child Left Behind Act of 2001. Chapter 1 of this report describes state educational technology policies and related programs, including the role of the EETT program in state efforts. Chapter 2 presents individual state profiles that present data summarized in Chapter 1.
Key findings include: Of the 42 states that had student technology standards in place by the fall of 2004, 18 reported having "stand-alone" standards, and 16 reported embedding technology standards with other academic content standards. The remaining eight states reported having both stand-alone technology standards and integrated standards. Two states reported that they used statewide assessments of students' proficiency with technology. In addition to student technology standards, more than half of states (27) reported that they had technology standards for teachers in order to specify the knowledge and skills that teachers need to use technology for administrative or instructional purposes. Five states formally assessed teachers' technology skills at the state level.
National Educational Technology Trends Study State Strategies Report: Vol. 2 (2007). Using data collected for the 2005 National Assessment of Educational Progress (NAEP), this report examines educational technology access and use in fourth- and eighth-grade mathematics classrooms all across the country. Chapter 1 provides national data and documents differences between states in teachers' and students' access to instructional technology, in teachers' efforts to integrate technology in mathematics instruction and assessment, in students' use of technology in mathematics learning, and in the technology-related development and support that states provide to teachers. Chapter 2 reports similar information on a state-by- basis.
Key findings include: Only about 10 percent of fourth- and eighth-graders were in classrooms in which teachers used technology at least once a week to present mathematics concepts to them in 2004-05. State-by-state data showed as much as a 25 percentage-point difference between states in the proportion of students whose teachers used computers at least weekly to present mathematics concepts. More than 30 percent of students were in mathematics classes that did not make use of computers at all in 2004-05. In addition, survey responses of mathematics teachers suggest that in 2004-05 almost half of America's students were in classrooms where teachers lacked access to district- or school-provided professional development on the use of computers for mathematics instruction.
National Educational Technology Trends Study: Local-level Data Summary (2008). This summary presents data from the National Educational Technology Trends Study (NETTS), a multiyear evaluation that documents the implementation of the Enhancing Education Through Technology (EETT) program from fiscal year (FY) 2002 to FY 2007. It briefly reviews the methods used to collect and analyze the NETTS data collected from states in the winter of 2004-05, from districts in the spring of 2005 and from teachers in fall of 2005. It also provides descriptive analyses of district and school implementation of the EETT program, focusing on issues that are central to the program: distribution of funds; EETT district investment in educational technology; teacher and student access to technology; technology-related teacher professional development; and technology integration in teaching and learning.
Key findings include: A small percentage of teachers (often under 5%) report using technology to support advanced instructional practices with their students on a weekly basis, such as inquiry based strategies (3%) and solving real-world problems (3%) in school year 2004-05. However, more teachers in high-poverty than in low-poverty schools reported that their students regularly used technology to work cooperatively with other students and solve real-world problems.
Federal Funding for Educational Technology and How It Is Used in the Classroom: A Summary of Findings from the Integrated Studies of Educational Technology (2003) summarizes the three final reports produced by the Integrated Studies of Educational Technology (ISET). ISET consisted of a nested set of state, district, school, and teacher surveys, to provide nationally representative information on federal funding for, and uses of, educational technology.
Smaller Learning Communities
Implementation Study of Smaller Learning Communities: Final Report (2008). This report describes the strategies and practices used in implementing SLCs based on surveys and case studies of the first cohort of grantee schools funded under this program, as well as analysis of annual performance reports submitted by SLC grantees. The study examines the principal strategies, models, and practices that these schools implemented, the factors facilitating and inhibiting implementation in SLC schools, and how outcomes for SLC schools, as measured by student achievement and school behavior, change over time.
An Evaluation of the Participation of Faith-Based and Community Organizations in U.S. Department of Education Discretionary Grant Programs and as Supplemental Educational Services Providers (2007) discusses the success of Faith-Based and Community Organizations (FBCOs) in winning discretionary grants in two specific programs, whether the pool of higher-quality applicants has increased with the participation of FBCOs, and how many Faith-Based Organizations are approved as supplemental educational services providers.
With the exception of FY 2003, between FY 2002 and FY 2004, the success rates for FBCO applicants to the Carol M. White Physical Education Program (PEP) and the Mentoring Program remained fairly stable while they declined for FBCO applicants to the Community Technology Centers (CTC) grant program. In FY 2003, both PEP and CTC saw large increases in the percentages of successful FBCO applicants. The participation of FBCOs was associated with an increase in the pool of higher-quality applicants to PEP and Mentoring as measured by applicant scores; the evidence for CTC is neutral. Finally, states approved an increasing number of faith-based organizations as supplemental educational services providers between December 2002, when states first began approving SES providers, and March 2005.
The Study of the Voluntary Public School Choice (VPSC) Program: Final Report (2008) uses a mixture of quantitative and qualitative methods to measure the progress VPSC sites have made in meeting the program's legislative goals to: 1) maximize choice, 2) encourage students to transfer to higher achieving schools, and 3) promote interdistrict transfers. The evaluation draws from multiple data sources, including: site visits, surveys, program documents, and student achievement records. Data collection started in the fall of 2002 and continued through the end of the five-year grant cycle in the spring of 2007.
The study found that in 2005-06, the overall participation rate in the program was 2.8 percent of the students eligible to enroll. The VPSC program made progress on the first statutory priority, providing a wide variety of choice options overall, but less progress on the second and third statutory priorities. Only five of the 13 sites either limited or tracked their transfers from low- to higher-performing schools, and in these sites, only 21.8 percent of the total transfers in 2005-06 were from low- to higher-performing schools. In addition, most of the VPSC sites limited their choice initiatives to within-district options, rather than develop interdistrict options. The study's achievement analyses, conducted among six cohorts across four of the 13 VPSC sites for at least three years, found that students enrolling in the VPSC initiatives had higher achievement gains in mathematics and reading than those not enrolling.
Study of the Voluntary Public School Choice Program: Interim Report (2007) uses a mixture of quantitative and qualitative methods to measure the progress VPSC sites have made in meeting the program's legislative goals to: 1) maximize choice, 2) encourage students to transfer to higher achieving schools, and 3) promote interdistrict transfers. The interim report discusses the evaluation findings during the first three years of implementation of the VPSC Program.
Evaluation of Flexibility Under No Child Left Behind (2007) examines the Transferability, REAP-Flex, and Local-Flex provisions of No Child Left Behind. Volume I is an executive summary of findings about all three types of flexibility examined in this study. Volumes II and III present more detailed findings on Transferability and REAP-Flex provisions, based on surveys of nationally representative samples of eligible districts. Volume IV is a case study of the single district that is implementing Local-Flex (Seattle). These reports examine the extent to which districts choose to utilize these flexibility provisions, why districts choose to participate, what barriers prevent districts from participating, and potential strategies for increasing participation.
Key findings include: REAP Flex is widely used by eligible rural districts, but districts were less likely to participate in Transferability, and only one district opted to participate in Local Flex. Districts that chose to participate in the three flexibility programs did so in order to focus funds on achieving their goals of making AYP by targeting particular areas of need. Rural districts found flexibility particularly useful because of the small allocations for individual programs and funding constraints associated with declining enrollments. Lack of information and districts' inability to distinguish clear benefits from the flexibility programs were the two main reasons districts gave for not using flexibility provisions.
- Volume I - Executive Summary — PDF (1.1 MB) | MS Word (2.1 MB)
- Volume II: Transferability — PDF (2.47 MB) | MS Word (1.61 MB)
- Volume III: REAP-FLEX — PDF (2.48 MB) | MS Word (1.28 MB)
- Volume IV: Local Flex — PDF (1.17 MB) | MS Word (2.32 MB)
Comprehensive School Reform
The Evaluation of the Comprehensive School Reform Program Implementation and Outcomes: Fifth-Year Report presents overall findings from the evaluation of the comprehensive school reform (CSR) program, including an examination of whether CSR funding had a positive influence on academic achievement. The CSR program was first established as a demonstration program in 1998. It was subsequently authorized under the Elementary and Secondary Education Act of 1965, as amended by the No Child Left Behind Act of 2001 (NCLB). The program emphasized two major concepts: 1) that school reform should be comprehensive in nature, strengthening all aspects of school operations—curriculum, instruction, professional development, parental involvement, and school organization and, 2) that reform should involve the use of scientifically based research models—that is, models with evidence of effectiveness in multiple settings.
The study found that the CSR program did not yield comprehensively reformed schools nor was it associated with widespread achievement gains. While states largely succeeded in providing CSR funds to schools most in need, these schools implemented only some of the legislatively mandated program components. Achievement gains in CSR schools were largely indistinguishable from comparable non-CSR schools. CSR schools that implemented models with a stronger scientific base had some promising impacts, especially in mathematics. However, only one-third of the schools receiving CSR awards selected such models.
Achieving Dramatic School Improvement: An Exploratory Study presents findings describing 11 initially low-performing elementary and middle schools receiving support under the federal Comprehensive School Reform (CSR) program that were able to make dramatic improvements in academic performance. Intensive case studies were conducted on these schools, some of which made noteworthy student achievement gains in a relatively short (one- to two-year) time frame, while others improved at a slower, steadier pace over a longer period. This exploratory study examined the extent to which the reform processes of the schools reflected characteristics and strategies found in the research, whether schools improving at different rates differed in systematic ways, and the most significant challenges faced in both securing and sustaining dramatic school improvements.
Key findings include:
- Nationwide, very few of the initially low performing CSR schools experienced subsequent dramatic and sustained improvement (i.e., less than five percent).
- The dramatic improvement schools consistently adopted well-recognized school reform components: leadership, school climate, instructional improvement, and external support.
- However, these schools implemented reform components differently – there was no single ideal approach to significant school improvement as reform strategies were orchestrated and combined in a variety of ways.
- Sustaining school improvement proved as challenging as achieving it in the first place, with some of the visited schools that had made dramatic short-term achievement gains losing ground over time.
The Evaluation of the Comprehensive School Reform Program Implementation and Outcomes: Third Year Report (2008) provides third-year study findings regarding schools receiving comprehensive school reform (CSR) assistance awards in 2002, focusing on 1) how CSR award receipt was related to subsequent changes in achievement; 2) whether aspects of program implementation were associated with achievement gains. Findings are based on analyses of survey, case study, and assessment data collected from grantees and comparison schools from fall 2002 through spring 2005. Key findings show: 1) receipt of a CSR award was not associated with gains in mathematics or reading achievement through the first three years of award; 2) there was limited evidence that schools adopting models with scientific evidence of effectiveness experienced positive gains, especially in math.
The Longitudinal Analysis of Comprehensive School Reform Implementation and Outcomes evaluation (LACIO) (2004) is a five-year evaluation of the Comprehensive School Reform program (CSR). LACIO examines how schools that first received CSR funds in 2002-03 implemented CSR and the relationships between CSR implementation and student achievement outcomes. This first year report focuses on how states target CSR program funds and what reform activities schools undertake in the first year of implementation.
Implementation and Early Outcomes of the Comprehensive School Reform Demonstration (CSRD) Program (2004) summarizes data relating to CSRD schools from a variety of sources including surveys and case studies, state assessment data, and a database of grantee information. The report examines the targeting of CSRD funds, how well CSRD schools are implementing the nine components of comprehensive school reform described in the 1998 law, and achievement trends in CSRD schools compared with non-CSRD schools.
Field-Focused Study of the Comprehensive School Reform Demonstration Program (2003) presents findings from the Field-Focused Study (FFS), one of several components in the National Evaluation of the Comprehensive School Reform Demonstration (CSRD) Program. FFS provides information on the initial implementation of the CSRD program, including progress in implementing the components of CSRD, the role of district and state influences on implementing the program, and early signs regarding the potential sustainability of the program at the school level. The report is presented in two volumes. Volume I contains the main text. Volume II contains six appendices to Volume I, including short summaries of the 18 schools that were studied.
- Executive Summary, Volume I: Final Report
- Volume I: PDF (774K) | Word (693K)
- Volume II, Appendices: PDF (1.8M) | Word (1M)
A Descriptive Evaluation of the Federal Class-Size Reduction Program (2004) presents findings from the 2000-2001 school year on the distribution and use of federal Class-Size Reduction (CSR) funds, the implementation of CSR, and the effects of CSR on class size. Surveys of district staff and school principals provide generalizable information about the federal CSR program, while qualitative information from site visits to six states, 12 districts (two in each state), 24 schools (two in each district), and 48 CSR classrooms (two from each school) illuminates and verifies the survey findings.
- Report website includes highlights, executive summary, and full report
The Reading First Implementation Study 2008-09 Final Report (2011) examined states' planned responses to the Reading First (RF) budget reduction which took place in FY 08. (Funding for the program was reduced from approximately $1 billion to $400 million and has since been eliminated.) The study found that RF funds were used to support strategies to improve instruction in both RF-funded districts and schools as well as in non-funded districts and schools. State respondents discussed a variety of specific strategies to support continuation of RF teaching practices such as use of reading coaches, use of RF materials and curricula, use of data driven instruction, use of reading assessments, and scientifically based reading instruction.
Reading First Implementation Evaluation Final Report (2008) compares reading practices in a national representative sample of Reading First and non-Reading First Title I schools, and analyzes reading achievement trends in both groups. Results are based on surveys completed in spring 2005 and 2007 by K-3 teachers, principals, and reading coaches, as well as from state and national databases of school-level reading scores on state assessments.
The study found that RF schools devote more time to reading instruction in K-3 classrooms, and are more likely to: a) have reading coaches who assist teachers in implementing their reading programs; b) use reading materials aligned with scientifically based reading research; c) use assessments to guide instruction; d) place struggling readers into intervention services; and e) have their teachers participate in reading-related professional development. Based on analyses of states' reading assessment scores, there is limited but statistically significant evidence that successive cohorts of third- and fourth-grade students in RF schools improved their reading performance over time more quickly than did their counterparts in non-RF Title I schools.
The Reading First Implementation Evaluation (2006) presents findings on providing scientifically based reading instruction in grades K-3, the amount of time for reading instruction, the interventions for struggling readers, the uses of assessment, and professional development activities.
Analysis of State K-3 Reading Standards and Assessments (2005) examines the extent to which the five essential components of effective reading instruction (identified by the National Reading Panel in 2000) have been incorporated into state standards and assessments.
The study found that comprehension and, to a lesser extent, vocabulary are better represented by sampled states K-3 reading standards than are the other three essential elements of reading instruction. States with larger numbers of K-3 reading standards organized to make the five essential elements more visible were judged to represent these elements better. With the possible exception of vocabulary and comprehension in grade 3, statewide reading assessments in 2003-04 do not significantly address expected student outcomes from reading instruction in the five essential areas.
The Second Evaluation of the Improving Literacy Through School Libraries Program (2009) provides findings on how grant funds are targeted to schools, uses of the grant funds, staff collaboration and professional development, and the relationship between participation in the program and reading achievement scores. The report analyzes data from a survey of school libraries, district performance reports, case studies and test scores.
The study found that grantees roughly tripled their expenditures on books and subscriptions and computer hardware, while nongrantees showed little change. In the first evaluation, grantees roughly doubled their expenditures on these items. In schools that participated in LSL in 2003-04, the percentage of students who met or exceeded the proficiency requirements on state reading assessments increased by an extra 2.7 percentage points over the increase observed among nonparticipating schools during the same time period. However, some or all of the increase may be associated with other school reform efforts that also appeared in the schools. Thus, no definitive statement can be made based on these data on whether LSL participation was associated with improved test scores that was separate from these other programs.
The Improving Literacy through School Libraries Evaluation Final Report (2005) provides findings on how grant funds are allocated to districts and targeted to schools, how these funds are being used, and how program participation relates to staff collaboration and professional development.
Private School Participants in Programs under the No Child Left Behind Act and the Individuals with Disabilities Education Act: Private School and Public School District Perspectives (2007) describes participation of private school participants in federal education programs, the consultation process between private schools and public school districts, and public school district allocation of federal funds for services for private school participants. The results presented in this report are based on surveys conducted in 2005-06 among a nationally representative sample of public school districts with at least one private school located within their boundaries and a nationally representative sample of private schools located within the boundaries of the sample districts.
The study found that less than half of private schools reported having at least one participant (students, teachers, or parents) in an ESEA program, though Catholic schools were more likely than other private schools to have at least one participant in an ESEA program (80 percent). Forty percent of private schools with no ESEA participants reported not participating in ESEA programs because they had no knowledge of these programs. Public school districts and private schools with participants in a particular ESEA program generally reported similar levels of consultation about that ESEA program.
Single Sex Education
Early Implementation of Public Single-Sex Schools: Perceptions and Characteristics (2008) has two components. The first component is a review of literature on single sex schooling that was published in the fall of 2005. The second component is based on surveys and site visits conducted in 2005, which provide additional descriptive data on public single sex schools in the U.S. This report summarizes studies and programs implemented before ED issued its recent regulations on single sex schools and classes. Therefore, educators should not rely on the study's findings about perceived effects of single-sex schools or classes in establishing single-sex programs consistent with the ED's regulations. This report may be helpful in considering topics on which further research and evaluation may be warranted.
The results of the systematic review are mixed, though the findings suggest some support for the premise that single-sex schooling can be helpful and little evidence that it is harmful. Among the concurrent academic accomplishment outcomes, 53 percent were null (favored neither single-sex nor coed schooling), 10 percent had mixed results across sex or grade levels, 35 percent favored single-sex schooling, and only 2 percent favored coed schooling. Among the concurrent socio-emotional outcomes, 39 percent were null, 6 percent were mixed, 45 percent favored single-sex schooling, and only 10 percent favored coed schooling.
Single-Sex Versus Coeducation Schooling: A Systematic Review (2005) examines the existing literature on single-sex schooling at the elementary and secondary level, using an unbiased, transparent, and objective selection process adopted from the What Works Clearinghouse (WWC). The review was conducted as part of the ongoing study of single-sex public schools.
Evaluation of the Implementation of the Rural and Low-Income School (RLIS) Program: Final Report (2010) provides information on the implementation of the Rural and Low-Income School (RLIS) program in school year 2008–09 and on achievement trends in RLIS districts between school years 2002–03 and 2007–08. The RLIS program provides funds to support school improvement strategies being implemented by rural districts serving low-income students. The study included interviews with nine RLIS state coordinators and 43 RLIS district coordinators, and analyses of achievement data.
Key findings include: Districts primarily used RLIS funds to support activities authorized under Title I, Part A, such as hiring teachers, providing professional development, and purchasing instructional materials, and educational technology, such as computers, Smart Boards, and software. District coordinators reported that the flexibility of the RLIS program allowed them to use these funds to meet specific needs in their districts. In 2006–07, per-pupil spending was substantially lower in RLIS districts than in all districts nationally or in other rural districts. Though the evaluation did not examine causality and achievement gains cannot be attributed to the RLIS program, the rate of academic improvement in mathematics and reading for districts that received RLIS funding was significantly greater than for non-RLIS rural districts between 2002–03 and 2007–08.
Evaluation of the Implementation of the Rural and Low-Income School (RLIS) Program: Interim Report (2009) provides information on the implementation of the Rural and Low-Income Schools (RLIS) program in school year 2007–08. The RLIS program provides funds to support the improvement strategies that are being implemented by rural districts serving low-income students to make adequate yearly progress (AYP). This report draws on interviews with nine RLIS state coordinators and demographic district-level data.
Key findings include: Most districts (70 percent) that are eligible for RLIS funding were located in the South. Per-pupil spending was substantially lower in RLIS districts than in all districts nationally or in other rural districts. State RLIS coordinators primarily viewed RLIS as an additional funding source to help rural, low-income schools make AYP, not as a separate, stand-alone program. However, state RLIS coordinators also reported that RLIS was the last of their federal grants to be allocated by the U.S. Department of Education, making it difficult for some districts to effectively plan their budgets for the school year.
Evaluation of Flexibility Under No Child Left Behind (2007) examines the Transferability, REAP-Flex, and Local-Flex provisions of No Child Left Behind examines the REAP-Flex provision of No Child Left Behind. Findings related to rural education include: REAP Flex was widely used by eligible rural districts. Rural districts found flexibility particularly useful because of the small allocations for individual programs and funding constraints associated with declining enrollments.
National Assessment of Education Progress
Evaluation of the National Assessment of Educational Progress (2009) addresses issues identified by Congress concerning the quality of the National Assessment of Educational Progress (NAEP), including whether NAEP is properly administered, produces high quality data that are valid and reliable, and is consistent with relevant widely accepted professional assessment standards; and whether student achievement levels are reasonable, valid, and reliable.
Key findings include: The procedures for developing and maintaining NAEP are generally consistent with professional testing standards, but NAEP lacks an organized program of research for providing evidence of the validity of intended uses and interpretations. Many of the procedures for setting NAEP achievement levels are consistent with professional testing standards, with a notable exception regarding external evidence to inform policy decisions for where to make the cut scores defining basic, proficient, and advanced levels of achievement. Although NAEP data are available to compare state performance levels, the appropriateness of doing so is affected by many factors, including alignment between NAEP content frameworks and state education programs, and differences in state student participation rates in NAEP. NAEP’s Web site contains both depth and breadth of information, but the information may not be reaching some intended stakeholders in ways facilitating distinguishing between NAEP achievement levels and those that were developed by states for ESEA reporting purposes.
21st Century Community Learning Centers Descriptive Study of Program Practices (2010) focuses on the implementation of reading and mathematics activities, student attendance, and hiring and retaining qualified staff in 21st Century Community Learning Centers. Results are based on data from surveys and case studies from the 2005-06 school year.
The study found that three-quarters of the centers reported that a typical student participated in reading activities (75 percent) and mathematics activities (81 percent) for less than 4 hours per week. Centers reported that about half of their students attended roughly 2 days a week or more. About half of centers reported offering professional development opportunities to staff through training courses or conferences.
Towards the Identification of Features of Effective Professional Development for Early Childhood Educators – Literature Review (2010) analyzes the research on professional development of early childhood educators to characterize features of effective professional development. This review examines findings from research on four targets of early childhood professional development: 1) strengthening human and/or social capital; 2) strengthening practices at institutions or organizations providing professional development; 3) strengthening early educator practices related to specific child outcomes; and, 4) strengthening overall quality in classroom or group settings.
Although the strength of evidence varies, the literature review concludes that professional development for early childhood educators is more effective when:
- There are specific and articulated objectives for professional development.
- Attention is given to linking the focus on early educator knowledge and practice, and practice is an explicit focus of the professional development.
- There is collective participation of teachers from the same classrooms or schools in professional development.
- The intensity/duration of the professional development is matched to the content being conveyed.
- Educators are prepared to conduct child assessments and interpret their results as a tool for ongoing monitoring of the effects of professional development
- Professional development is aligned with standards for practice and appropriate for the organizational context.
Academic Competitiveness and National SMART Grant Programs: Lessons Learned: 2006-07 Through 2009-10 (2012). This report highlights implementation issues, and describes program participations and grant renewal rates. The research undertaken for this report found that both the ACG and the National SMART Grant were relatively small programs that operated differently than most Title IV programs, and required longer lead times to establish processes for clarifying new requirements and disseminating information than typically required for those programs that represent variations on existing processes. In addition, many students who enrolled in both programs lost their awards the following year because of their inability to meet the academic requirements for renewal. While all recipients were from low-income families, both ACG and National SMART Grants were most likely to benefit recipients at the higher end of the Pell Grant-eligible group.
Academic Competitiveness and National SMART Grant Programs: 2006–07 through 2008–09 (2011). This is the third report from a five-year study that examined program participation in the Academic Competitiveness Grant (ACG) and the National Science and Mathematics Access to Retain Talent (National SMART) Grant programs. Among the major purposes of the study were to determine whether or not the financial incentives provided by the ACG program induced more economically disadvantaged high school students to complete a rigorous high school program and enroll and succeed in postsecondary education and whether the National SMART Grants motivate more students to major and receive degrees in science, technology, engineering, mathematics (STEM) fields or languages critical to national interest . This third report summarizes participation data from the first three years of the ACG and National SMART Grant programs (2006-2007 through 2008-09), and major findings include:
- The number of ACG and National SMART Grant recipients has increased, although the percentage of Pell Grant recipients with these grants has remained low.
- Many recipients could not meet the strict conditions required to renew their grants the following year.
- First-year ACG recipients and third-year National SMART Grant recipients persisted at higher rates than their counterparts with only a Pell Grant.
The Impacts of Upward Bound Math-Science On Postsecondary Outcomes 7-9 Years After Scheduled High School Graduation (2010) updates the report Upward Bound Math-Science: Program Description and Interim Impact Estimates published in 2007 (Olsen et al. 2007). The 2007 interim report contained descriptive findings from a survey of Upward Bound Math-Science (UBMS) grantees from the late 1990s at the time of the study's initiation and impact estimates through the period four to six years after expected high school graduation of the study sample. The current report presents impact estimates for the period seven to nine years after scheduled high school graduation. For context purposes this report includes descriptive information from the initial reports that gives a picture of the UBMS program as it was operating shortly after the time when the study sample members were participating in UBMS (1993-1995). It should be noted that the study sample and results represent the UBMS program as it was operating in the early years of its initiation.
The study found an association between UBMS participation and:
- an increase in enrolling in more selective four-year institutions.
- an increase in postsecondary degree completion overall and at four-year institutions.
- an increase in the likelihood of earning a degree in a social science field of study. Although UBMS students showed positive effects in the direction of increasing the likelihood of majoring in math and physical science fields, effects were not statistically significant.
National Evaluation of Student Support Services: Examination of Student Outcomes After Six Years (2010) compares the educational outcomes of Student Support Services (SSS) program participants and non-SSS program participants six years after enrolling in college as first-year students. SSS programs can offer a mix of academic and support services such as professional or peer tutoring, study labs, and instructional courses at institutions of higher education. This study was designed to estimate the impact of supplemental services on college retention, transfer, and completion using quasi-experimental methods.
Key findings include: Analytic models that account for differences in service levels generally showed positive and statistically significant effects. Participation in SSS projects as measured by the amount of services received during the freshman year is associated with moderate increases on the key measures of college retention and degree completion but neither increases nor decreases student transfers from two-year to four-year institutions and neither increases nor decreases the outcomes on some of the key measures in the HLM models. Although these models controlled for student demographics and, whenever possible, prior achievement, one limitation of this model is the potential selection bias of participants who received more services. Models that measure supplemental services regardless of whether they were offered by the SSS project are associated with positive and statistically significant effects on all outcome measures of retention, transfers, and degree completion. In addition, this report includes analyses that simply consider whether or not the student was classified as being in SSS as a college freshman. A major limitation of this analysis is that it does not account for the level of service received by SSS participants; nor does it account for the fact that comparison students may have received similar services that were not funded by the federal SSS program. This measure did not show any effect from participating in SSS as a college freshman.
A Study of Four Federal Graduate Fellowship Programs: Education and Employment Outcomes (2008) describes the academic and employment outcomes as of 2006 for graduate students who received financial support between 1997 and 1999 through one of four federal fellowship programs: The Fulbright-Hays Doctoral Dissertation Research Abroad (DDRA) fellowship program, the Foreign Language and Area Studies (FLAS) fellowship program, the Graduate Assistance in Areas of National Need (GAANN) fellowship program, and the Jacob K. Javits fellowship program. The programs vary significantly with respect to their goals, the number of fellowships supported, and the amount of funding dispersed. Despite their differences, however, all of these programs are intended to encourage academically talented students to become experts in fields important to the national interest.
Key findings include:
- Despite differences among these four fellowship programs in purpose and implementation, there are noteworthy similarities in their outcomes. With respect to education outcomes, the majority of fellows in each of the four programs completed their degrees, with the percentage of degree completions ranging among programs from about two-thirds to nine-tenths of fellowships.
- Fellows who completed their degrees tended to do so in less time than graduate students overall. National surveys indicate that doctoral students who complete their degrees do so in seven to twelve years, depending on their field of study, with students in the humanities and social sciences taking more time than students in the natural sciences.
- With respect to employment outcomes, large proportions of students who received fellowships participated in the labor force after completing their fellowships, most commonly in work that was related to their fellowship-gained expertise and was part of a career they were pursuing.
- Full Report MS Word (21.6M)
The study findings are based on a random assignment design implemented in a nationally representative sample of 67 Upward Bound projects hosted by two-and four-year colleges and universities. About 1,500 eligible applicants were randomly assigned to the evaluation's treatment group, and allowed to participate in Upward Bound, and about 1,300 students were randomly assigned to the control group. Data were collected periodically on high school and postsecondary outcomes for both groups from an initial baseline in 1992-1994 through a final survey in 2003-2004. Impact estimates are based on a comparison of outcomes for students in the treatment and control groups.
The study concluded that Upward Bound 1) had no detectable effect on the rate of overall postsecondary enrollment, or the type or selectivity of postsecondary institution attended; 2) increased the likelihood of earning a postsecondary certificate or license from a vocational school but had no detectable effect on the likelihood of earning a bachelor's or associate's degree; and 3) increased postsecondary enrollment and completion for students with lower educational expectations at baseline.
Academic Competitiveness and National SMART Grant Programs: 2006-07 and 2007-08 (2010) examines participation in the Pell Grant, ACG, and National SMART Grant programs in 2007-08 and examines renewal ratesthat is, how many students who received a grant in 2006-07 received another one in 2007-08. Key questions of this report are: 1) whether the financial incentives provided by the ACG program induce more economically disadvantaged high school students to complete a rigorous high school program and enroll and succeed in postsecondary education, and 2) whether the availability of National SMART Grants motivate more students to major and receive degrees in mathematics, science, engineering, technology, and critical languages. Definitive answers to these questions will require time. Analysis to date, therefore, has focused on documenting implementation and early participation.
Although many stakeholders were initially frustrated by the lack of awareness about the programs, information on the grant programs is now more widely available, and some of the difficulties and concerns have been eased by clarifying regulations. Subsequent legislation expanded eligibility to a wider population of Pell recipients. Participation in both grant programs has been lower than initially predicted (425,000 ACGs and 80,000 National SMART Grants). In 2006-07, the first year of the program, 301,700 students received an ACG. In 2007-08, participation increased by 32 percent to 398,700. Among the first-year students who had received an ACG in 2006-07, just over one-quarter (27 percent) received another one in 2007-08. Almost half (48 percent) received another Pell Grant in 2007-08, but not an ACG. The remaining 26 percent received neither an ACG nor a Pell Grant, either because they did not meet the income or enrollment requirements for a Pell Grant or were not enrolled. In 2006-07, 62,400 students received a National SMART Grant, increasing by 5 percent to 65,400 in 2007-08. Overall, more than one-half (57 percent) of third-year students who had received a National SMART Grant in 2006-07 met the requirements to renew it as a fourth-year student.
Academic Competitiveness and SMART Grant Programs: First-Year Lessons Learned (2009). Two new grant programs, Academic Competitiveness Grants (ACG) and National SMART Grants (NSG) were created in Higher Education Reconciliation Act of 2005 (HERA). ACGs are intended to encourage students to take more challenging courses in high school-making success in college more likely. NSGs are intended to encourage post-secondary students to take college majors in high demand in the global economy, such as science, technology, engineering, and mathematics (STEM) and critical foreign languages. Students eligible for Pell Grants who completed a 'rigorous program of study' in high school received an ACG of up to $750 in their first year and, if they earned a 3.0 or better grade point average (GPA), up to $1,300 in their second year. Pell-eligible students who majored in a STEM field or critical foreign language and maintained a 3.0 GPA received an NSG for up to $4,000 for their third and fourth years.
Academic Competitiveness and SMART Grant Programs: First-Year Lessons Learned describes the early implementation of the new legislation and the number and distribution of students receiving the first grants in academic year 2006-2007. Key findings are:
- Given the rapid implementation of the programs, many stakeholders reported difficulties in identifying eligible students.
- Of the $790 million appropriated for these programs for the initial year FY 2006, approximately $448 million (57 percent) was disbursed. Fewer students received awards than estimated: About 300,000 ACGs and 60,000 NSGs were awarded, as compared to initial budget estimates of 425,000 ACGs and 80,000 NSGs.
- About three-quarters of ACG recipients were first-year students, suggesting that second-year students had difficulty meeting the 3.0 GPA requirement.
- Of 3,600 postsecondary institutions eligible to award Pell Grants and ACGs, about 2,800 (78 percent) participated.
Based on site visits, student and parent surveys, and school records, this report Early Outcomes of GEAR UP Program: Summary of Evaluation Findings (2008) discusses findings from an analysis of the Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) grant program. GEAR UP provides services to improve parents' and students' knowledge and preparation for postsecondary attendance. The GEAR UP model is to provide services to the entire grade cohort starting no later than the 7th grade and stresses partnership with schools, districts, community organizations and postsecondary institutions. The report presents middle school outcomes for a small sample of schools participating in the program and a matched comparison group of non-GEAR UP schools.
The evaluation described early implementation (during junior high school) and describes the association between GEAR UP participation and early parent and student outcomes. Compared to the comparison group, attending a GEAR UP school was associated with improvement in parents' knowledge of opportunities and benefits of a postsecondary education, improved student knowledge of available postsecondary education opportunities, increases in parents' involvement in the school and their children's education, increases in parents' expectations for their children, and increased enrollment in above-grade level science courses. GEAR UP schools were also more likely to offer honors and above-grade-level classes.
The Educational and Employment Outcomes of The Ronald E. McNair Postbaccalaureate Achievement Program Alumni (2008) This study examines the extent to which former McNair Program participants enrolled in the program between 1989-1998, completed doctoral studies, and obtained faculty or research positions at institutions of higher education. A telephone survey was conducted with the census of 813 doctoral and professional degree recipients, and a sample 580 McNair participants who had earned master's degrees, 615 who had earned bachelor's degrees, 604 who had not earned any degree between the fall of 1989 and the spring of 2000. Estimates presented throughout this report use weighted data to account for probability of selection, non-response, and post-stratification. Data analyses indicated that 14.4 percent of the 1989-93 cohort of former McNair program participants and 3.9 percent of the 1994-1998 cohort reported earning doctorates. Findings also suggest that a high percentage (73 percent) of former participants with bachelor's degrees had enrolled in graduate school at some time within a five- to sever-year period after receiving their bachelor's degree. As a point of reference, 30 percent of typical B.A. recipients surveyed in the NCES' Baccalaureate and Beyond Survey entered graduate schools within five years after college graduate. Also, among McNair participants with either a Ph.D. or other doctorate, 65 percent indicated that they were employed in higher education (75 percent of the Ph.D. recipients and 50 percent of other doctoral degree holders).
A Study of the Effect of Talent Search on Secondary and Postsecondary Outcomes in Florida, Indiana, and Texas (2006). This report presents an analysis of the effectiveness of the Talent Search in Florida, Indiana, and Texas, drawing upon project, state, and federal administrative records to examine short-term program outcomes for program participants, and a quasi-experimental design to create matched comparison groups. Study findings suggested that Talent Search participants were more likely than comparison students to apply for federal financial aid and enroll in public postsecondary institutions in Florida, Indiana, and Texas.
In addressing the efficacy of using large state databases to inform policy, the study also that this is a challenging approach given that not all data on student characteristics and on secondary or postsecondary outcomes of interest were available in any one state, and the type of information that was missing differed across states. However, such data should be easier to attain as more states develop systems for compiling secondary and postsecondary school records, and federal programs are more consistent in reporting information on the participants served and maintain records electronically.
Partnerships for Reform: Changing Teacher Preparation Through the Title II HEA Partnership Program: Final Report (2006) provides information on the implementation of the Title II Partnership grant program from the 2000-01 school year through the 2003-04 school year. The study collected information on the 1999 grantees through surveys of representatives from institutions of higher education, participating school districts, and participating school principals. The study also included secondary data analyses on school characteristics, achievement data, and pass rates on teacher assessments.
Borrower Debt Burden (2004) is one of the major issues in student financial assistance. PPSS has undertaken several analyses combining data on federal borrowing from a sample of borrowers from the National Student Loan Data System (NSLDS) maintained by the Department of Education, with income data from the Internal Revenue Service (IRS). These data allow us to calculate the percentage of a borrowers’ income, and their spouse if married filing jointly, that is needed to meet their scheduled federal student loan payments.
- Data from our analysis has been made available on the NCES website.
The Review of the Fund for the Improvement of Postsecondary Education (FIPSE) Comprehensive Program (2004) assesses the extent to which the FIPSE Comprehensive Program meets its key objectives and is the first outside review of the program in over two decades. The report is based on a sample of 60 randomly selected projects funded from 1996 through 1998, 16 of which were selected for a more intensive review by individuals with expertise specific to the individual project. Information for the study was obtained mainly from project reports and documentation maintained by the FIPSE office, supplemented by discussions with project staff members, staff members at institutions replicating FIPSE projects, and FIPSE staff.
The Impact of Regular Upward Bound: Results from the Third Follow-up Data Collection (2004) assesses the impact of program participation on students' preparation for college, college enrollment, persistence, and completion. Findings are based on a nationally representative sample of 67 Upward Bound projects hosted by two-and four-year colleges, from which 2,800 eligible applicants were randomly assigned to Upward Bound or to a control group. At the time of the most recent follow-up data collection, all sampled students had sufficient time to complete high school and about two years in which to enter college. This report, therefore, focuses on high school preparation for college, enrollment, and early college persistence.
Upward Bound Math-Science Program Description and Interim Impact Estimates (2006). This report presents descriptive findings from an Upward Bound Math Science (UBMS) project survey conducted in 1999 and impact estimates from post high school follow up to 2002 of early cohorts of project participants between 1993 and 1995. The Upward Bound Math Science initiative was established in 1990 within the Upward Bound program to foster increased math and science participation among disadvantaged students in high school as a preparation for success in college in math and science and to increase in the number of students participating in math and science fields. The UBMS sample of about 1,500 UBMS participants was added to the existing National Evaluation of Upward Bound, and utilized a propensity analysis matched comparison group selected from the existing UB sample. Descriptively the project survey found that the average program provided a total of 240 hours of academic instruction per participant, most of it in a 6-week summer program in which much of the instruction focused on math and science subjects. The impact study found there was an association between UBMS participation and improved high school grades in math and science and overall, an increase in the likelihood of taking chemistry and physics in high school, and an increase in enrolling in more selective four-year institutions. The study also increased the likelihood of math and science major field choice and completing a four-year degree in math and science.
Implementation of the Talent Search Program, Past and Present - Final Report from Phase I of the National Evaluation (2004). This report presents a comprehensive, in-depth description of the implementation of Talent Search throughout the country. Study relied on information obtained from multiple sources: a survey of Talent Search project directors, student-centered case studies conducted in 14 Talent Search projects, performance reports, and other education data sets.
Descriptive findings indicated that despite recent modest increases in average funding per participant, Talent Search remains a generally nonintensive program. For the most part, participation in program services is optional; basic services might be offered biweekly or even just once a month; and many students spend less than 10 hours in program activities over the course of a year. Overall, the program still adheres to the original assumption that small amounts of service, delivered at crucial times, can make a difference in students' decisions concerning college preparation and enrollment. Talent Search projects, both at the middle and high school levels, typically provided many diverse activities rather than focusing on just a few types of services. Service delivery approaches varied, too, by type of service, time, place, target group, and providers. Additionally, some projects are finding it increasingly difficult to pursue the traditional pull-out approach of delivering services during the regular school day due to increased pressure on schools to improve academic performance. Many students stay in Talent Search a relatively short time-and not just those who join toward the end of high school.
- Complete Report Website
Partnerships for Reform: Changing Teacher Preparation Through the Title II HEA Partnership Program Interim Report (2004) is an interim report providing descriptive data from the evaluation of the Title II HEA Partnership Grant Program. It examines how 1999 Partnership grantees are implementing reforms to improve preservice teacher preparation and meet the teacher quality needs of school districts. The evaluation focuses on several key topics: 1) changes to the content and structure of grantees' teacher preparation programs over the grant period; 2) connections between the Partnership grants and changes to the teaching force; 3) connections between collaborative activities among institutions of higher education and schools and school-level student achievement; 4) organizational changes and relationships among Partnership members; and 5) efforts to institutionalize Partnerships.
National Evaluation of GEAR UP (2003) evaluates the early effects and implementation of Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP), which seeks to increase postsecondary access and completion through partnership grants that require recipients to begin providing services to students by seventh grade and to continue services to these students in participating high schools until graduation, and through grants to state agencies.
The Performance Measurement Study Of The Title III Institutional Aid Program (2000) presents the results of a survey of all 1995-96 Part A and Part B Title III grantees as well as in-depth case studies conducted at 19 institutions. The Aid for Institutional Development programs (commonly referred to as the Title III programs) support improvements in educational quality, management, and financial stability at qualifying postsecondary institutions that enroll large proportions of minority and financially disadvantaged students. The study indicates that institutions used Title III grants primarily to strengthen academic programs rather than on institutional management. The study also found that most Title III schools were not in severe financial difficulty.
National Study of the Operation of the Federal Work-Study Program (2000) analyzes student and institutional experiences with the FWS program and describes how the program is operated, based on a nationally representative survey of Federal Work-Study (FWS) administrators and recipients conducted in 1998.
Factors Related to College Enrollment (1998) examines factors related to postsecondary education enrollment. The emphasis is on how early indicators,such as expectations and course-taking behavior in the eighth grade, are related to college attendance six years later. The report examines attendance at all types of postsecondary education: 4-year public, 4-year private, less than 4-year public, and less than 4-year private institutions.
Career and Technical Education
National Assessment of Career and Technical Education: Final Report (2014) summarizes the most recent available data on the implementation of Carl D. Perkins Career and Technical Education Act of 2006 (Perkins IV), as well as student participation and outcomes for CTE more generally. The report uses information from studies commissioned for the NACTE, reviews of existing research, and analyses of extant data from state performance reports and from the National Center for Education Statistics (NCES). Topics include student participation in CTE programs at the secondary and postsecondary levels, changes in Perkins funding levels and targeting, implementation of Perkins IV provisions regarding POS and accountability, and educational and employment outcomes for CTE students.
Putting "career" in "college and career ready": The report of the Independent Advisory Panel of the National Assessment of Career and Technical Education (2014) provides the conclusions and recommendations of the independent panel that was congressionally mandated to provide advice on the design and implementation of the NACTE.
- Report PDF (299KB)
National Assessment of Career and Technical Education: Interim Report (2013) describes the overall research approach for conducting the national assessment required under the Carl D. Perkins Career and Technical Education Act of 2006 and presents preliminary findings on participation, outcomes, and international comparisons of secondary school career and technical education (CTE). The interim report provides baseline information and lays the groundwork for a comprehensive final report.
Supplemental Reports of the National Assessment of Career and Technical Education (2013) are studies commissioned from independent researchers and evaluators that examine different aspects of career and technical education in the United States, such as student outcomes and the implementation of career and technical education programs. The supplemental reports are source materials for the National Assessment of Career and Technical Education. Links to these reports are available on the NACTE reports page.
National Assessment of Vocational Education: Final Report to Congress (2004) presents a synthesis of evidence on the implementation and outcomes of vocational education and of the 1998 Carl D. Perkins Vocational and Technical Education Act (Perkins III). It examines questions about the effectiveness of vocational education in improving student outcomes, the consequences of new funding and accountability provisions for programs and participants, the implementation and quality of vocational education, and the extent of its alignment with other reform efforts. The report also provides options for the future direction of vocational education legislation.
- Executive Summary PDF [388K] | Word [502K]
- Full Report PDF [1.4M] | Word Part I [3.6M] | Word Part II [2.8M]