K–8 Charter Schools: Closing the Achievement Gap
Innovations in Education
Downloadable File PDF (2 MB)

Appendix A

Research Methodology

The research approach used to develop this guide is a combination of case study methodology and benchmarking of "best practices." Used in businesses worldwide as they seek to continuously improve their operations, more recently benchmarking has been applied to education. Benchmarking is a structured, efficient process that targets key operations and identifies the following: promising practices in relationship to traditional practice, previous practice at the selected sites (lessons learned), and local outcome data. The methodology used here is further explained in a background document,24 which lays out the justification for identifying promising practices based on four sources of rigor in the approach:

  • Theory and research base;
  • Expert review;
  • Site evidence of effectiveness; and
  • Systematic field research and cross-site analysis.

The steps of the research process were: defining a study scope, seeking input from experts to refine the scope and inform site selection criteria, screening potential sites, selecting sites to study, conducting site visits, collecting and analyzing data to write case reports, and writing a user-friendly guide.

Study Framework and Data Collection

A conceptual framework was developed to guide the study of the selected sites. While many things happen at a busy school site, each case study needed to focus on those practices most likely to contribute to a school’s success. The framework for this study was an adaptation of that used in a previous guide on charter schools in this Innovations in Education series,25 which was derived from the research literature on charter schools and on organizational effectiveness. The dimensions of the conceptual framework were mission-driven school, school operations and educational program, family involvement and external partnerships, and governing for accountability. A site visit to each school was conducted to gather the information for this guide. Each visit lasted for one or two days and included informal observations throughout the school, attendance at events, and interviews. The primary source of data was interviews with a variety of key groups, including parents, teachers, board members, administrators, and school partners. An interview protocol was developed based on the study framework and adapted to each role group. That is, separate but overlapping sets of questions were developed for teachers, administrators, parents, and other interviewees. Interviews were digitally recorded with key interviewees and later transcribed for more detailed analysis.

Documents from each school served as an additional source of information. Collected during the site visit, these documents included such items as school schedules, sample assessments, lesson plan forms, teacher planning protocols, newsletters, application forms, brochures, charter plans, and report cards. Principals and executive directors also completed a standard form to facilitate consistent compiling of school demographic and outcome information.

Site Selection Process

For this guide, a school had to show evidence of closing the achievement gap. Based on state standardized test data, closing the achievement gap means that students are outperforming local district public schools that serve a similar population of students in mathematics and reading. Alternately, closing the gap may mean that certain subgroups—including African-American and Hispanic students, those receiving special education services, English language learners, and students eligible for free or reduced-price lunch (an indicator of family poverty)—are outperforming either state averages or district students from the same subgroup. Schools included in this guide also must have met adequate yearly progress (AYP) for at least the past two consecutive years. They must each serve a student body where at least half of the students qualify for free or reduced-price lunch and half are students of color.

After examining AYP, student demographics, and achievement gap data from Washington, D.C. and all 40 states with charter school legislation, an initial list of candidates included approximately 187 schools from 30 different states and Washington, D.C. Based on recommendations from an external advisory group of charter school researchers, charter school practitioners, and representatives from various organizations working to support charter schools, information from state department staff and state association leaders, and review of achievement data, the initial list was narrowed down to 22 schools. These schools served students in kindergarten through eighth grade, had met AYP for at least the past two consecutive years, had a full grade set of elementary school-age students, and served at least 50 percent of students who qualified for free or reduced-price lunch programs, at least 50 percent of the school population were students of color, and demonstrated evidence of closing achievement gaps. After screening data, 18 schools were selected and information about program features and additional outcome data were collected using phone interviews to fill gaps in information.

From this group of 18 schools, seven schools were ultimately chosen as case study sites, based on the compiled information and criteria ratings on a screening matrix. Demographic variation, a range of promising practices, geographic location, and achievement data all were considered in the final site selection.

Selection Criteria

A cross-section of schools were selected to highlight K–8 charter schools successfully meeting the needs of traditionally underserved populations of students (e.g., low income, special education, African-American and Latino students), with strong academic programs serving a range of grade configurations (e.g., pre-K–8, K–7, 5–8, K–12) and in a range of geographic locations, all making academic achievement gains. Schools were selected based on the following criteria, prioritized by the advisory group as key issues for consideration.

Demographic Criteria

Many schools demonstrated that they were working hard to educate students, who have been largely underserved in traditional public schools. The schools that were selected had student populations with two or more of the following characteristics: Sixty percent or more qualify for free or reduced-price lunch, 60 percent or more are African-American, Hispanic, or both; 10 percent or more receive special education services, and 60 percent or more are English language learners.

Achievement Criteria

Schools selected met AYP targets for at least two consecutive years, including the most recent year for which data were available. Researchers looked for schools that scored at least at the 50th percentile in mathematics or reading on state standardized tests with demonstrated evidence of continued improvement over several years, or for schools that were consistently high achieving, ranking in the 90th percentile range annually. Data from state departments of education Web sites and the Web site provided achievement information.

Achievement Gap Criteria

Researchers looked for additional evidence that schools were making progress eliminating achievement gaps. A school was considered to be narrowing the achievement gaps if, internally, gaps among students of different races and socioeconomic backgrounds were closing over time or if the school was demonstrating higher achievement for low-income, minority, or students receiving special education services in comparison with a similar population of students in its local district public schools.

Analysis and Reporting

A case report was written about each site, and reviewed by site administrators for accuracy. From these case reports, artifacts, and transcripts of interviews, the project team identified common themes that contributed to success across the sites. This cross-site analysis was built using both the research literature as reflected in the study scope as well as emerging patterns in the data.

This descriptive research process suggests promising practices, including ways to do things that other educators have found helpful and lessons they have learned, as well as practical how-to guidance. This is not the kind of experimental research that can yield valid causal claims about what works. Readers should judge for themselves the merits of these practices, based on their understanding of why they should work, how they fit the local context, and what happens when they actually try them. Also, readers should understand that these descriptions do not constitute an endorsement of specific practices or products.

Using the Guide

Ultimately, readers of this guide will need to select, adapt, and implement practices that meet their individual needs and contexts. Schools coming together in learning communities may continue to study, using the ideas and practices from these sites as a springboard for their own action research. In this way, a pool of promising practices will grow, and schools can support each other in implementation and learning.

   13 | 14 | 15
Print this page Printable view Bookmark  and Share
Last Modified: 11/17/2009