U.S. Department of Education: Promoting Educational Excellence for all Americans - Link to ED.gov Home Page

Chapter 1: Overview of the Evaluation and The Year One Report

The Public Charter Schools Program (PCSP), established in 1994, represents the federal government's commitment to help charter schools meet planning, start-up, and early implementation costs. By helping charter schools overcome financial barriers, the PCSP is also designed to increase the number of charter schools nationwide. As of October 1999, 1,692 charter schools were in operation in 30 states, the District of Columbia, and Puerto Rico.1

Researchers at SRI International are conducting the first national evaluation of the PCSP. In addition to gathering systematic information about the program at the local, state, and federal levels, the 52-month study will continue to document the evolution of the charter school movement (a process begun under another study funded by the U.S. Department of Education and conducted by RPP International). Data collection and analysis for the SRI study are driven by a series of research questions on the operations and impacts of the PCSP, the characteristics of charter schools, charter school flexibility, and charter school accountability.

The Public Charter Schools Program

Federal interest in supporting the development of the charter school movement began in 1993, when President Clinton first proposed the Public Charter Schools Program (PCSP) and several Senators and Representatives proposed the Public Schools Redefinition Act. No action was taken, however, until the reauthorization of the Elementary and Secondary Education Act (ESEA) in 1994. At that time, the PCSP was enacted as Title X, Part C, of ESEA, with an initial appropriation of $6 million in FY 1995. The PCSP is a discretionary grant program, administered in the Office of School Improvement Programs in the Office of Elementary and Secondary Education. The PCSP is intended to support the planning, development, and/or initial implementation of charter schools, providing relatively unencumbered seed funding for states with charter school laws to distribute to charter school groups during the first 3 years of a charter school?s existence. The statute also makes provision for individual charter schools to apply directly to the Secretary of Education for a grant if their states choose not to participate or have been denied a grant.

Congress reauthorized the PCSP in 1998 by passing the Charter School Expansion Act of 1998 (P.L. 105-378). Eligibility for subgrants was expanded to include mature charter schools, which could apply for funds to disseminate promising school practices. The appropriation for FY 2000 was $145 million.

According to the authorizing legislation, the purpose of the Public Charter Schools Program is to "increase national understanding of the charter schools model by (1) providing financial assistance for the planning, program design and initial implementation of charter schools; (2) evaluating the effects of such schools, including the effects on students, student achievement, staff, and parents; and (3) expanding the number of high-quality charter schools available to students across the Nation" (Sec. 10301[b]).

These purposes are addressed by providing grants to states. States determine the process for putting 95 percent or more of the state grants into the hands of charter school planning groups and charter schools in the form of subgrants. These subgrants can be used for planning, implementation, and?for schools that have been in operation for 3 years or more?dissemination of promising charter school practices and programs. If a state has not applied for or received a PCSP grant, charter school planners may apply directly to the U.S. Department of Education (ED) for funds.

In addition, the Secretary of Education is authorized "[t]o provide for... evaluations or studies that include the evaluation of the impact of charter schools on student achievement" and "[t]o provide . . . for the collection of information regarding the financial resources available to charter schools. . . " (Sec. 10305[a]). The research reported in this document was supported by PCSP funds set aside for research and other national activities.

The Charter School Movement

Even though charter schools in the United States predate the PCSP, the movement is still less than 10 years old. Since the first charter school opened in 1992 in Minnesota, the number of these schools has increased steadily, and President Clinton has repeatedly challenged educators to increase the number to 3,000 by 2002. As the movement has grown, it has also struggled. Stories began to emerge early about the difficulties that charter schools?particularly those that were being created from scratch?faced in their first months and years (RPP International and the University of Minnesota, 1997). Depending on the specifications about state and local funding streams in a state?s charter school legislation, groups seeking to open charter schools were often obliged to capitalize the planning and early development of their schools out of their own pockets or by incurring debt. Finding, renting or buying, and renovating space were particular barriers, according to early surveys of the charter school field.

Exhibit 1-1 illustrates the growth of the charter school movement. The National Study of Charter Schools (Nelson et al., 2000) reports several interesting statistics about this growth, which the SRI data will supplement in this and later years. For an overview of charter school activity by state, see Appendix A.

SRI International?s Evaluation of the PCSP.

The PCSP legislation authorized an evaluation of the program, along with other national activities. Researchers at SRI International began this first evaluation of the PCSP in October 1998. The 52-month study has two purposes: (1) evaluation of the rapidly growing Public Charter Schools Program and (2) continued documentation of the evolution of the charter school movement begun under the National Study of Charter Schools, conducted by RPP International.2 The findings reported in this document are from the first year of the SRI evaluation, which included telephone surveys of (1) state charter school coordinators in 36 states, Washington, D.C., and Puerto Rico (N=38); (2) representatives of agencies that are authorized to award charters to charter schools under state charter school laws (charter school authorizers) (n=48); and (3) directors of charter schools that received PCSP grants directly from the U.S. Department of Education (ED) (n=14). Year 1 data collection also included the extraction of information from federal files maintained on PCSP grantees, analysis of state charter laws, interviews with federal staff, and focus groups with charter school planners, operators, and charter school authorizers.

Exhibit 1-1
NUMBER OF CHARTER SCHOOLS AND CHARTER SCHOOL STATES

Beginning of School Year

Number of States with Charter School Laws

Number of States with Charter Schools

Number of Charter Schools

 

Source

1992-1993

2

1

2

Nelson et al. (2000). (Note: Data include Washington, D.C., starting in 1996-1997, but not Puerto Rico, which authorized charter schools in 1993.)

1993-1994

8

3

36

1994-1995

11

6

100

1995-1996

18

10

254

1996-1997

26

17

432

1997-1998

30

24

721

1998-1999

34

28

1,122

1999-2000

38

32

1,692

SRI. (Includes D.C. and Puerto Rico.)

Research Questions and Data Sources

The dual purposes of this evaluation are evident in the broad research question clusters that guide the data collection, analysis, and reporting requirements. Some questions are concerned specifically with the Public Charter Schools Program, and others are about the evolution of the charter school movement in a more general sense. Exhibit 1-2 displays the research questions under each cluster and summarizes the data collection methods that are being used to address them.

During the 4-year evaluation, answers to the research questions will be obtained through several sources of quantitative and qualitative data. The sources will differ for each of three planned reports. The data sources that were used this year are described below. The sources that will be used in later years are revisited in the last chapter of this report. Data sources for this report consisted of the following:

Telephone survey of state charter school coordinators. The charter coordinators (or equivalent) for all 38 states with charter school legislation were surveyed by phone in the summer and fall of 1999. The survey instrument incorporated a combination of open- and closed-ended items on charter school-related operations, requirements, and flexibility, and the state?s PCSP grant. Members of the SRI study team administered the surveys, which took approximately one hour per survey respondent. The response rate was 100 percent. (The basic characteristics of the state universe are displayed in Appendix A.)

Telephone survey of charter school authorizers. A sample of 50 authorizers that have awarded charters to charter schools was selected during the spring and summer of 1999. Among the states with charter school laws, many types of entities are authorized to issue charters. Because of uncertainties about the respondent pool for this part of the charter school story, the evaluation team and the study?s Technical Work Group favored a purposive sampling strategy for the first year of data collection to maximize the diversity of authorizers surveyed. Data collection took place at the same time as the state coordinator survey and involved the same researchers. The charter school authorizer survey also took about one hour to complete. Survey questions were designed to document charter school authorizers? experiences in assisting and monitoring charter schools; granting, renewing, and revoking charters; and financial issues (including the federal PCSP funds). Forty-eight charter school authorizer surveys were completed within the data collection window, for a response rate of 96 percent.

Exhibit 1-2
RESEARCH QUESTIONS AND DATA SOURCES

 

 

 

Federal File Extraction

(Years 1-3)

PCSP Evaluation: Telephone Surveys

Other

Surveys

PCSP Evaluation: Other Data Collection

Charter Schools

(Year 2-3)

State Coordi-nators (Years 1 & 3)

Charter School Authorizers

(Years 1-3)

PCSP Charter Schools

Non-PCSP Charter Schools

 


RPP

 


SASS

Federal Interviews

(Years 1-3)

Site Visits*

(Years 2-3)

How does the PCSP encourage the development of charter schools?

How the PCSP awards grants.

x

x

x

       

x

 

Proportion of charter schools receiving PCSP grants?nationally and by state.

x

x

 

x

 

x

 

x

 

Reasons for differences in the proportion of funded charter schools among funded states.

x

x

x

   

x

 

x

 

Accessibility, usefulness, and timeliness of federal technical assistance to states, schools, planners.

 

x

x

x

x

x

 

x

x

Impact of the PCSP on state policies.

x

x

         

x

x

How do state PCSP grantees and charter school authorizers encourage the development of charter schools?

How states define "start-up" in establishing funding eligibility.

x

x

     

x

     

Number of states that allow planning grants; number of schools opened as a result.

x

x

 

x

         

How states award subgrants.

x

x

x

       

x

 

State requirements for use of federal funds.

x

x

 

X

       

x

Accommodations by charter school authorizers for low-income community charter applications.

x

x

x

         

x

Accommodations of states for low-income community charter applications for federal assistance.

x

x

             

Extent to which differences in subgrant amounts between and within states reflect differences in start-up needs.

x

x

x

X

x

   

x

x

Sustaining charters after federal funding expires.

 

x

 

X

       

x

Linkage with policies to intervene in failing schools.

 

x

x

         

x

Accessibility, usefulness, and timeliness of state technical assistance to schools and school planners.

x

   

X

x

x

   

x

Exhibit 1-2
RESEARCH QUESTIONS AND DATA SOURCES (CONTINUED)

 

 

 

Federal File Extraction

(Years 1-3)

PCSP Evaluation: Telephone Surveys

Other

Surveys

PCSP Evaluation: Other Data Collection

Charter Schools

(Year 2-3)

State Coordi-nators (Years 1 & 3)

Charter School Authorizers

(Years 1-3)

PCSP Charter Schools

Non-PCSP Charter Schools



RPP



SASS

Federal Interviews

(Years 1-3)

Site Visits*

(Years 2-3)

How do federally funded charter schools/school planners use their PCSP subgrants?

Average and range of federal subgrant awards?nationally and by state.

x

x

 

x

x

       

Proportion of start-up costs accounted for with the federal grant.

 

x

 

x

x

     

x

Kinds of planning and implementation activities supported by federal subgrants; extent activities are dictated by state authorizing legislation.

x

x

x

x

       

x

Start-up barriers overcome by federal funds.

 

x

 

x

x

x

   

x

Importance of access to federal grant money in obtaining or pursuing charter.

x

   

x

       

x

Differences between charter schools that receive PCSP grants and those that do not.

 

x

 

x

x

x

 

x

 

What are the characteristics of charter schools and the students and families who are involved with them?

Grade levels and student population served.

     

x

x

x

x

 

x

Educational approaches implemented.

     

x

x

x

x

 

x

Background and demographic characteristics of instructional and administrative staff.

     

x

x

 

x

 

x

Newly created vs. public or private school conversions.

 

x

x

x

x

x

x

 

x

Selection criteria for admission and the frequency/extent of waiting lists.

     

x

x

x

x

 

x

Demographic composition of charter schools, compared with the traditional public schools in their surrounding areas.

x

   

x

x

x

x

 

x

Targeting and accommodation of special populations.

x

x

x

x

x

x

x

 

x

Professional development opportunities available for teachers.

   

x

x

x

x

x

 

x

Exhibit 1-2

RESEARCH QUESTIONS AND DATA SOURCES (CONCLUDED)

 

 

 

Federal File Extraction

(Years 1-3)

PCSP Evaluation: Telephone Surveys

Other

Surveys

PCSP Evaluation: Other Data Collection

Charter Schools

(Year 2-3)

State Coordi-nators (Years 1 & 3)

Charter School Authorizers

(Years 1-3)

PCSP Charter Schools

Non-PCSP Charter Schools



RPP



SASS

Federal Interviews

(Years 1-3)

Site Visits*

(Years 2-3)

What flexibility provisions are charter schools granted?

Extent to which flexibility provisions granted by state and local laws are realized in practice.

 

x

X

x

x

x

 

x

x

Factors associated with successful and unsuccessful implementation of flexibility provisions.

 

x

X

x

x

x

   

x

Most critical flexibility provisions.

     

x

x

     

x

Parent involvement activities and parent requirements.

     

x

x

x

x

 

x

How do charter schools measure student performance, and are charter school students making progress on these and other measures?

Measures of accountability.

x

x

X

x

x

x

x

 

x

Measures of student performance for which charter schools are accountable and how assessed.

x

x

x

x

x

x

x

 

x

Extent to which students meet student performance goals.

   

x

   

x

   

x

Range and frequency of corrective action for charter schools that do not meet the student performance terms of their charters.

 

x

x

x

x

     

x

Students meeting or exceeding the state performance standards, if applicable.

 

x

           

x

Comparability of student achievement gains to those of their traditional public school counterparts.

 

x

           

x

Conditions under which charter schools improve student achievement.

   

x

x

x

x

   

x

Parent beliefs about charter schools and other schools.

     

x

x

x

   

x

* = Site visits include the written parent survey as a data source.

Note: Italicized research questions were added by the SRI evaluation team

In Exhibit 1-3, the charter school authorizer sample is displayed according to the type of agency. To facilitate the analysis of broader types of charter school authorizer, the local, state, institution of higher education, and other categories were created by collapsing similar types of agency.

There are limits to the conclusions that can be drawn from the purposive and therefore nongeneralizable sample of charter school authorizers. However, the sample supplies interesting information regarding the basic features of and variation across charter school authorizers. As is true across the country, most of the charter school authorizers in the Year 1 sample were local school districts (n=34). The other authorizers included in the sample were either state-level bodies (state boards of education, state educational agencies), institutions of higher education (IHEs), or independent chartering boards (n=14). The type of charter school authorizer appears to be closely linked to the number of schools chartered¾local agencies in our sample chartered fewer schools than other types of charter school authorizers (grouped as "nonlocal" agencies in most of the analyses for this report). Exhibit 1-4 displays the variation in average number of schools chartered by local and nonlocal authorizers and other descriptive statistics about them.

Exhibit 1-3
TYPES AND NUMBERS OF CHARTER SCHOOL AUTHORIZERS IN YEAR 1 SAMPLE

Type of Charter School Authorizer

n

Collapsed Category

n

Local school board

13

Local educational entity

34

Local school district

19

County board or office of education

2

State board of education

5

State entity

8

Chief state school officer

1

State educational agency

1

Multiple state entities

1

University or college

3

Institution of higher education

3

Community college

0

Independent or special charter school board

2

Other entity

3

Municipal government

1

Total sample

48

 

48

Exhibit 1-4
MEAN NUMBER OF SCHOOLS CHARTERED, BY TYPE OF CHARTER SCHOOL AUTHORIZER

Type

of

Authorizer*

Number of Schools Chartered

(mean)

Number of Authorizers in Year 1 Sample

Percent of Year 1 Sample (n=48)

Local

6

34

71%

Nonlocal

46

14

29%

* Because of the small number of "state," "higher education," and "other" authorizers included in the sample, the team collapsed these data into one category "nonlocal" and the data by type of authorizer are presented by "local" and "nonlocal" categories for most of the remainder of this report.

It is important to note some details that do not show up in Exhibit 1-4. First, approximately one-third of local charter school authorizers chartered only one school (none of the other types of agencies had chartered only one school). Second, only 9 percent of local charter school authorizers chartered 16 or more schools, compared with 64 percent of nonlocal authorizers. The Year 1 surveys did not explore why the number of schools chartered is so closely linked to type of authorizer, but from past charter school work the team has learned that local charter school authorizers tend to charter in their local attendance area, whereas nonlocal agencies charter schools across the state or region.

Telephone survey of charter schools receiving PCSP grants directly from ED. In states that have not applied for or do not receive PCSP grants, charter schools are eligible to apply for start-up and implementation grants directly from the federal government. A total of 23 charter schools either currently receive these grants or have in the past. A survey of PCSP-related questions was administered to representatives from these schools by a member of the study team. Fourteen charter schools responded, for a response rate of 61 percent.3

Extraction of information from federal files on PCSP grantees. With the cooperation and assistance of PCSP staff at ED, members of the study team retrieved information for the states, charter schools, and school districts that received PCSP grants before 1999. The federal files for 25 states, 21 charter schools, and 2 school districts were reviewed before Year 1 data collection occurred. The federal files provided background information on grantees that team members used for "precoding" several items prior to telephone surveys. In addition to federal files, team members also gathered information on states from various other (principally Web-based) sources, including state laws, regulations and guidelines, and charter school profiles. By supplying information on the survey from these other sources, the precoding step saved time for the survey respondents. As a result, especially in the case of PCSP-related items, respondents often simply confirmed or corrected the information gleaned from the federal files.

Interviews with PCSP staff and other government respondents. In an effort to place the PCSP in the context of other school reform and public school choice issues in the federal government, members of the study team conducted eight semistructured individual and group interviews with 12 Department of Education employees. Interviews were conducted with members of the PCSP program staff, staff of the Planning and Evaluation Service at ED, and those individuals from other ED offices involved in the Departmentwide school choice team. Although interviews were informally tailored to interviewees, team members sought to elicit general information about the nature and involvement of the respondents in the choice team, ED?s monitoring of the PCSP law, and the nature and types of technical assistance provided to the field. More specific information on the information gathered from these interviews is incorporated anecdotally in Chapter 2.

Focus groups with charter school operators, charter school planners, and charter school authorizers. Through an addition to the original evaluation design and in consultation with the U.S. Department of Education, members of the SRI study team designed and conducted a series of 18 charter school-related focus groups in January and February 2000. The focus groups were conducted to help the Department determine the format and content of future technical assistance activities. The groups consisted of individuals planning charter schools (six focus groups), persons who were already operating charter schools (nine focus groups), and representatives of authorizers that had awarded charters (three focus groups). Logistical support (scheduling, recruiting participants) was provided by the Charter Friends National Network and their local contacts. Exhibit 1-5 indicates where the focus groups of each type were conducted.

Exhibit 1-5
LOCATION AND TYPE OF FOCUS GROUPS

Location

Type of Focus Group

Charter School Planners

Charter School Operators

Charter School Authorizers

Boston, MA

x x  

Denver, CO

  x x

Detroit, MI

x x  

Miami, FL

  x x

Minneapolis, MN

x x  

Oakland, CA

  x x

Philadelphia, PA

x x  

Phoenix, AZ

x x  

San Antonio, TX

x x  

Data Collection and Analysis

Telephone surveys were administered in the summer and fall of 1999 by members of the SRI evaluation team. The surveys of PCSP grantees were relatively straightforward since the team had names, contact information, and job titles for state charter school coordinators and the leaders of charter schools that received PCSP funding directly from the federal government. Even where staff turnover had occurred, it was usually easy to identify the new respondent, schedule an appointment, and administer the survey. In a few cases, a second respondent was contacted to obtain information that was not provided by the first (typically because he or she was new to the job).

Identifying the appropriate respondents for the charter school authorizer survey was a larger and more time-consuming challenge. Team members usually had to call more than one office in a charter school authorizer to determine the right person to take the survey. The variety of types of charter school authorizer contributed to this situation. Even within local educational agencies (LEAs), the largest type of charter school authorizer in our sample and among the universe of agencies that have awarded charters, different kinds of personnel responded, ranging from the district superintendent to the district?s general counsel. Other respondents included the vice president of a state board of education and a district administrator who served part-time as the principal of a charter school.

Once data were collected and tabulated, a variety of data sorts and cross-tabulations were performed. Formal statistical estimates of the universe were not conducted on the Year 1 data, either because the entire universe was sampled (in the case of state coordinators and charter schools receiving PCSP funds directly from ED) or because the sample was not generalizable to the relevant universe (in the case of charter school authorizers). Hence, most of the information reported in the following sections is based on simple, descriptive statistics.

Organization of the Report

The Year 1 data in this report paint a comprehensive picture of a number of issues: the development of the Public Charter Schools Program, state and charter school authorizer perspectives on charter school flexibility and accountability, and the charter school activities of states and a sample of charter school authorizers. This picture, however, is also a "snapshot" of a rapidly evolving movement during a narrow time interval (summer and fall 1999).

Readers who would like more detail about future data collection activities for this evaluation are referred to Appendix B.

The report is organized according to the evaluation themes and findings from the first year of data collection and analysis. The broad themes are identified in Exhibit 1-6.

Exhibit 1-6
LIST OF THEMES, BY CHAPTER

Chapter 2: The Public Charter Schools Program (PCSP)

Evaluation Theme 1: Like the charter school movement itself, the Public Charter Schools Program has grown and matured since its implementation in 1994.

Evaluation Theme 2: Public Charter Schools Program funds flow as Congress and the U.S. Department of Education intended?as grants to states and then directly to charter schools as subgrants. Overall, 95 percent or more of PCSP funds are spent at the charter school level.

Evaluation Theme 3: In addition to providing financial support, the Public Charter Schools Program has provided national leadership in the charter school movement through policy-setting, research, networking, and technical assistance to the field.

Chapter 3: Flexibility

Evaluation Theme 4: States, in general, are working toward increasing the avenues available to charter applicants, either by expanding the types of agencies that can charter or by loosening limits on the numbers of charter schools permitted.

Evaluation Theme 5: Reports from states and charter school authorizers suggest that charter schools have certain freedoms that other public schools do not, but that they are also subject to many of the same regulations and requirements. Perceptions of these freedoms differ between state and charter school authorizer respondents, and among charter school authorizers.

Chapter 4: Accountability

Evaluation Theme 6: Both states and charter school authorizers are establishing processes to hold charter schools accountable, often focusing on student achievement.

Evaluation Theme 7: States and charter school authorizers have many corrective actions at their disposal; most have been used in moderation.

As this list of themes indicates, the key issues explored in the Year 1 report are the Public Charter Schools Program, charter school flexibility, and charter school accountability, mostly from the perspective of respondents to the state coordinator and charter school authorizer surveys. Information from federal interviews and the focus groups is incorporated anecdotally.

The report concludes with a chapter that summarizes the study findings as of Year 1 and provides information on the evaluation team?s plans for future rounds of data collection, analysis, and reporting.

Two additional reports are planned under this contract. They will include charter school-level data from multiple sources: (1) the National Study of Charter Schools conducted by RPP International; (2) the National Center for Education Statistics? Schools and Staffing Survey (SASS), which included a charter school supplement in its 1999-2000 data collection effort; (3) the SRI survey of charter schools in 2000-2001 and 2001-2002, and (4) site visits to charter schools in 2000-2001 and 2001-2002. In addition, the SRI evaluation team will conduct a substudy of student performance in charter schools compared with student performance in regular public schools in a limited number of states where such comparisons can be made under rigorous design considerations.


1 Although 36 states, the District of Columbia and Puerto Rico have enacted charter school laws, charter schools are not yet in operation in all of these states. In the remainder of this report, the District of Columbia and Puerto Rico are usually grouped with the other states.

2The National Study of Charter Schools tracked the development of charter schools nationwide from 1995 to 1999. This study produced four annual reports and a number of topical reports that are or will be available on-line. The fourth-year report is available at http://ed.gov/PDFDocs/4yrrpt.pdf.

3 Although the response rate is low, the respondents did not appear to have any characteristics that distinguish them from the nonrespondents (e.g., state, status of PCSP grant, etc.).

Previous Introduction | Table of Contents | Chapter 2 Next