 |
|
||
| |||
| |||
|
|
|
|
|
|
||||||||||
Evaluation of the Public Charter Schools Program: Year One Evaluation Report
Appendix B: FUTURE DATA COLLECTION AND ANALYSIS
Appendix B : FUTURE DATA COLLECTION AND ANALYSIS This appendix presents information on data collection activities scheduled for future rounds of SRI International?s evaluation of the Public Charter Schools Program. Exhibit B-1 shows the remaining data collection activities and other sources of information for the SRI evaluation. It also shows a basic timeline for these activities. Below the exhibit, the remaining data sources are described in full. Sampling for future data collection activities is described in the second section. The appendix concludes with an table that shows where each research question was addressed in the Year 1 report and how future data collection activities will address them. Exhibit B-1 : REMAINING DATA COLLECTION ACTIVITIES
Data Sources Telephone Survey of State Charter School Coordinators. The charter coordinators (or equivalent) of all 36 states with charter school legislation (plus the District of Columbia and Puerto Rico) were surveyed by phone in the summer and fall of 1999. What is likely to be a larger number of charter school coordinators will be surveyed again in the winter of 2002. As with the original administration, the survey instrument will consist of a combination of open- and closed-ended items on charter school-related operations, requirements, and flexibility, and the state?s PCSP grant. Members of the SRI team will administer the survey to all states with charter school legislation as of the end of 2001. Telephone Survey of Charter School Authorizers. A revised version of the Year 1 survey instrument will be administered to a representative sample of 150 charter school authorizers in the winter and spring of 2001. A version of the same survey will also be administered in winter 2002. The revised survey is similar to the exploratory survey administered in Year 1, with several adjustments. First, the number of items specifically focused on PCSP have been reduced to reflect the year 1 finding that charter school authorizers are not involved with the program except in minimal ways (e.g., fiscal agents). Second, because this instrument will be administered to a larger sample in years 2 and 3 than the sample surveyed in year 1, SRI International will use a telephone survey subcontractor (using a computer-assisted telephone interview system) for the administration of this survey. The Year 1 telephone survey was administered by SRI researchers. This change compels the SRI team to reduce the number of open-ended items. Third, in many cases, the closed-ended items in the revised survey were shaped by the responses to open-ended questions on the Year 1 survey. Telephone Survey of Charter Schools. One of the key sources of data for the remainder of this evaluation is the telephone survey of charter schools, which will be conducted in the winter and spring of 2001. Based on SRI?s review and update of RPP?s list of charter schools (current as of September 30, 1999), a representative sample of 600 charter schools will be drawn for this survey. A telephone survey subcontractor will conduct the data collection. The survey instrument will consist mostly of closed-ended items. The charter school survey will include items on all the school-level research questions for this evaluation and will document the basic characteristics of charter schools nationwide as of the 2000-2001 school year. The survey will also gather information about the use of PCSP funds at the school level, charter school experiences with flexibility and accountability, charter schools? relationships with their charter authorizers and states, and the ways charter schools assess student performance and other school outcomes.Site Visits. One round of exploratory site visits will be conducted in fall and winter 2000. Two additional rounds are currently planned for the second and third rounds of data collection (Spring 2001 and the 2001-2002 school year). Data collection will focus on student performance in charter schools and the accountability relationships between charter schools, parents, charter school authorizers, states, and other audiences. Members of the study team are currently responding to requests from ED to add a large-scale analysis of student performance in some or all of the six states selected for the charter school site visits. These analyses will include comparisons between charter schools and noncharter schools that serve similar populations. Parent Surveys. Written surveys will be administered to parents in conjunction with the Year 2 and Year 3 site visits. Extraction of Information from Federal PCSP Files and State Charter Laws. SRI?s access to the federal files on PCSP applicants and grantees and its review of state charter laws were very helpful activities for orienting team members to the charter operations, policies, and PCSP details in individual states and charter schools that received PCSP grants directly from ED. It also reduced the data collection burden imposed by this evaluation on telephone survey respondents and federal interviewees. Because the team will not be collecting data directly from PCSP grantees until the winter of 2002, the annual review of federal files and continued review of state laws will keep members of the study team up-to-date with the activities in their assigned states. In theory, the grantee files could be used to create an ongoing database to which queries could be addressed at any time. The evaluation team has been asked to create this database, which could then be transferred to ED for continued updating and maintenance. However, SRI's review of the files in 1998-99 found considerable inconsistency among them in terms of the material included. In the remaining years of the evaluation, team members will work with the PCSP staff to identify the most important and consistently available data fields for inclusion in a database that is likely to focus on very basic information, such as grantee contact information, annual grant amounts, dates and sources of correspondence, and presence or absence of grant applications and annual performance reports. Federal Interviews. Interviews with PCSP and Planning and Evaluation Service (PES) staff members at ED, and with congressional staff, will continue as needed. Charter School Data from RPP International and SASS. The SRI evaluation team has been granted access to data from the RPP study (based on four rounds of telephone surveys with the universe of charter schools). SRI will use these data to address the PCSP evaluation?s research questions. ED is also committed to having descriptive data from the universe of charter schools in every year of the evaluation while simultaneously minimizing the data collection burden faced by these schools. Hence, between RPP?s data collection in the 1995-1996 through 1998-1999 school years, the Schools and Staffing Survey (SASS) in 1999-2000, and SRI?s survey of charter schools in 2000-2001, the goal of annual school-level data collection will be met by coordinating federal data collection efforts and not unduly burdening charter schools with these requests. Mapping Research Questions To The Year 1 Report And Future Activities Exhibit B-2 displays a summary table of where each of the study?s research questions is addressed in the Year 1 report and notes that indicate how they will be addressed by future data collection activities. Exhibit B-2 : MAPPING RESEARCH QUESTIONS TO REPORT
Note: Future data collection and analysis activities are marked with bullets.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
