Data & Research EVALUATION OF PROGRAMS
Evaluation of the Magnet Schools Assistance Program, 1998 Grantees: Year 1 Interim Report
Executive Summary

For nearly four decades, magnet schools have been an important element in American public school education, offering innovative programs not generally available in local schools and providing opportunities for students to learn in racially diverse environments. Federal support for magnet schools began in 1972 with the Emergency School Assistance Program and continued under the Emergency School Aid Act (ESAA), which authorized grants for planning and implementing magnet schools in districts that were desegregating their schools. ESAA funding ended in 1981, but support for magnet schools resumed in 1984 with the authorization of the Magnet Schools Assistance Program (MSAP).[1] In reauthorizing the program in 1994, Congress concluded that it is in the best interest of the Federal Government to continue to support school districts in implementing court-ordered or voluntary desegregation plans, ensure equitable access to quality education for all students, and assist local educational agencies in implementing innovative programs that contribute to systemic reform efforts.

From 1985, when the MSAP began awarding grants, through 1998, 379 MSAP grants have been awarded to 171 school districts in 35 states and the District of Columbia. Do these federally funded magnet schools help school districts bring students of different races, ethnicities, and socioeconomic backgrounds together? Do MSAP-funded magnet schools offer innovative programs and promote systemic reform? Do these schools help students increase their academic achievement?

To answer these and related evaluation questions, the American Institutes for Research (AIR) was awarded a four-year contract in 1998 to evaluate the MSAP. This report is the first in a series presenting the results of our evaluation, and subsequent reports will appear in 2001 and 2002. Our focus is on the 57 projects funded by the U.S. Department of Education (ED) for three-year MSAP grants in 1998. These projects support magnet programs in public elementary, middle, and high schools throughout the country.[2]

Districts that apply for MSAP grants must describe the desegregation and achievement objectives that they will set for their MSAP-supported schools and the ways in which their projects will support systemic reform efforts and provide innovative instruction. MSAP grantees must also prepare annual performance reports. Our Year 1 report is based on information we gained from:

  • reviews of the MSAP applications for the 57 projects
  • the data that grantees provided in their first year performance reports (for 1998-99) to ED
  • the Common Core of Data and other extant data sources
  • data we gathered from the 57 projects through interviews, surveys, and data requests during the 1999-2000 school year

In this Year 1 report, we provide details about the types of districts that are funded by MSAP and the projects that they establish. We describe the desegregation and achievement objectives that the projects have set, but the data were not yet available to report on the extent to which they have met those objectives. Our Year 2 report will include that information. Our Year 1 report also includes descriptive information about the systemic reforms that MSAP projects are designed to support, the innovative methods and practices that those projects are implementing in federally funded magnet schools, and the support that is available to MSAP projects from ED and other sources.


Findings in Brief

About the types of schools, programs, and students included in MSAP projects:

  • MSAP-funded magnet programs are most commonly found in elementary schools. Among the 292 schools supported by MSAP grants in 1998, 60 percent are in elementary schools, 24 percent are in middle schools, 14 percent are in high schools, and 2 percent are in schools that serve combined levels such as grades 4-12.

  • MSAP-supported schools most often offer programs to all of the students in a particular school or grade, rather than only some students. Of the 292 schools supported by MSAP in 1998, 89 percent have whole school programs (i.e., they are offered to entire schools or specified grades). The remaining 11 percent feature programs-within-schools (PWSs) (i.e., they are offered to only some students, such as those who elect to take courses in a performing arts or vocational skills program) and are generally found in high schools. Thirty (30) percent of the MSAP high schools are PWSs; only 6 percent of the elementary schools are. This is a change from 1992, when 38 percent of all magnet schools (i.e., both MSAP-supported and other magnet schools) had PWSs.[3]

  • About three quarters of students enrolled in MSAP schools are members of racial or ethnic minorities. In 1998-99 (the first year of the 1998 MSAP grants), over 165,000 students were enrolled in MSAP schools, and, as might be expected in schools that are trying to desegregate, 74 percent of the students in whole school programs were minority students; 66 percent of those in PWSs were minority students. Black students accounted for a larger percentage of students in MSAP schools than in non-MSAP schools in 1998-99; students from other racial-ethnic minorities were represented in about equal numbers in MSAP and non-MSAP schools.

  • About 70 percent of MSAP schools operate Title I programs (i.e., federal programs serving disadvantaged students), compared to about 50 percent of U.S. public schools overall. School-wide Title I programs, available to schools with 50 percent or more of students qualified to receive Title I support, are found in twice as many MSAP schools as in U.S. public schools overall (54 percent and 27 percent, respectively).

About the context for these MSAP projects--the districts in which they operate:

  • A little more than half (54 percent) of the districts funded by MSAP in 1998 operate under a voluntary desegregation plan (i.e., one approved by their Board of Education). The remainder (46 percent) operate under a required desegregation plan--that is, one required by a court, state agency, or ED's Office for Civil Rights.

  • Of the 57 districts that were funded by MSAP in 1998, nearly all are located in metropolitan areas and have large student populations. Two thirds (66 percent) of the MSAP districts serve primarily central city students in large metropolitan areas, and almost one third (32 percent) serve mainly suburban students. Only 10 districts have fewer than 10,000 students; 11 districts (19 percent) have more than 50,000 students.

  • On average, minority students represented nearly two-thirds (62 percent) of students in MSAP districts for the 1997-98 school year (the year before the 1998 MSAP grants were implemented). Black students were the largest minority group (35 percent) among all students, then Hispanics (21 percent), Asian and Pacific Islanders (5 percent), and American Indian or Alaskan Natives (less than 1 percent).

  • In 1997-98, two thirds of the MSAP districts had more minority students than white students. Black students constituted the majority in 21 percent of the MSAP districts; Hispanics represented the majority in 10 percent of the MSAP districts.

  • In 1997-98, the average proportion of minority students attending minority-isolated schools in MSAP districts was four to five times greater than the average for school districts nationwide. Minority-isolated schools are those in which minority group students constitute more than 50 percent of the school enrollment. The comparisons were based on data from the 1997-98 Common Core of Data.

  • Almost two thirds of MSAP districts maintain waiting lists for one or more of their MSAP programs. Nearly all of these districts keep the lists into the school year and admit students if spaces become available.

  • In trying to attract students, more than half of the MSAP projects face strong competition from other schools (public and private) in their districts. In districts with required desegregation plans, many MSAP schools compete with other magnet schools, funded by the district, for students.

About the desegregation objectives that projects have established for their schools and the recruitment strategies used to help schools meet those objectives:

  • Two thirds of MSAP schools have the desegregation objective of reducing minority group isolation, while 10 percent are trying to eliminate it, and 6 percent are trying prevent it. Schools that have the objective of reducing minority group isolation have minority enrollments that exceed 50 percent, and their objective is to lower the percentage.

  • Some MSAP districts operate under required desegregation plans that establish a different standard for defining minority group isolation. Because six MSAP districts have such desegregation plans, we have identified their 56 schools (19 percent of all schools) as having uniquely required desegregation objectives.

  • To attract students to magnet schools, MSAP projects have full-time staff who devote their time to recruiting students. In districts with required desegregation plans, the average full-time equivalent is 2.4 persons; in districts with voluntary plans, 1.9 persons.

  • MSAP projects develop a wide array of recruitment and outreach materials to inform students, parents, and the public about their MSAP programs. Recruitment strategies include distributing brochures and other informational materials, using internet sites to publicize MSAP schools, providing school tours, and making visits to other schools.

  • One third of all MSAP districts focus their recruitment efforts on all eligible students. Districts with required desegregation plans focus more of their efforts on targeted groups of students than do districts with voluntary plans.

  • Few MSAP programs base student admissions on test scores, demonstrated skills, or past academic records. Although critics sometimes accuse magnet schools of using admission procedures that favor high-achieving students, only 10 percent of MSAP programs establish performance standards.

About the extent to which MSAP districts and projects support systemic reform:

  • Almost all of the MSAP districts report that they place a major emphasis on standards-based reform. More than 80 percent of MSAP districts report that they place a major emphasis on establishing high standards for students and on aligning curricula with standards. When compared with large high-poverty districts in 1998-99, MSAP districts appear to place somewhat more emphasis on new approaches to curriculum and instruction (technology and reform models). Many MSAP districts also report placing major emphasis on approaches to curriculum and instruction.

  • Most MSAP projects report that the themes and goals of their magnet schools and programs are consistent with their state and district standards. About 82 percent of the MSAP Project Directors report a strong influence of standards in mathematics and in language arts; 64 percent report a strong influence in science; and 55 percent report a strong influence in social studies. Nearly 90 percent of Project Directors also indicate that the state and district standards match the goals of their MSAP schools to a great extent.

  • Most MSAP schools have set quantifiable goals for the academic growth of their students. About 90 percent of the MSAP principals report that their schools have set quantifiable goals for academic growth in reading and mathematics; about half have set such goals in one or more other subject areas. For schools with poor student performances, accountability most often takes the form of technical assistance. About 16 percent of MSAP schools have been identified as needing improvement.

  • MSAP projects coordinate with other federal and district programs in varying degrees. In planning MSAP activities, MSAP Project Directors report that they interact most frequently with district curriculum coordinators (about 65 percent report that they interact to a great extent) and directors of choice programs (almost half report interacting to a great extent). The Project Directors interact somewhat less with coordinators of other federal and district programs, and considerable variation was found across districts.

  • MSAP project staff provide technical assistance to their MSAP schools on project planning and budgeting. Nearly two thirds of the MSAP Project Directors report providing such assistance at least once a week during the 1999-2000 school year. District-level MSAP project staff also provide frequent technical assistance on student recruitment, principal leadership, and teacher motivation to MSAP schools.

About the innovative practices and methods in MSAP schools:

  • MSAP schools have adopted a variety of themes, or focus areas, with many focusing on science, technology, the arts, communication, and careers. These themes help nurture the development of a unique school identity and promote a coherent curriculum and program of instruction. Many MSAP schools have also adopted programs based on externally developed comprehensive models, such as International Baccalaureate, Montessori, and Success for All.

  • Many MSAP schools (especially Title I schools) have introduced changes in the instruction of mathematics, science, language arts, and social studies since receiving the MSAP award. Overall, about half of MSAP principals report that their schools have changed their instructional practices in mathematics and science; nearly 60 percent, in language arts; and about 40 percent, in social studies. These changes frequently involve giving additional emphasis to higher order thinking, including problem solving, reasoning inquiry, and applications.

  • MSAP schools vary considerably in establishing high (specified) standards for students. Among eight required practices for all students (suggested in the literature on effective schools), nearly all MSAP elementary schools require two practices: at least one hour of reading and one hour of math each day. A smaller but still substantial proportion of MSAP schools requires each of the other specified practices; however, considerable variation was found across MSAP elementary schools. Similar results were found for increasingly rigorous practices at the secondary levels.

  • MSAP schools vary in the sense of professional community among their teachers. Principals of most MSAP schools strongly agree that their teachers care about students, and about half strongly agree that their teachers support each other, are willing to put in extra hours, and collaborate. Fewer principals strongly agree that rules are enforced or that there is coordination across grades, and elementary principals tended to report more positive conditions that did secondary principals.

  • A majority of MSAP principals report positive school climates; however, poverty and students arriving unprepared to learn are serious problems in some of the MSAP schools. More than 30 percent of MSAP elementary, middle, and high school principals report that poverty is a serious problem in their schools, and more than 20 percent report that students arriving unprepared to learn is a serious problem. Other problems cited include lack of parental involvement and student transience, which are regarded as serious problems by principals in 10 to 20 percent of the MSAP schools (with percentages increasing by level).

  • MSAP schools have reportedly developed programs that respond to individual students' needs. For example, more than half (57 percent) of MSAP school principals report using a parent satisfaction survey for school self-assessment, and about 60 percent report using a student survey. Nearly all MSAP schools provide additional time for low achievers, individualized instruction, and tutoring by non-school staff, having adopted such practices either before their MSAP award, or, for between 31 and 41 percent of MSAP schools, either adopting the practices after their award or planning to adopt the practices in the remaining two years of the grant period. High percentages of MSAP schools also include students with Individual Education Plans (IEPs) and students with limited English proficiency (LEP) in their regular MSAP activities.

About the measurement of student achievement in MSAP schools:

  • Some of the achievement objectives set by MSAP grantees are either changing or unclear. The overall patterns in the achievement objectives are clear and are described in our report. Because of a variety of factors, including the imprecision of objectives in MSAP applications, changes in state and district assessment programs, projects' reliance on alternative assessments that were to be developed in the early stages of their projects, and delayed submission of student achievement data to ED, however, the objectives may not be those on which student achievement is measured in all projects.[4]

  • MSAP projects propose multiple achievement objectives for the students in their magnet schools. The average numbers of objectives per program[5] are 8.8 for elementary school programs, 9.7 for middle school programs, 11.9 for high school programs, and 12.8 for programs in six combined-level schools (e.g., grades 6-12). Many of these objectives pertain to multiple grades within a school; most objectives are to be tracked separately for each grade.

  • Virtually every MSAP program has objectives for student achievement in language arts and mathematics. Substantial numbers of programs have objectives pertaining to achievement in other academic subjects, performance and applied learning skills, job-related skills, career awareness, and behaviors relating to current and future academic success.

  • Standardized test scores are by far the most common measure of student achievement in MSAP schools. Over 95 percent of the MSAP programs at all three levels (elementary, middle, and high school) based their achievement objectives on standardized tests; over 70 percent of the MSAP programs specified alternative assessments. Other, less common measures include benchmarks in students' educational careers, course-taking, and grades.

  • Most MSAP programs have at least some achievement objectives that are framed in terms of steady improvement over a baseline level. A substantial number of objectives do not make explicit reference to prior performance levels of students in the same schools, however. Some of the objectives call for MSAP students' performance to compare favorably with that of an external reference group that is measured at the same time. Other objectives do not include baseline data because the achievement measure is still being developed or is in transition.

  • Most MSAP objectives do not involve direct comparisons between the achievement of MSAP students and external reference groups. When programs do specify such objectives, the most common comparison groups are similar non-magnet students within the district and district averages.

About the activities and purchases enabled by MSAP and ED's role in awarding grants and assisting grantees:

  • About one third of the MSAP projects devoted the first MSAP grant year to planning for one or more of their schools. That time was devoted primarily to providing professional development for staff members.

  • More than half of the MSAP projects are looking to their districts for program support after the MSAP grant period ends. Other strategies for continuing MSAP programs include applying for 2001 MSAP grants, seeking other state and federal grants, and developing self-sustaining schools that do not require outside support.

  • MSAP projects spend about 45 percent of their budgets on staff salaries and fringe benefits, about 20 percent on equipment, and about 20 percent on supplies. This is also reflected in MSAP principals' reports that the most important thing MSAP grants enable them to do is provide additional staff for their schools.

  • On average, MSAP grants provide about $300,000 to each MSAP school per year. This amount varies widely across the projects; for example, in 1998-99, school budgets ranged from $11,000 (for a school with a planning year) to more than $800,000 (for a school establishing a technology-based program).

  • ED awards MSAP grants in a competitive process and utilizes a review panel that assesses the strengths and weaknesses of each application. Three panelists review each application, and points are awarded within specific categories (e.g., Plan of Operation, Quality of Personnel). Awards are made from a ranking list.

  • The primary source of technical assistance for MSAP projects is ED's Office of Elementary and Secondary Education (OESE). OESE staff assist districts by providing conferences and workshops and by responding to individual telephone calls. Nearly all of the MSAP grantees assess the technical assistance as very useful.


Future Reports

As noted above, this Year 1 report describes the objectives for desegregation and achievement that the 57 1998 MSAP grantees have set for their projects, and it provides information about the reform efforts and innovative methods and practices that are being implemented to support those objectives. Our Year 2 report will report on our findings as to the extent to which the desegregation and achievement objectives have been met. Our Year 2 report will also include data from later performance reports submitted to ED and information from another round of data collection (interviews, project surveys, and principal surveys administered during 2000-01) from the 57 projects.

An important component of our future reports will be Case Studies based on visits we made to eight MSAP projects in spring 2000 and will re-visit in spring 2001. In addition to descriptions of our interviews and observations at these project sites, our reports will include data from surveys of principals in comparison schools, surveys of teachers in both MSAP and comparison schools, and student focus groups in the Case Study districts. We also will gather student-level data to permit more rigorous analyses of student achievement in these districts. All of these data sources will enable us to provide in-depth reports of the MSAP projects and their progress in attaining their objectives.


[1] 20 U.S.C. 7202

[2] In this report, we refer to the MSAP (the U.S. Department of Education source of federal funding and assistance for magnet schools), the 57 school districts receiving MSAP grants in 1998, the projects that the districts developed with MSAP funds, and the MSAP schools (with one or more magnet programs) supported by the projects.

[3] Steel, L., and R. Levine. Educational Innovation in Multiracial Contexts: The Growth of Magnet Schools in American Education. Washington, DC: U.S. Department of Education, 1994.

[4] We will describe changes in achievement objectives and provide data on the extent to which objectives are met in our Year 2 report.

[5] We report on the number of magnet programs that have at least one achievement objective in a particular category, rather than the number of objectives of each type because MSAPprojects vary widely in the number of objectives they report for their programs.

Return to Report Contents


 
Print this page Printable view Send this page Share this page
Last Modified: 01/07/2004