A r c h i v e d  I n f o r m a t i o n

School Reform and Student Diversity - September 1995

G. Study Aims and Study Questions

The goal of the Student Diversity Study was to identify, describe, and analyze exemplary school reform efforts for language minority students in grades 4 through 8 in three curricular areas: language arts, science, and mathematics. The focus on language arts curriculum was directed at grades 4 through 6 and the focus on mathematics and science curriculum was in grades 6 through 8. The study team identified, studied intensively, and wrote case studies of eight exemplary schools that offer state-of-the-art curriculum and instruction in one or more of the three curricular areas in a restructured school. More specifically, this research identified theory-based and practice-proven strategies to effectively teach language arts, mathematics, and science to students from linguistically and culturally diverse backgrounds.

While the limitations of study resources made it impossible to include sites representing a wide range of contexts (such as demography, geographic region, and language mix), schools were selected to reflect a variety of contexts. Exemplary schools selected for the study demonstrated innovative, high quality curricula in a reformed school context, as well as excellent language development programs for LEP students. In addition, case study sites implemented innovative school reform approaches beyond the standard observed in excellent but otherwise traditional school settings.1

1. Areas of Inquiry

Three areas were identified for specific inquiry for this study--1. Design of effective instructional strategies for culturally and linguistically diverse students, 2. Implementation of those strategies under various conditions, and 3. Impact of those strategies on students. Each of the three areas of inquiry is described below.

Design

Research into the design of effective instructional strategies required the research team to identify, examine, analyze, evaluate, and describe these practices as they are exhibited in exemplary schools and classrooms. The following Analysis Questions relate to program design; they were addressed by a range of data sources and analytical approaches, including a review of the literature, commissioned papers, site selection, case studies, and cross-site analyses.

Analysis Questions Related to Design

  1. Do cooperative and group learning strategies foster the success of LEP and formerly LEP students, especially in mathematics and science?

  2. How important are opportunities for instructional discourse to the educational attainment of LEP and formerly LEP students?

  3. What characteristics of effective programs are common across exemplary programs in language arts, mathematics, and science?

  4. In mathematics, how are higher order thinking skills developed in exemplary programs?

  5. In science, how do exemplary programs provide hands-on experiential opportunities for learning science?

  6. What are the special issues relating to acquisition of English writing skills for language minority students; what are effective practices in teaching English writing to LEP students?

  7. How are the social and cultural contexts (e.g., traditions, norms, values, and aspirations) of linguistically and culturally diverse students drawn upon in exemplary programs?

  8. What are the characteristics of successful models that incorporate student culture when there are multiple and diverse cultures represented in a school?

  9. How do successful programs structure the articulation of curriculum and instructional strategies between elementary and middle schools and middle schools and high schools?

  10. How do successful programs ease the transition from elementary school to intermediate school or from intermediate school to high school for culturally and linguistically diverse students?

  11. What are the characteristics of effective transition programs for LEP students in grades 4-6?

  12. Do exemplary transition programs serving students who continue to develop their primary language literacy skills in grades 4-6 differ systematically from exemplary transition programs in which students do not continue to develop primary language literacy in grades 4-6?

Implementation

Many reforms and models fail live up to their promise when implemented under the real conditions of American schools. Consequently, the study sought to identify exemplary practices that have demonstrated their effectiveness over time. Field investigations focused on uncovering factors and conditions that helped bring the program into being or had to be overcome to achieve success. The following Analysis Questions about the implementation of reform were informed by a range of data sources and analytical approaches.

Analysis Questions Related to Implementation

  1. What factors helped to initiate, develop, and sustain the reform?

  2. What were the incentives for the reform?

  3. What barriers were encountered and how were they overcome?

  4. How do diverse conditions of demography affect implementation?

  5. What is the history of reform efforts in the district?

  6. What are the prevailing community attitudes toward educating linguistically and culturally diverse students?

  7. How is the school organized, governed, and managed? What is the school context for implementation of reform?

  8. How is the reform program staffed? What is the training and preparation of the staff?

  9. How do the schools or districts recruit, hire, and retain appropriate staff for the program?

  10. What is the impact of differences in the nature of the cultural and linguistic minority population--whether immigrant, migrant, or second generation?

  11. How do reform programs differ from previous practice in that school or district?

  12. What resources, both human and financial, were required to develop, implement, and sustain the reform program?

  13. How was research and research-based information applied in the reform program?

  14. What policies and practices at the federal, state, district, and school level helped or hindered reform?

Impact

A major challenge for this research was to collect data about how new instructional approaches affect student learning. The duration of this research project was too short--and the budget was too limited--to conduct a longitudinal study of student outcomes. Moreover, it was not possible to gather data that would have allowed us to compare student outcomes across sites for several reasons. LEP student test scores often are hard to come by in schools and are generally not comparable across sites because LEP students are often not given the standardized tests that districts or states require of most students. The transiency and mobility of LEP students is another factor that makes comparable data very difficult to obtain. Therefore, the study could not demonstrate quantitatively that the eight case study sites are exemplary in the sense of demonstrated evidence of significantly higher student achievement scores. Nonetheless, relying on local evaluation data, the research team was able to examine some outcome measures. Those data were used in the analysis of an individual case study site but were less helpful in the cross-site analysis. However, data on program outcomes was gathered from interviews and structured focus groups across sites. These data allowed a great deal of cross-site analysis and provided much of the insight for this research. The following Analysis Questions relate to both student and program outcomes; they were addressed by a range of data sources and analytical approaches.

Analysis Questions Related to Impact

  1. How do reform programs assess their own progress in improving student learning?

  2. Do schools have evidence that student learning outcomes improved?

  3. What is the overall assessment of the program, its strengths and weaknesses?

  4. How can assessments be used to refine reforms?

  5. What are the anticipated and unanticipated benefits derived from and difficulties encountered in the implementation of reform programs?

  6. What was the role of research, research-based knowledge, and other information in program assessment?

2. Case Study Research Questions

Five Case Study Research Questions were derived from the three Areas of Inquiry and the corresponding Analysis Questions discussed above.
  1. What is the context for reform? What factors helped to initiate, develop, and sustain reform? What were the major barriers to reform and how were they overcome? What are unique programmatic and demographic conditions?

  2. What is the design of the reform and how is it implemented? Restructuring: What elements of restructuring are in place? How is the program organized and governed? How is the program staffed? How are students grouped for instruction? Curriculum: What type of curriculum is used? How is it integrated across content areas? How is it developed? Language Acquisition Approach: What approaches are used for language instruction? How does the program build on the cultures of the students?

  3. What is the role of research-based information on the reform? What has been the role of research and research-based information in designing, implementing, and evaluating the reform? the language acquisition program? the curriculum?

  4. What resources are required for the reform? What were the sources of financial support for the program: federal, state, local, or private? What is the approximate cost of the reform: development, operation, evaluation, and training?

  5. What is the impact of the reform? What has been the approach to assessing the student learning outcomes from the reform? What are the results of the assessment? What are the strengths and weaknesses of the reform?
For each of these questions, the researchers identified Operational Elements for Data Collection. Data gathered from the schools in response to the Case Study Research Questions and the corresponding Operational Elements for Data Collection are presented in the Cross-Site Analysis Tables in the Appendix to this volume. Tables 1 through 8 display data on the elementary grade case study sites. Data from the middle grade case study schools are presented in Tables 9 through 16. The first Case Study Research Area, The Context for Reform, was divided into two parts: Demographic Conditions (Tables 1 and 9) and Factors Affecting Implementation (Tables 2 and 10). The second Case Study Research Area, Design and Implementation of the Reform, was divided into three parts: School Restructuring (Tables 3 and 11), Curriculum and Instruction (Tables 4 and 12), and LEP Student Program (Tables 5 and 13). Case Study Research Question 3 is addressed in Tables 6 and 14; Question 4 is addressed in Tables 7 and 15; and Question 5 is addressed in Tables 8 and 16.

3. Key Research Questions for Cross-Site Analysis

The study examined eight exemplary sites with high concentrations of limited English proficient (LEP) students in order to identify the features of these sites that make them outstanding. The study focused on the school as a whole system, rather than limiting the research to descriptions of programs for LEP students. This more comprehensive focus is important for two reasons. First, in the standard school context, LEP programs tend to be separated from the main flow of classes. Such separation contributes to the isolation of LEP students, the difficulty of making transition to standard classes, and a tendency for lower expectations and limited curriculum for LEP students.

Second, based on our previous study of exemplary LEP programs, we found that the challenge of language diversity requires more than good teachers or a good instructional approach: most aspects of schools have to be engaged in reform. We searched for and selected schools for case studies exemplary schools that were undergoing systemic reform, in which all aspects of schools were open to change. The overarching question for the cross-site analysis was thus:

  1. What are the key characteristics of systemic reform evidenced by the exemplary schools whose students include LEP students? This question led the researchers to analyze how the exemplary schools implemented systemwide change. In particular, the researchers looked at how the schools integrated three critical elements--school restructuring, strategies for learning, and approaches to language development and English language acquisition--to create systemic reform. Though the core triad of school restructuring, learning strategies, and language program were interlocked in the exemplary sites, they define three research questions which we examined in turn in the cross-site analysis.

  2. How did these sites design their language development programs for LEP students? In particular, the researchers looked across the exemplary sites to identify the range of design options that the exemplary sites used in order to respond to their students’ language development needs, their community and school demographic situation, district policies, and their school vision. In addition, researchers examined the choices made at these sites in regard to the use of primary language models, English-based program models, and programs for recently arrived immigrants. Finally, the research team explored the critical issue of how the exemplary sites approached the transition of LEP students to English language instruction.

  3. What strategies did these sites employ in order to maximize their students’ learning? In particular, the research team focused on the curricular and instructional strategies employed in language arts in grades 4 through 6 and mathematics and science in grades 6 through 8.

  4. In what ways did the exemplary sites organize to maximize LEP (and all ) students’ learning? In response to this question, the research team looked at the variety of ways that the exemplary sites aimed to redesign schooling. The analysis focused on the most mature and well-implemented aspects of restructuring selected from the sites, rather than evaluating the extent to which any one school had accomplished the full range of potential restructuring elements. This approach provided empirical information on what school designs might enhance the development, maintenance and renewal of active learning environments.

    In addition to the three core questions, four additional questions guided the cross-site analysis. These questions relate to the outside support structures that impacted the exemplary schools: external partners, districts, states, and the federal government.

  5. What role did external partners play in the development of the exemplary schools? The researchers identified which areas of help from an external partner and what types of relationships had the greatest impact on creating systemic reform for diverse schools.

  6. What types of district support aided the development and maintenance of the exemplary schools? The study did not try to select exemplary districts. Exemplary schools were chosen without regard to their relationship to their districts. The study team identified district policies and programs that supported the exemplary schools for LEP students.

  7. How did state policies and programs support the exemplary schools? The researchers reviewed state policies and programs which impacted how the exemplary sites were able to address the learning needs of their LEP students.

  8. What federal policies and programs aided one or more of the study schools? Researchers examined the ways that federal policies and programs supported LEP student learning at the exemplary schools.

1For more information on the site selection process, see Section I: Cross-Site Analysis. For more complete information on the research design and methodology, see School Reform and Student Diversity Study, Volume III: Technical Appendix .
-###-


[Summary Review of the Literature (part 2 of 2)] [Table of Contents] [Case Study Summaries]