Current information about this program can be found under the Education Research program.
Florida State University
Principal Investigator: Dr. Douglas Harris
Assessment Project: Assessing Teacher Effectiveness: How Can We Predict Who Will be a High Quality Teacher?
Research shows that teacher quality is associated with student outcomes. Most states certify teachers based on a combination of college coursework, degree attainment, and performance on pedagogical and/or content-based examinations. Despite their importance, however, relatively little is currently known about the ability of these assessment schemes to identify the most effective teachers-those who will contribute the most to student learning. This project is designed to address the question of what assessments and strategies can be used to select teachers who are effective in raising student achievement.
The first part of the study examines associations between teacher characteristics and student outcomes using data on students in grades 3 through 10 and their teachers. These data are drawn from a Florida statewide database that includes longitudinal information on nearly the entire population of teachers and students in the state. Statistical modeling procedures will be applied to the data in order to derive "value-added" scores for more than 5,000 teachers; these scores represent estimates of the contributions that individual teachers make to student achievement. To address questions regarding what teacher characteristics best predict effectiveness, associations between these effectiveness scores and a set of teacher characteristics will then be examined. Teacher characteristics to be included in these analyses include measures of general verbal and quantitative skills, college course-taking, and certification test scores. The size of the database will allow for detailed breakdowns for sub-groups of students, teachers, and school.
The second part of the project involves the use of a mixed methods approach to compare principals' opinions of the factors that predict teacher effectiveness with the measures found to be the best predictors of value added in the first set of analyses. In addition, the degree to which principals can predict and identify which of the teachers in their own schools produce the largest value added to student outcomes is being examined. For this part of the project, value-added scores are calculated for each teacher, as in the first part of the project, and are compared with coded data from principal interviews.
The primary goal of this research is to shed light on both the merits of general strategies used to select high quality teachers (professional judgments versus standardized measures) and on the specific assessment criteria that are most highly associated with student outcomes. The unique database also allows for methodological advancements in the calculation of value added, which is becoming an increasingly important tool for research on teachers.
Principal Investigator: Dr. Douglas Powell
Development Project: Professional Development in Early Reading
Children from economically disadvantaged families often have lower levels of knowledge and skills at kindergarten entry than do children from non-poor families—skills that have been found to predict achievement during the school years. Well-planned pre-kindergarten programs have been shown to be effective in narrowing these kindergarten-entry gaps in knowledge and skills. In particular, high quality preschool interventions focused on early reading have been found to improve skills that are predictive of subsequent reading ability. Currently, however, there are few research-based models of professional development to improve the quality of early reading instruction in general; less work has been conducted to develop and test professional development models for teachers in rural areas in which easy access to coaches or master teachers may be limited.
The purpose of the current study is to examine the relative effects of two variants of a semester-long professional development intervention incorporating coaching, workshops, and case examples to improve teacher practices and child literacy outcomes. Participants include 80 Head Start teachers and six children from each teacher's classroom (for a total of 480 student participants). Teachers are randomly assigned to receive either on-site or web-based remote coaching. Teachers in the on-site condition receive classroom visits from an expert coach twice each month, with immediate feedback designed to improve early reading instructional practices. Teachers in the remote coaching condition use digital video to record themselves twice a month while engaged in specific activities designed to support early reading and writing development. Videos are mailed to the coaching program; coaches then provide feedback to teachers via the program's website. Feedback includes commentary with links to cases demonstrating best practices. Case-based hypermedia that feature videos of best practices with links to explanatory text and additional instructional resources are provided for teachers in the remote coaching conditions; these cases are also available to teachers in the on-site coaching condition through a website, but are not emphasized. Teachers in both conditions participate in workshops designed to introduce and review key information about best practices for supporting early reading and writing development.
Observational assessments of teacher practices and classroom characteristics, and direct assessments of children's literacy skills will be conducted prior to and following the coaching intervention. Classroom-level observational data will be collected in the semester prior to the coaching semester as well as at the beginning and end of the semester during which coaching occurs, permitting analysis of both the effects of coaching on teachers' practice and the differential effects of the two coaching interventions. Child data, collected before and after the intervention using standardized measures of language and early reading competence, will be used to determine whether there are differential effects of on-site versus remote coaching on children's language and literacy outcomes. The overall goal of this project is to develop efficient professional development methods that enable teachers to effectively implement best practices in early reading instruction that enhance the language and literacy skills of children from economically-disadvantaged backgrounds as they enter kindergarten.
Principal Investigator: Dr. Richard Buddin
Assessment Project: Teacher Licensure Tests and Student Achievement
One approach to improving the quality of instruction in U.S. elementary and secondary schools has been to screen out potential candidates for teaching positions based on teacher licensure tests, and teacher testing in general. To date, however, little is known about associations between teacher test performance and student outcomes. The purpose of this study is to examine the validity of licensure tests as assessments of teacher skills and knowledge, with a focus on whether scores on these tests are associated with teacher effectiveness, as reflected in student achievement test performance.
Test data are being obtained from three cohorts of graduates of California State University (CSU) teacher training programs. These data are being linked to achievement test data from third through twelfth grade students in approximately six Southern California school districts. In total, data from nearly 6,000 teachers and their students is being obtained.
Several research questions are addressed in this study. First, background information on current state teacher standards, and on how and why the tests in current use were selected, are being obtained from the State Department of Education website and from interviews with state officials. Second, technical characteristics of the three teacher tests used in California are being examined using existing technical reports and test score data; criteria put forth by the National Research Council serve as a framework for this review. To examine whether multiple forms of evidence converge in identifying the same individuals as high or low in areas of teaching expertise, available validity evidence is being obtained, as well as information on correlations among scores on the three tests and between test scores and grade point averages in teacher education programs. To assess whether test scores differ systematically for different subgroups of teacher candidates, initial and eventual passing rates are being examined by race/ethnic group, and by program type (i.e., traditional teaching programs versus alternative certification programs).
Finally, to determine if scores on licensure tests are a valid measure of teacher effectiveness, value-added modeling is being used to examine whether these scores and other teacher characteristics are associated with gains in student performance on achievement tests, independent of background characteristics of students assigned to particular teachers. Three years of student data are being collected for the first two cohorts of teachers included in this study, and two years are being collected for the final cohort. Analyses will be conducted separately for multiple- and single-subject credential holders; doing so will also allow a comparison of student achievement gains for teachers with single-subject credentials who demonstrate subject knowledge by passing required single-subject proficiency tests, versus teachers who are allowed to waive testing and to demonstrate proficiency through completion of approved undergraduate coursework. The study will also address the question of whether middle school teachers with more subject-matter expertise produce greater pupil learning gains than those with multiple-subject credentials (more typical of elementary-level teachers).
Findings from this study will have important implications for policymakers attempting to measure and ensure the quality of beginning teachers.
University of North Carolina, Chapel Hill
Principal Investigator: Dr. Virginia Buysse
Development Project: Improving Teacher Quality to Address the Language and Literacy Skills of Latino Children in Pre-Kindergarten Programs
Latino children lag behind their peers when they enter kindergarten and the gap in academic achievement appears to widen with age. The pre-kindergarten period is a critical time for Latino children, many of whom face the difficult task of developing language skills in a new language while experiencing the beginning-to-read process. Very little is known, however, about the effects of pre-kindergarten schooling on Latino children who are English language learners, and teachers are largely unprepared to work with this population. The purpose of this study is to develop and test an intervention designed to improve the quality of teaching practices related to literacy and language learning among Latino children enrolled in North Carolina's More at Four Pre-Kindergarten program for at-risk children.
More at Four currently serves over 10,000 low-income children, of which approximately 20% are Latino children. A total of 60 More at Four teachers and classrooms in which Latino children are enrolled will be recruited and randomly assigned to either a treatment or comparison group; a minimum of 240 Latino four-year-olds and their families will be recruited to participate in this study (120 from treatment and 120 from comparison classrooms).
The professional development intervention consists of three components: (1) acquisition of content knowledge through training institutes, (2) ongoing support from a bilingual consultant to help teachers implement new instructional strategies, and (3) opportunities for reflection and shared inquiry with other teachers through regular community of practice meetings. The content for the professional development activities will be based on Learningames (The Abecedarian Curriculum) and LiteracyGames, a special version of Learningames recently updated and expanded to promote language and literacy skills in early childhood classrooms. Both curricula will be adapted for use with Latino children who are English language learners.
A battery of measures will be administered in early fall and late spring to examine changes in teaching practices, the quality of the literacy environment, and children's developmental growth in language proficiency and literacy. Measures of treatment fidelity will assess implementation of the intervention on three levels: training facilitation, consultation content and process, and classroom teaching practices. The study also will identify a variety of child, family, and program factors that may moderate children's language and literacy outcomes. The overall goal is to provide teachers with the knowledge and skills necessary to facilitate early English language learning and literacy skills among Latino preschoolers who are English language learners.
University of Chicago
Principal Investigator: Dr. Anthony Bryk
Efficacy Project: Can Literacy Professional Development be Improved with Web-based Collaborative Learning Tools: A Randomized Field Trial
A large body of research has revealed that teacher training is a critical factor in making a difference in students' literacy learning. A widely used approach to improve teacher quality is professional development delivered through coaching. Districts are also investing heavily in web-based technologies to support teacher learning. Both of these strategies, while promising, lack rigorous evidence about their efficacy.
The goal of the current study is to examine the efficacy of an existent professional development program, the Literacy Collaborative (LC), and whether the efficacy of LC is enhanced through the addition of a web-based collaborative learning environment, the Literacy Coaching toolkit (LCtk). The study examines the efficacy of LC and the supplemental LCtk services on the work of school-based literacy coaches, the professional development of teachers whom these coaches support, and on kindergarten through third grade students' literacy learning. In addition to examining overall efficacy, possible mechanisms that might contribute to variability in LC and LCtk effects among schools and among classrooms within schools, and for different types of students categorized by race/ethnicity and family income, are being explored.
The field trial is being conducted in 18 public elementary schools with significant proportions of African-American, Latino and low-income students. The effects of adding LCtk to the standard LC program is being assessed with a randomized design in which the 18 schools are matched on the basis of selected student and school characteristics, and one of each pair of schools is randomly assigned to receive the standard LC program, while the second school receives the LC program combined with LCtk. Embedded in this study is also a second, quasi-experimental design for assessing the overall effects of LC programs on both changes in teacher practice and student learning. This aspect of the research employs a "value-added" design where the three-year trends in student learning gains and teacher practice in each classroom after the introduction of LC are compared to data from the same classroom during a non-treatment baseline period for this study (year 1)
In summary, the goals of the project are to examine the efficacy of providing trained on-site literacy coaches in improving literacy instruction and student outcomes in kindergarten through third grades. The project will also advance the development of a new set of web-based tools to support teacher learning. If these tools prove successful, this research may affect the nature of the services that commercial firms provide to schools. In addition, through the process of designing these tools, the extant knowledge base about the professional work of coaching and its improvement will be deepened.