Principal Investigator: Sian Beilock
Title: Improving the Assessment Capability of Standardized Tests: How High-Stakes Testing Environments Compromise Performance
Purpose: There is substantial concern over the ability of high-stakes tests to assess competence and predict future academic performance. This research team proposes that some aspects of test-taking environments reduce critical thinking skills available for completing test problems, resulting in less-than-optimal performance. This may be particularly true for members of groups that have traditionally not done well when tested in particular subject areas. For example, when aware of the stereotype that “women are bad at math,” women do not perform as well on math tests as they do when they complete the same problems in a non-test-taking situation. This phenomenon has been called "stereotype threat" by social psychologists.
The purpose of this project is twofold: to examine how stereotyping undermines women’s math performance and to develop and test new assessment tools that are designed to reduce the negative effects of stereotyping. The results of this project should provide new guidance on how standardized assessments can be developed that accurately reflect all students’ abilities and potential.
Setting: The research is being conducted in a medium-sized Midwestern town and a large Midwestern city.
Population: Approximately 900 female undergraduates and women in a residential program for female students with interests in math and science are participating. Because stereotype threat is most likely to occur for those highly invested in performing well, women pursuing math and science careers provide an ideal population to examine in a real-world education assessment setting.
Research Design and Methods: The first component of this work explores how stereotype threat interferes with women’s math performance. Studies 1 and 2 test the idea that stereotype threat induces verbal worries that consume phonological resources needed for solving certain types of math problems. Studies 3 and 4 test the counterintuitive predictions that individuals with greater working memory capacity are more susceptible to stereotype threat. Studies 5 and 6 examine if stereotype threat carries over and impairs subsequent tasks that rely on working memory but are not implicated by stereotype threat.
The second component of this work investigates techniques to reduce the negative impact of high-stakes testing situations in real-world education assessment by modifying the major test used in college admissions (i.e., SAT) based on the above findings and administering it to students in the residential program for women in math and science. Studies 7 and 8 modify the quantitative portion of the SAT using principles (tested above) designed to diminish stereotype threat effects. It is anticipated that these new tests will produce better overall math performance and reveal greater predictive validity for academic achievement (e.g., college math grades) than those currently used. Study 9 examines SAT performance following training designed to inoculate students against the negative consequences of high-stakes situations.
Students are randomly assigned to experimental or control conditions in all experiments.
Control Condition: Students in the control conditions are not provided with framing information that precipitates stereotype threat.
Key Measures: Student performance on math problems is the primary data being collected.
Data Analytic Strategy: Analysis of variance techniques are being used to examine student performance on mathematical problems as a function of participation in the experimental or control condition.