November 21, 2005
November 21, 2005
Dear Chief State School Officers:
Earlier this year in a speech at Mount Vernon, I announced that the Department would take a commonsense approach to implementing the No Child Left Behind Act of 2001 (NCLB). This approach, crafted as an equation called Raising Achievement: A New Path for No Child Left Behind, established that, when States show results and follow the tenets of NCLB, they will be eligible for new tools to help them meet the law's goal of getting every child to grade level proficiency by 2013-14. As part of this approach, and in response to educators across the country, the Department has been exploring how accountability models that measure improvements in student achievement (i.e., "growth models") could be one such tool to help schools meet the requirements of NCLB.
This summer the Department convened groups of various experts, including researchers, stakeholders, and policy officials, to inform this work. We discussed a number of critical issues related to growth models that are worth highlighting here. First, we discussed the importance of accelerated student progressthe reality that not just any level of growth is enough to meet the bright line principles of NCLB. Substantial growth is needed to ensure all students are proficient by 2013-14 and to close achievement gaps. A proposed growth model that only expects "one year of progress for one year of instruction" will not suffice, as it would not be rigorous enough to close the achievement gap as the law requires. Second, we discussed the infrastructure required to implement a growth model. To implement this sophisticated approach, a State needs certain foundational elements in its standards and assessment system. NCLB has played a critical role in helping States build these elements. The annual assessments that form the basis of a growth model must work together as a coherent systemthe achievement standards must be aligned with state content standards and must produce comparable results from grade to grade and year to year. These conversations were productive and have informed how we are moving forward.
These discussions confirmed our belief that growth models may show promise for measuring school accountability, giving schools credit for improvement over time, and measuring individual student progress. We also believe that this is the appropriate time to move forward with a pilot project on this issue, as the assessment and data infrastructure system of many States is more sophisticated now than when NCLB was first implemented. This pilot gives us an opportunity to build on the critical elements of NCLB, State resources, and the current AYP model.
Using this letter as a guide, I am inviting States to propose to the Department, a growth model to be used for NCLB accountability purposes as a part of this new pilot project. States that wish to use growth models as part of their NCLB accountability systems must demonstrate that these models can raise student achievement and enhance school accountability.
Our purpose in inviting these proposals is two-fold. First, the Department wants to evaluate how these growth models might help schools meet the NCLB proficiency goals and close the achievement gap. Second, this pilot project will provide the Department and other States useful information to improve the quality of data, assessment, and accountability systems. Consistent with the particular model a State is proposing, a State proposal must show how it meets a set of core principles, detailed below and in the enclosure, thus ensuring that its growth model aligns with NCLB and that its standards and assessment system contains necessary foundational elements.
- The accountability model must ensure that all students are proficient by 2013-14 and set annual goals to ensure that the achievement gap is closing for all groups of students.
- The accountability model must establish high expectations for low-achieving students, while not setting expectations for annual achievement based upon student demographic characteristics or school characteristics.
- The accountability model must produce separate accountability decisions about student achievement in reading/language arts and in mathematics.
- The accountability model must ensure that all students in the tested grades are included in the assessment and accountability system. Schools and districts must be held accountable for the performance of student subgroups. The accountability model, applied statewide, must include all schools and districts.
- The State's NCLB assessment system, the basis for the accountability model, must include annual assessments in each of grades three through eight and high school in both reading/language arts and mathematics, must have been operational for more than one year, and must receive approval through the NCLB peer review process for the 2005-06 school year. The assessment system must also produce comparable results from grade to grade and year to year.
- The accountability model and related State data system must track student progress.
- The accountability model must include student participation rates in the State's assessment system and student achievement on an additional academic indicator.
Further details on these core principles are included in the enclosure that follows. These details explain what we mean by each principle and also provide direction to States regarding the evidence needed to show how their proposal addresses these issues.
In addition to meeting these seven principles, we are interested in approving accountability models under this pilot project that are reasonable, based on common sense, and understandable to educators and the public. We know that States may meet each of these core principles through various means, given differences in their standards, assessments, data systems, and other foundational elements. We welcome that creativity and thoughtfulness, as no two models will likely be alike.
It is important to emphasize that the Department will only approve a proposed growth model that can meet each of the core principles. I understand that this is a high bar and that not every State's data or assessment infrastructure will currently meet these requirements. It is, however, an attainable bar, as we have seen examples of State systems that, for instance, measure individual student progress or have multiple years of annual assessment data. This task will immediately be easier for those States who have developed coherent data and annual assessment systems over the years. These pieces, when put together and combined with the goal of universal proficiency by 2014, can form a growth model that meets the core principles and achieves the requirements of NCLB. I also know that while the State growth models in use today have utility and value, to the best of our knowledge, none currently meets all of these core principles.
The attachment includes four parts. Part I describes the process by which States may submit growth model proposals to the Department. The deadline for submitting such proposals is February 17, 2006. The Department will review submitted proposals, a process that will include the use of outside peer reviewers, and may attach reasonable conditions of approval. In addition, we will consider how States are meeting the Raising Achievement principles (such as narrowing the achievement gap, and creating easily accessible and understandable school and district report cards) that I outlined at Mount Vernon. For those States that propose a model meeting the elements outlined in this letter, we are willing to enter into a flexibility agreement to permit them to develop and use a growth model to hold schools and districts accountable under NCLB.
Please note that States must receive approval from the Department, and make any required amendments to their accountability plans, before implementing their proposed growth models. States that receive approval from the Department to be a part of this pilot project would be able to apply the growth model to their accountability decisions for the 2005-06 school year.
Part II of the attachment covers the core principles of NCLB that must be preserved under any growth model proposal. Part III presents a set of key questions that States must answer to provide evidence that their growth models meet NCLB requirements and help the Department evaluate their proposals. The answers will be considered as a whole by the Department in reviewing each State's proposal.
In addition, we know that many States do not have the assessment systems or data capabilities to be able to meet the core principles outlined above and in the enclosure. Currently, nine states are using an index to calculate AYP. In those instances where implementing a growth model is not possible or desirable, a State may want to consider amending its accountability plan to incorporate an index, especially those States who may not have all the necessary elements at this time to implement a growth model. Part IV of the attachment outlines how States may use an index to calculate AYP.
The Department will rigorously evaluate this pilot, review information on how the pilot project is working, and share results with other States, policymakers, and the public. To that end, States receiving approval to implement a growth model will be expected to provide data showing how the model works compared to the current AYP model, and to analyze the effects of a growth-based accountability model on student achievement. The Department is interested in using the data generated by approved States to conduct an evaluation so that we can share information on the results of the pilot with Congress, State officials, educators, and the public. Outside of accountability plan amendments to adopt an index as a way of measuring AYP, I plan on approving no more than ten high-quality State proposals within the context of this pilot this year. The approval of a growth model applies to the accountability system during the 2005-06 school year and is subject to a review at the end of the school year.
Please do not hesitate to contact the Department with any additional questions or concerns. We look forward to working with interested States on these issues.
Growth Model Pilot Proposals
What should the State include in its proposal? States should submit applications seeking approval for the use of growth models as part of their No Child Left Behind (NCLB) accountability systems to Dr. Henry Johnson, Assistant Secretary of Elementary and Secondary Education. Applicants should clearly and concisely describe their proposals, with particular attention to how these proposals meet each of the requirements outlined below in sections II and III. A State should also include any relevant evidence. States submitting a proposal are encouraged to send it electronically to firstname.lastname@example.org.
When should a State submit its proposal? A State that wishes to incorporate a growth model in calculating adequate yearly progress (AYP) for the 2005-06 school year should submit its proposal to the Department by February 17, 2006. This timeline is designed to provide sufficient time for the Department to review the proposal, and notify the State of its decision so that the State can apply it without delaying AYP determinations and school improvement identifications. A State that wants to submit other amendments to its accountability plan should do so by April 1, 2006, and follow the guidance released previously by the Department: http://www.ed.gov/admins/lead/account/amproc.doc
What is the approval process? Upon receipt of a growth model proposal, the Department will review it and initiate discussions with the State. The Department will use a peer-review process to help evaluate the proposal. Similar to State accountability plan amendment requests, the Department will ask a State whose growth model is approved to submit electronically the final proposal as part of its amended accountability workbook with the date of submission and amended elements noted on the cover page.
Additionally, depending on the particular model a State is proposing, the Department may work with the State to enter into a flexibility agreement. In the flexibility agreement, the Department may require reasonable conditions on the State in exchange for the additional flexibility granted to permit the use of a growth model.
After the flexibility agreement has been finalized and the amended accountability workbook is received, the Department will notify the State that its proposal has been approved and will post this information on the Department's website so that it is readily accessible to the general public. Please note that a State should not implement its proposed growth model until these steps are completed.
Each State's proposal must include information and evidence to show how it meets the core principles outlined below.
The accountability model must ensure that all students are proficient by 2013-14 and set annual goals to ensure that the achievement gap is closing for all groups of students.
How does the model ensure that schools and districts are held accountable for getting all students to proficiency by 2014? A proposed growth model that only expects "one year of progress for one year of instruction" will not suffice, as it would not be rigorous enough to close the achievement gap by 2014.
How does the model set reasonable, challenging, and continuously improving annual expectations for growth? Growth models that rely on substantial increases in the growth rates of students or schools in the last few years are not acceptable, but the Department is open to models that set a point in time as the goal (e.g., end of grade in a particular school; within four years). In setting these standards, the State should demonstrate how accountability is distributed among all grades and not postponed to this point in time. The Department is concerned that if the State's growth model allows attainment of the proficiency standard by individual students to be delayed or is tied to standards that are not considerably more rigorous with each consecutive grade, then it becomes too easy to minimize or delay the importance of accelerated growth.
How does the proposal incorporate the growth model into measures of status and safe harbor? States receiving approval to implement a growth model will be expected to provide data showing how the model works compared to the current AYP model. As a possible condition for implementing a growth model, the State may need to explain how it plans to focus its school intervention efforts incorporating the results from a growth model. For instance, a State should be prepared to explain how a school that does not meet either traditional AYP goals or growth-based accountability goals might be subject to more rigorous intervention efforts than schools not making AYP on only one accountability measure.
The accountability model must establish high expectations for low-achieving students, while not setting expectations for annual achievement based upon student demographic characteristics or school characteristics.
How does the model ensure that student growth expectations are not set or moderated based on student demographics or school characteristics? The model must have the same proficiency expectations for all students while setting individual growth expectations for students to enable them to meet that grade level standard.
How does the model ensure that growth is measured relative to achievement standards and not "typical" growth patterns or previous improvement?
The accountability model must produce separate accountability decisions about student achievement in reading/language arts and in mathematics.
How does the model ensure that students are improving in their achievement in reading/language arts and mathematics? How does the model hold schools accountable for such achievement? The growth model proposal should include separate decisions for reading/language arts and mathematics, although we recognize that it is also important to design accountability systems that maintain validity and reliability, minimize measurement error, and support empirical integrity in the accountability system. How does the model achieve these specifications, especially in small schools or schools with high mobility?
Does the model include assessments for other content areas? If so, the state should demonstrate that achievement on those other assessments does not compensate for low achievement in reading/language arts and mathematics tests.
The accountability model must ensure that all students in the tested grades are included in the assessment and accountability system. Schools and districts must be held accountable for the performance of student subgroups. The accountability model, applied statewide, must include all schools and districts.
How does the model maximize inclusion of students, particularly in light of missing data problems or student mobility? Because of these concerns, and depending upon the proposed State model, the Department may expect States to continue calculating and using the "status" approach to AYP.
If appropriate, how does the State proposal address the needs of students displaced by Hurricanes Katrina and Rita? In particular, the Department will need details on how this proposal interacts with any State plans to develop a separate subgroup of displaced students, consistent with the Secretary's guidance of September 29, 2005. http://www.ed.gov/policy/elsec/ guid/secletter/050929.html
How does the State measure growth of students who are in the first tested grade? How do these issues interact with the State's "full academic year" definition? How do the Department's policies about assessing limited English proficient students and students with disabilities affect missing data policies?
How does the model hold schools accountable for the achievement of all student groups?
How does the model ensure that all schools receive an AYP determination each year? The Department expects that States will implement the accountability model in a consistent fashion across the State. Subject to that requirement, the Department is open to States using one model for schools with consecutively tested grades (i.e., elementary/middle) and a different AYP model for schools without consecutively tested grades (i.e., high schools).
How does the model ensure that all schools and districts are accountable for student achievement, even when the number of tested students in the school or district is small or constantly changing?
The State's NCLB assessment system, the basis for the accountability model, must include annual assessments in each of grades three through eight and high school in both reading/language arts and mathematics, must have been operational for more than one year, and must receive approval through the NCLB peer-review process for the 2005-06 school year. The assessment system must also produce comparable results from grade to grade and year to year.
How does the State's assessment system support the use of a growth-based accountability model? Please provide evidence explaining how students will be assessed with tests that produce comparable results from grade to grade and year to year, such as through vertical scales or vertically moderated achievement levels.
How does the State demonstrate that its assessment system is stable and produces reliable and valid data?
The accountability model and related State data system must track student progress.
How does the State demonstrate that its data system can track student growth over time, across schools, and across districts? The State should provide evidence on the capacity of its data system to implement a growth model.
How does the model use individual student growth (such as through an individual student identifier)?
The accountability model must include student participation rates in the State's assessment system and student achievement on an additional academic indicator.
How does the model ensure that students participate in the assessment system and that schools are held accountable for such participation? School and district accountability must continue to take participation into consideration. Calculation of participation rates must be based on all students enrolled in the grades tested during the testing window.
How does the model ensure that schools are accountable for performance in one other academic indicator besides reading/language arts and math? School and district accountability must take another academic indicator into consideration as is the case with current systems.
Questions to be answered by the State:
A State's proposal must also address the issues below. The Department will evaluate how these elements work together to create a coherent accountability system during the review process.
- Uniform averaging. What role, if any, will the uniform averaging procedures have in a model that incorporates "growth" in the State's AYP definition?
- AYP formula issues. What is the State's minimum group size? Does the State use confidence intervals? How will each of these continue in a State AYP definition that uses growth?
- Assessments. What happens to the model when States use various assessments to assess the same content (e.g., a regular assessment, a native language assessment, or an alternate assessment aligned with grade-level achievement standards)? What happens under the model when, for instance, a student takes a reading assessment based on regular achievement standards in year one and a reading assessment based on alternate achievement standards in year two?
- Higher-achieving students. Does the model assign schools any credit for higher achieving students of two types: 1) students who were previously low-achieving and are now above proficient, and 2) students who have always been high achieving.
- Reporting. How will the State report data in a transparent, understandable manner to districts, schools, the community, and the Department?
Under NCLB, States currently have the flexibility to take into account the percentage of students who improve their achievement levels through the development of what is commonly referred to as an "index model."
An index model provides a way for States to recognize schools that improve achievement among students at the below proficient level(s), thus providing an alternative to the "status" model of AYP decision-making that emphasizes the percentage of students at or above the proficient level. An index model does this by assigning a value (determined by the State) to various levels of achievement, with the highest value assigned to students at the proficient/advanced level and progressively lower values for each level below that.
Although an index does not measure individual student growth, it gives schools and districts "credit" for improving the achievement of students who are "below proficient." For example, a school will receive some credit if it moves students from "below basic" to "basic" even if they are not yet proficient. The end target of having 100 percent of students proficient in reading and math by 2014 still remains, however.
The Department has allowed States to base AYP on an index score when the following conditions are met:
The index does not give extra weight to students scoring above proficiency so as to mask performance in the lower achievement levels;
The index can be calculated separately for reading and mathematics and for each relevant student subgroup;
The index does not allow schools to make AYP without also increasing the percentage of students who are proficient; and
The index must be consistent with NCLB and the regulations promulgated under it, including provisions on annual measurable objectives and intermediate goals.
Currently, nine States are using such index models in their AYP determinations under their approved accountability plans: Massachusetts, Minnesota, Mississippi, New Mexico, New York, Oklahoma, Pennsylvania, South Carolina, and Vermont.
For examples, more information about the Minnesota and Massachusetts index models can be found online at: