October 25, 2005
Dr. Rick Melmer
Secretary of Education
South Dakota Department of Education
700 Governors Drive
Pierre, South Dakota 57501-2291
Dear Secretary Melmer:
Thank you for submitting South Dakota's assessment materials for review under the standards and assessment requirements of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support your State's efforts to monitor student progress toward challenging standards.
External peer reviewers and U.S. Department of Education (ED) staff evaluated South Dakota's submission and found, based on the evidence received, that it did not meet most of the statutory and regulatory requirements of Section 1111(b)(3) of ESEA. Further, based on consultations with your staff, ED expects that South Dakota can take the necessary steps to come into full compliance this school year. The status of the South Dakota assessment system is Final Review Pending. In this status, a State must clearly articulate to ED how it will meet the remaining requirements and be able to fully implement its standards and assessment system by the end of the 2005-2006 school year.
Summaries of the critical elements that met the NCLB standards and assessment requirements follow. The summaries also provide South Dakota with the critical elements that have been approved and will not need further review as well as those that partially met the requirements.
- 1.0 - Content Standards: South Dakota partially meets this requirement for reading and mathematics; however, as revisions are made to the content standards for these subjects, South Dakota must submit evidence listed in the summary section.
- 3.0 - Full Statewide Assessment System: South Dakota meets this requirement.
- 7.0 - Reports: South Dakota meets this requirement; however, it is recommended that South Dakota explore ways to incorporate parent and non-educator feedback regarding the interpretability of state assessment reports and submit sample reports generated for the STARRS (Statewide Team-led Alternate Assessment and Reporting System) alternate assessment.
Additional evidence is needed to show how South Dakota meets the critical elements identified as partially or not meeting the requirements under the NCLB standards and assessment peer review guidance. More detailed information of the additional evidence that South Dakota will need to be in compliance with NCLB is given beginning on page three.
When the required additional evidence has been submitted, it will be subject to peer and ED staff review. A second peer review will need to be scheduled consistent with the previously announced dates. Enclosed with this letter are detailed comments from the peer review team that evaluated the South Dakota assessment materials. The peer reviewers are experts in the areas of standards and assessments. This panel of experts review and discuss a State's submission of evidence and prepare a consensus report that is documented as the Peer Notes. I hope you will find the reviewers' comments and suggestions helpful.
We look forward to working with South Dakota to support a high-quality assessment system. If you would like to discuss this further, please do not hesitate to call Valeria Ford (202-205-2213) or Meredith Miller (202-401-8368) of my staff.
Henry L. Johnson
cc: Dr. Gary Skoglund
Mrs. Diane Lowery
Summary of Additional Evidence that Must be Submitted to Meet NCLB Requirements for the South Dakota Assessment System
1.0 - CONTENT STANDARDS
- Documentation pertaining to the process used to establish rigor for the content standards, such as data from the comparison to the National Assessment of Educational Progress standards, and
- Board approval of the revised standards for reading and for the alternate assessment as they become available.
2.0 - ACADEMIC ACHIEVEMENT STANDARDS
- Minutes of the January 2003 South Dakota Board of Education (SDBOE) meeting or meeting in which performance descriptors for reading and mathematics were approved,
- State Board Approval for the achievement standards for the Extended Standards approved in September 2004,
- Minutes of the SDBOE agenda for March 2005 for the revised science academic content standards and the performance descriptors for K-12,
- Documentation of Board approval of the revised achievement standards for both Dakota STEP (State Test of Educational Progress),
- Alternate achievement standards for STARRS (Statewide Team-Led Alternate Assessment and Reporting System) when developed,
- Explanation of how the Science Content Standards performance descriptors for English Language Learners, Draft for Comment, January 2005, and other students will be used. It appears that these establish one set of standards for English Language Learners and another for other students,
- Revised cut scores for the reading and the alternate assessments, and
- Resolution of findings from the January 2003 Study which included an observation that, " Problems were also identified in the ability of the test to accurately and reliably make proficiency level classifications for students (#85, p.11)."
4.0 - TECHNICAL QUALITY
- A plan for conducting reliability studies for the scores of subpopulations and studies related to accommodated test situations to ensure that they are instructionally appropriate and that the scores yield valid and reliable results,
- Resolution of concerns from the Technical Advisory Committee (#89) notes regarding the Technical Reports provided by Harcourt for the Fall, 2003 and Spring, 2004 administrations (#120-121),
- Outline of the State's timeline for changing test forms: a plan describing how South Dakota (SD) will establish and maintain validity and reliability of its assessments over time; clarification of whether the State intends to use the same test forms for the Dakota STEP from year to year,
- A plan to improve the low point bi-serial coefficients (below .30), e.g., third and fourth grade tests in reading and mathematics and the low reliability indices (below .90) for the grade level alphas,
- Documentation from the June 3 bias review for reading and how results are used; similar review for mathematics,
- Standard error of measure statistics to examine the reliability of the cut points, by content and grade level assessment, and
- Statistics of the item performance across the various groups of students, e.g., DIF statistics.
5.0 - ALIGNMENT
- Results from comprehensive alignment studies that include indicators of depth of knowledge, higher-order thinking skills, depth of content, comprehensiveness and range, cognitive complexity and level of difficulty, content and process, degree and pattern of emphasis, and range of achievement standards for both Dakota STEP and STAARS. The Buros analyses only address the match between the test questions and the content standards; however, there is not sufficient evidence to substantiate a comprehensive alignment,
- Interpretation and/or use made of results for the Webb Alignment Tool, and
- A plan detailing how weaknesses identified in a series of alignment studies previously conducted between its assessments and content standards will be resolved.
6.0 - INCLUSION
- Explanation for the discrepancies in data presented in Documentation #98 and #99 and the Technical Manual (#121),
- Documentation on participation/exclusion rates, how scores for all students are reported, and how non-participating students are accounted for and reported,
- Disaggregated reports that include participation of students with disabilities in alternate assessment vs. regular; % of students with disabilities assessed against alternate achievement standards and those included in the Dakota STEP, and
- Evidence that all students, including migrant and Hutterite Colony students who comprise 3.5% of the LEP population, are included in the reading and mathematics assessments.