ED.gov

ATTACHMENT A
SPECIAL EDUCATION ELEMENTARY LONGITUDINAL STUDY
PERFORMANCE WORK STATEMENT

PURPOSE

The U.S. Department of Education (ED), Office of Special Education Programs (OSEP) requires a Contractor for forty-eight (48) months with a twenty-four (24) month option period to conduct a longitudinal study of elementary school students with disabilities. This study will collect data from students, parents, teachers, and school principals to form a comprehensive picture of the experiences of elementary school students with disabilities. It will also provide data for some of OSEP’s Part B GPRA indicators.

BACKGROUND

Section 674a of Part B of the Individuals with Disabilities Education Act (IDEA) authorizes the collection of data and the conducting of studies to measure and evaluate the impact of IDEA and the effectiveness of State efforts to provide a free, appropriate public education to all children with disabilities. In 1983, a special National Longitudinal Transition Study for students with disabilities was mandated by Congress under Section 8 of P.L. 98-199. That study followed 8,000 students, aged 13-21 in the 1985-86 school year, for a five (5)-year period from the 1985-86 school year through the 1989-90 school year. The focus of that study was on the secondary school performance and transition from secondary school to post-secondary education and/or employment. However, that study did not examine the elementary school progress of those students leading up to their performance in secondary school.

Until now, there has been no national longitudinal study of students with disabilities at the elementary school level focusing on school experiences during the elementary school years and on preparation for transition from elementary to middle and secondary school. The elementary school years constitute an important period in the lives of children with disabilities. It is in this time period that many of these children are first referred for special education, assessed, identified, and provided with services. Although federally mandated data collection activities provide some information on the age, disability category, and placement of these children, little is known on a national basis about the experiences of elementary school children as they receive services provided to them under IDEA.

The Special Education Elementary Longitudinal Study (SEELS) is part of a comprehensive program of longitudinal research related to the 1997 Amendments to the Individuals with Disabilities Education Act (IDEA). SEELS will involve a large, nationally representative sample of students in special education who are ages 8 through 12 in the first year of the study. Students will be selected randomly from rosters of students in special education. Statistical summaries generated from SEELS will generalize to special education students nationally as a group, to individual special education disability categories, and to each single-year age cohort. Information about students will be collected repeatedly as they transition from elementary to middle school and from middle to high school. Thus, SEELS will provide the first national picture of the experiences and outcomes of students in special education as students move through the formative years of their education.

Sample

Prior to the award of this contract, a nationwide sample of Local Education Agencies (LEAs) has been selected and participation agreements are being obtained. To meet its purposes, the SEELS sample includes only LEAs that have teachers, students, administrators, and operating schools (i.e., "operating" LEAs).

To create the sampling frame or master list of LEAs, two lists were considered: the public school universe maintained by Quality Education Data (QED, 1998) and the School District Name and Address File maintained by the National Center for Education Statistics in the U.S. Department of Education (1997). The two lists were compared on variables indicating LEA name and location. The list with the most current master list of 13,346 LEAs, QED, was used to construct the sampling frame.

The SEELS LEA sample is stratified for four principal reasons: (1) to increase the precision of estimates by eliminating between-strata variance, (2) to ensure that low-frequency types of LEAs (e.g., large urban districts) are adequately represented in the sample, (3) to improve comparisons with the findings of other research, and (4) to make SEELS responsive to concerns voiced in policy debate.

The universe of eligible LEAs was stratified by key factors to enhance representativeness; these factors are geographic region, district enrollment, and district/community wealth. These stratifying variables were selected on the basis of conceptual soundness and the likelihood of providing a gain in precision over simple random sampling.

Stratification by geographic region captures essential political differences, as well as subtle differences in the organization of schools, the economic conditions under which they operate, and the character of public concerns. For SEELS, the regional classification variable selected is used by the Department of Commerce, the Bureau of Economic Analysis, and the National Assessment of Educational Progress.

LEAs vary considerably by size (i.e., student enrollment), and a host of organizational and contextual variables are associated with size. These variables exert considerable potential influence over the operations and effects of special education and related programs. They include the extent of district administrative/supportive capacity, the degree of specialization in administrative structure, the nature of citizen and interest group activity in education, and the characteristics of relationships with State and Federal governance systems. The QED data base provides enrollment data from which LEAs were sorted into four categories serving approximately equal numbers of estimated enrollment in grades 2 through 7: very large (> 17,411), large (4,707 - 17,411), medium (1,548 - 4,706), and small (10 - 1,547).

LEAs differ greatly in the resources they have available and in the demands placed on those resources by low-income students whose needs put them at risk for a variety of problems, including school failure. Policies and programs may differ in LEAs that face these differential demands of disadvantaged students. A well-accepted measure of district wealth, the Orshansky index, was used as the basis for stratification in the SEELS. This index provides the proportion of the student population living below the Federal definition of Poverty. The distribution of the Orshansky index was organized into four categories of district/community wealth, each containing approximately 25% of the student population in grades 2 through 7: high (0% - 12%), medium (13% - 34%), low (35% - 45%), and very low (> 45%).

Progress on the SEELS LEA sampling will be documented on the World Wide Web. The address for the link to this study is:

http://www.ed.gov/offices/OSERS/OSEP

Data Collection Instruments

Given the broad range of topics encompassed in the SEELS conceptual framework will, information for the study will be collected from a variety of sources. These include parents/guardians, teachers, principals, school records, and direct assessment of the students themselves. Data collection instruments will be constructed and cleared prior to their implementation in this project.

Parent Telephone Interview. A child’s nonschool experiences, such as extracurricular activities and friendships; historical information, such as age when disability was first identified; household characteristics, such as socioeconomic status; and a family’s level and type of involvement in school-related areas are crucial to student outcomes. These aspects of students’ lives are the basis for the parent interview. The instrument will be in the form of computer-assisted telephone interviewing (CATI). This instrument will contain skip patterns to eliminate the need for respondents to answer items that are inappropriate to the age, disability, or other circumstance of the household or student. Most of the questions from the wave 1 instrument will be repeated in the instruments for waves 2 and 3. However, items that do not change with time, such as when the child was first diagnosed with a disability, will not be repeated. Items that are appropriate for older students, such as questions about employment, pregnancy/parenting, arrest history, and having a driver’s license, will be included only in the wave 3 instrument. The average length of time for the telephone interview is expected to be 40 minutes.

Direct Assessment. The academic performance of students in their elementary and middle school years is central to their postsecondary outcomes. Academic performance will be assessed in a set of standardized testing instruments. This assessment is expected to take thirty (30) to forty-five (45) minutes, on average. The assessment instruments are scheduled for field testing in the fall of 1999.

Brief student interview instruments shall be constructed for use in conjunction with the academic assessment. These will contain questions regarding feelings about the student’s social, school, and other experience. The wave 3 instrument shall also contain questions about adolescent risk behaviors (e.g., smoking, drinking, drug use, and gang involvement). Student interviews are expected to take ten (10) to fifteen (15) minutes.

School Data Collection. The school data collection is composed of three instruments: a language arts teacher survey, a school program survey, and a school background survey.

The language arts teacher survey instrument shall be designed as a mailed, self-administered questionnaire. It will contain questions regarding classroom practices and student behaviors in the context of reading and language instruction. This instrument is expected to help explain variations in assessments of academic performance.

The school program survey instrument shall be designed for completion by the special education teacher or the person at the school best able to answer questions about the student’s overall school program. The purpose of this survey is to identify the student’s accommodations, supports, and related services; content of IEP goals; and overall school program and performance (e.g., achievement test scores, days absent from school). The survey will also include questions about the special education teacher’s experience and background. For students in middle and high school, respondents will also be asked to include a copy of the student’s transcript.

The school background survey instrument will be designed as a mail questionnaire for school principals. It will focus on school-wide characteristics and policies and will ask the principal to describe the general characteristics of the school and local school district. For example, items will include the demographics of the school, student enrollment, and grade levels served, as well as policies regarding special education services. This information is expected to help facilitate understanding of the organizational structure, placement, provision of services, and outcomes for students in special education.

An overview of the SEELS data collection plan is presented in Table 1. The data collection for SEELS will be in three (3) waves (i.e., data will be collected three (3) times from each source). Wave 1 parent data will be collected in the first year of the study. Wave 1 direct assessment and school data will be collected in the second year. All wave 2 data will be collected in the third year of the study, and all wave 3 data will be collected in the fifth year.

The content of the data collection instruments is presented in Table 2. Progress on the development of instruments will be documented on the World Wide Web (address above).

Table 1

OVERVIEW OF SEELS DATA COLLECTION

	Year 1 (Wave 1) 1999-2000	Year 2 (Wave 1) 2000-2001	Year 3 (Wave 2) 2001-2002	Year 4 2002-2003	Year 5 (Wave 3) 2003-2004	Year 6 2004-2005
Parent Interviews	P		P		P
Direct Assessment/ Student Interviews		P	P		P
Language Arts Teacher Survey		P	P		P
School Program Survey/ Record Review		P	P		P
School background (principal) survey		P	P		P

Table 2

CONTENT OF DATA COLLECTION INSTRUMENTS

	*PI	DA	SI	LATS	SPS	SBS
Student’s demographic characteristics	P
Student’s identified disabilities	P				P
Student’s physical functioning level	P
*Student’s strengths (e.g., talents)*	P			P	P
Student’s education/treatment history	P
Pattern of disability classification					P
Health insurance experiences	P
History of school transfers	P
Composition of student’s household	P
Socioeconomic status of household	P
Community characteristics (derived from household address)
Participation in group activities	P
After-school care needs/experiences	P
Parent/guardian participation in support activities at home	P
Expectations for student’s future	P		P
Services/supports outside of school provided by family	P
Student’s current grade level	P				P
Proportion of time spent in each classroom setting					P
Classroom characteristics				P
Exposure to regular education curriculum				P
TABLE 2 (continued)	PI	DA	SI	LATS	SPS	SBS
Instructional practices				P
Vocational experiences included in curriculum					P
Community-based experiences included in curriculum					P
Methods of progress evaluation				P	P	P
Educational assistive devices				P	P
Accommodations/modifications in instruction and assessment				P	P
Medical/disability-related assistive devices	P			P	P
Related special education services					P
IEP goal domains					P
Preparation/support for transition	P				P
Postsecondary transition planning					P
Contact between school and family	P			P	P
Family participation in IEP meetings	P				P
Characteristics of special education teachers				P	P
Preparation of special education teachers				P	P
Professional development of special education teachers				P	P
Use of paraprofessionals				P	P
School characteristics						P
School safety	P		P			P
School’s emphasis on academic success				P		P
School’s special education policies/practices						P
Options for placement within school						P
Options for types of schools available in district						P
Support programs operated at school						P
Personnel resources						P
Special education teacher’s caseload					P
Actions taken by school to facilitate family involvement						P
Implementation of school-wide reforms						P
Student’s academic performance	P	P		P	P
Student’s engagement in school	P		P	P	P
Student’s communication level	P	P	P	P	P
Student’s academic history	P				P
Student’s social engagement	P		P	P	P
Student’s peer relations	P			P	P
Student’s personal adjustment			P
Student’s adherence to rules	P			P	P
Student’s involvement in community service	P
Student’s level of self-sufficiency	P
Student’s use of leisure time	P		P
Older student’s acceptance of adult roles	P
Student’s health	P
Older student’s risk behaviors			P
Parent satisfaction with special education	P
Student satisfaction with school			P
Student satisfaction with personal/social life			P

*PI = Parent Interview LATS = Language Arts Teacher Survey

DA = Direct Assessment SPS = School Program Survey

SI = Student Interview SBS = School Background Survey

SCOPE OF WORK

The term of the contract shall be forty-eight (48) months with one (1) twenty-four (24) month option, which may be extended to the Contractor at the discretion of OSEP. The Contractor shall perform the tasks as specified. Where unspecified, allow a minimum of ten (10) working days for the review and approval of all contractual deliverables. Unless otherwise specified, three (3) paper copies of all reports are to be submitted. Where reports on disk are specified, three (3) copies are to be submitted.

Task 1: Planning Meeting

The Contractor shall set up a meeting with the Contracting Officer’s Technical Representative (COTR) and other relevant U.S. Department of Education (ED) officials. The purpose of the meeting is to discuss the details of the tasks to be performed. The Contractor shall submit a summary of the meeting.

Deliverables: 1.1: The Contractor shall submit a meeting summary within one (1) week after the meeting.

Task 2: Communication with COTR

The Contractor shall maintain ongoing communication with the COTR by telephone, email, and in the form of a brief quarterly progress report. This report provides a concise narrative of progress on each of the tasks worked on during the quarter.

Deliverables: 2.1: The Contractor shall submit brief quarterly progress reports.

Task 3: Student Sample

The Contractor shall contact sample LEAs to request rosters of students in special education between the ages of eight (8) and twelve (12). The Contractor shall then follow up with nonresponding LEAs to obtain the rosters. Upon receipt of the rosters, the Contractor shall select students using sampling fractions supplied by ED for age and disability category. The Contractor shall contact parents of sampled students to inform them of study participation and request consent for direct assessment and access to school records. Letters of Assurance for the parents will be supplied by ED. The Contractor shall implement tracking procedures to minimize sample attrition with a goal of less than or equal to eight (8) percent per year. Students’ attendance shall be verified at each wave of data collection. The Contractor shall submit a final sample report, indicating the number of students sampled in each cell of the stratified design and the number of students sampled in each LEA.

Deliverables: 3.1: The Contractor shall submit a final sample report by 3/31/00.

Task 4: Web Site

The Contractor shall establish a web site in preparation for posting documents, as specified in this Performance Work Statement. The Contractor shall submit the URL. All eventual postings on the site will be approved by ED and are subject to the relevant guidelines in U.S. Department of Education World Wide Web Policy & Procedures Technical Standards Document Submission Guide (accompanying this Statement of Work).

Deliverables: 4.1: The Contractor shall establish a SEELS web site and submit the URL by 6/30/01.

Task 5: Parent Interviews, Wave 1

The Contractor shall conduct the wave 1 parent telephone interviews. The Contractor shall analyze the parent interview data. The Contractor shall prepare a draft report on the analysis results. The Contractor shall make revisions based on the feedback from ED and prepare a final report. After approval by ED, the Contractor shall post the final report on the SEELS web site.

Deliverables: 5.1: The Contractor shall submit a draft report on the wave 1 parent interview data by 5/31/01. ED will provide feedback within one (1) month. 5.2: The Contractor shall submit a final report on the wave 1 parent interview data, revised according to ED feedback, on paper and on computer disk by 7/31/01. 5.3: The Contractor shall post the final report on the SEELS web site by 8/31/01.

Task 6: Direct Assessment, Wave 1

The Contractor shall prepare a draft of training materials for on-site professionals who will do direct assessment. The Contractor shall revise the training materials based on feedback from ED. The Contractor shall contact the LEAs to identify the person to do direct assessment at each site. The Contractor shall contract with the individuals to conduct direct assessments at each site, then distribute training materials. The Contractor shall provide technical support by telephone and email, as needed, during the wave 1 direct assessment. The Contractor shall implement follow-up procedures for data not submitted with a goal of a seventy-five (75) percent response rate. The Contractor shall compute score distributions for the direct assessment and survey question frequencies for the student interview data, then prepare a report.

Deliverables: 6.1: The Contractor shall submit draft direct assessment training materials by 9/29/00. Ed will provide feedback within one (1) month. 6.2: The Contractor shall submit final direct assessment training materials, revised according to ED feedback, by 11/30/00. 6.3: The Contractor shall submit a report of wave 1 score distributions from the direct assessment and item frequencies from the student interview by 10/31/01.

Task 7: School Data Collection, Wave 1

The Contractor shall mail the wave 1 school background survey to the principals for completion, the teacher survey to the principals for distribution to the language arts teachers, and the school program survey to the principals for distribution to the appropriate school staff who can best respond for the individual students. The Contractor shall follow up all data not received with a goal of a seventy-five (75) percent response rate. The Contractor shall compute frequencies and relevant cross tabulations for the wave 1 school data, then prepare a report.

Deliverables: 7.1: The Contractor shall submit a report of wave 1 school data frequencies and cross tabulations by 10/31/01.

Task 8: Analysis and Reporting, Wave 1

The Contractor shall prepare an outline for analyses combining all wave 1 data components. The outline shall include three (3) recommendations for a report on a special topic. The Contractor shall perform analyses of wave 1 data based on feedback from ED. The Contractor shall prepare a draft report of the data analyses. The report shall be in two (2) volumes: one (1) overview report and one (1) special topic report. The Contractor shall make revisions based on the feedback from ED and prepare the final report. After approval by ED, the Contractor shall post the final report on the SEELS web site.

Deliverables: 8.1: The Contractor shall submit a data analysis outline by 3/31/02. ED will provide feedback within one (1) month. 8.2: The Contractor shall submit two (2) draft reports on the wave 1 data by 5/31/02. Ed will provide feedback within one (1) month. 8.3: The Contractor shall submit two (2) final reports of the wave 1 data, revised according to ED feedback, on paper and on computer disk by 7/31/02. 8.4: The Contractor shall post the final reports on the SEELS web site by 8/31/02.

Task 9: Parent Interviews, Wave 2

The Contractor shall conduct the wave 2 parent telephone interviews. The Contractor shall compute frequencies and relevant cross tabulations for the wave 2 parent interview data, then prepare a report.

Deliverables: 9.1: The Contractor shall submit a report of wave 2 parent interview data frequencies and cross tabulations by 10/31/02.

Task 10: Direct Assessment, Wave 2

The Contractor shall contract with professionals to conduct the wave 2 direct assessment. For individuals who are not the same as those conducting the wave 1 direct assessment, the Contractor shall distribute training materials. The Contractor shall provide technical support by telephone and email, as needed, during the wave 2 direct assessment. The Contractor shall implement follow-up procedures for data not submitted with a goal of a seventy-five (75) percent response rate. The Contractor shall compute score distributions for the direct assessment and survey question frequencies for the student interview data, then prepare a report.

Deliverables: 10.1: The Contractor shall submit a report of wave 2 score distributions from the direct assessment and item frequencies from the student interview by 10/31/02.

Task 11: School Data Collection, Wave 2

The Contractor shall mail the wave 2 school background survey to the principals for completion, the teacher survey to the principals for distribution to the language arts teachers, and the school program survey to the principals for distribution to the appropriate school staff who can best respond for the individual students. The Contractor shall follow up all data not received with a goal of a seventy-five (75) percent response rate. The Contractor shall compute frequencies and relevant cross tabulations for the wave 2 school data, then prepare a report.

Deliverables: 11.1: The Contractor shall submit a report of wave 2 school data frequencies and cross tabulations by 10/31/02.

Task 12: Renew OMB Clearance for Instruments

The Contractor shall make adjustments to the data collection instruments, if necessary, as specified by ED. The Contractor shall prepare the necessary OMB clearance forms and justifications for renewal of the clearance for an additional three (3) years. The Contractor shall provide consultation and support during the forms clearance process. The Contractor shall submit written responses to questions raised regarding the instruments and shall submit revised instruments as needed.

Deliverables: 12.1: The Contractor shall make adjustments to the data collection instruments, if necessary, and prepare an OMB clearance package by 8/31/02. 12.2: The Contractor shall submit written responses to questions raised during the forms clearance process for the period 9/1/02 through 12/31/02. 12.3: The Contractor shall submit revised instruments as needed during the forms clearance process for the period 9/1/02 through 12/31/02.

Task 13: Analysis and Reporting, Wave 2

The Contractor shall prepare an outline for analyses combining all wave 1 and wave 2 longitudinal data components. The outline shall include four (4) recommendations for reports on special topics. The Contractor shall perform analyses of wave 1 and wave 2 longitudinal data, based on feedback from ED. The Contractor shall prepare a draft report of the data analyses. The report shall be in three (3) volumes: one (1) overview report and two (2) special topic reports. The Contractor shall make revisions based on the feedback from ED and prepare a final report. After approval by ED, the Contractor shall post the final report on the SEELS web site.

Deliverables: 13.1: The Contractor shall submit a data analysis outline by 3/31/03. ED will provide feedback within one (1) month. 13.2: The Contractor shall submit three (3) draft reports on the wave 1 and wave 2 longitudinal data by 5/31/03. Ed will provide feedback within one (1) month. 13.3: The Contractor shall submit three (3) final reports of the wave 1 and wave 2 longitudinal data, revised according to ED feedback, on paper and on disk by 7/31/03. 13.4: The Contractor shall post the final reports on the SEELS web site by 8/31/03.

Task 14: Parent Interviews, Wave 3

The Contractor shall conduct the wave 3 parent telephone interviews. The Contractor shall compute frequencies and relevant cross tabulations for the wave 3 parent interview data, then prepare a report.

Deliverables: 14.1: The Contractor shall submit a report of wave 3 parent interview data frequencies and cross tabulations by 8/31/04.

Task 15: Direct Assessment, Wave 3

The Contractor shall contract with professionals to conduct the wave 3 direct assessment. For individuals who are not the same as those conducting the wave 1 or wave 2 direct assessment, the Contractor shall distribute training materials. The Contractor shall provide technical support by telephone and email, as needed, during the wave 3 direct assessment. The Contractor shall implement follow-up procedures for data not submitted with a goal of a seventy-five (75) percent response rate. The Contractor shall compute score distributions for the direct assessment and survey question frequencies for the student interview data, then prepare a report.

Deliverables: 15.1: The Contractor shall submit a report of wave 2 score distributions from the direct assessment and item frequencies from the student interview by 8/31/04.

Task 16: School Data Collection, Wave 3

The Contractor shall mail the wave 3 school background survey to the principals for completion, the teacher survey to the principals for distribution to the language arts teachers, and the school program survey to the principals for distribution to the appropriate school staff who can best respond for the individual students. The Contractor shall follow up all data not received with a goal of a seventy-five (75) percent response rate. The Contractor shall compute frequencies and relevant cross tabulations for the wave 3 school data, then prepare a report.

Deliverables: 16.1: The Contractor shall submit a report of wave 3 school data frequencies and cross tabulations by 8/31/04.

Task 17: Analysis and Reporting, Wave 3

The Contractor shall prepare an outline for analyses combining all wave 1, wave 2, and wave 3 longitudinal data components. The outline shall include four (4) recommendations for reports on special topics. The Contractor shall perform analyses of wave 1, wave 2, and wave 3 longitudinal data, based on feedback from ED. The Contractor shall prepare a draft report of the data analyses. The report shall be in five volumes: one (1) overview report, one (1) technical report, and three (3) special topic reports. The Contractor shall make revisions based on the feedback from ED and prepare a final report. After approval by ED, the Contractor shall post the final report on the SEELS web site.

Deliverables: 17.1: The Contractor shall submit a data analysis outline by 3/31/05. ED will provide feedback within one (1) month. 17.2: The Contractor shall submit four (4) draft reports on the wave 1, wave 2, and wave 3 longitudinal data by 5/31/05. Ed will provide feedback within one (1) month. 17.3: The Contractor shall submit four (4) final reports of the wave 1, wave 2, and wave 3 longitudinal data, revised according to ED feedback, on paper and on disk by 7/31/05. 17.4: The Contractor shall post the final reports on the SEELS web site by 8/31/05.

Task 18: Electronic Data Base

The Contractor shall produce the SEELS database on CD-ROM. The Contractor shall prepare paper documentation of the data on CD-ROM.

Deliverables: 18.1: The Contractor shall submit a CD-ROM containing the SEELS data with documentation by 9/30/05.

Task 19: Dissemination

The Contractor shall present the results of this study at two (2) conferences. The conference shall be appropriate for the dissemination of special education data.

Deliverables: 19.1: The Contractor shall submit two (2) papers to be presented at conferences with evidence that they have been submitted for acceptance at two (2) conferences by 9/30/05.

SCHEDULE OF DELIVERABLES

#	Deliverable	Due
1.1	Planning meeting report (1 week after meeting)	*
2.1	Quarterly progress reports	Ongoing
3.1	Final student sample report	3/31/00
4.1	SEELS web site	6/30/01
5.1	Draft report on wave 1 parent interview	5/31/01
5.2	Final report on wave 1 parent interview	7/31/01
5.3	Final report on wave 1 parent interview on web	8/31/01
6.1	Draft direct assessment training materials	9/29/00
6.2	Final direct assessment training materials	11/30/00
6.3	Data report from wave 1 direct assessment	10/31/01
7.1	Data report from wave 1 school data collection	10/31/01
8.1	Wave 1 data analysis outline	3/31/02
8.2	Draft report on wave 1 data	5/31/02
8.3	Final report on wave 1 data	7/31/02
8.4	Final report on wave 1 data on web	8/31/02
9.1	Data report from wave 2 parent interview	10/31/02
10.1	Data report from wave 2 direct assessment	10/31/02
11.1	Data report from wave 2 school data collection	10/31/02
12.1	OMB package for instruments	8/31/02
12.2	OMB Q&A for instruments	9/1/02- 12/31/02
12.3	Revised instruments	9/1/02- 12/31/02
13.1	Waves 1 and 2 data analysis outline	3/31/03
13.2	Draft report on waves 1 and 2 data	5/31/03
13.3	Final report on waves 1 and 2 data	7/31/03
13.4	Final report on waves 1 and 2 data on web	8/31/03
14.1	Data report from wave 3 parent interview	8/31/04
15.1	Data report from wave 3 direct assessment	8/31/04
16.1	Data report from wave 3 school data collection	8/31/04
17.1	Waves 1, 2, and 3 data analysis outline	3/31/05
17.2	Draft report on waves 1, 2, and 3 data	5/31/05
17.3	Final report on waves 1, 2, and 3 data	7/31/05
17.4	Final report on waves 1, 2, and 3 data on web	8/31/05
18.1	SEELS database on CD-ROM with documentation	9/30/05
19.1	Two (2) papers for presentation at conferences	9/30/05

APPENDIX A: QUALITY ASSURANCE SURVEILLANCE PLAN

Introduction

This Performance-Based Quality Assurance Surveillance Plan (QASP) has been developed pursuant to the requirements of the Performance-Based State of Work in Contract No. _________. This plan sets forth procedures and guidelines that the U.S. Department of Education will use in evaluating the technical performance of the Contractor. In order for the Contractor to be aware of the methods that the Government will employ in evaluating its performance, a copy of this plan will be furnished to the Contractor at the time of the initial meeting in Washington with the Contractor and the COTR. At that time the Contractor may address any concerns in regard to the QASP and how it will be used to assess the Contractor’s performance.

Purpose of the QASP

The QASP is intended to accomplish the following:

a. Define the roles and responsibilities of participating Government officials;

b. Define the key deliverables that will be assessed;

c. Describe the rating elements and standards of performance against which the Contractor’s performance will be assessed for each key deliverable;

d. Describe the process of quality assurance assessment; and

e. Provide copies of the quality assurance monitoring forms that will be used by the Government in documenting and evaluating the Contractor’s performance.

Roles and Responsibilities of Participating Government Officials and Experts

The following Government Officials and/or experts will participate in assessing the quality of the Contractor’s performance. Their roles and responsibilities are described as follows:

1. The Contracting Officer’s Technical Representative (COTR) will be responsible for monitoring, assessing, recording, and reporting on the technical performance of the Contractor on a day-to-day basis. The COTR will have the primary responsibility for completing (Quality Assurance Monitoring Forms" which will be used to document the inspection and evaluation of the Contractor’s work performance. It is extremely important for the COTR to establish and maintain a team oriented line of communication with the Contractor’s Project Manager (PM) and the PM’s office staff due to daily interface necessary in performing monitoring functions. The COTR, the Contracting Officer (CO), and PM must work together as a team to ensure that required work is accomplished in an efficient and proper manner. Meetings should be held on a regular basis in order to resolve serious problems. Less serious problems should be discussed and resolved on an impromptu basis.

2. The Contract Specialist (CS) will have overall responsibility for overseeing the Contractor’s performance. The CS will also be responsible for the day-to-day monitoring of the Contractor’s performance in the areas of contract compliance, contract administration, cost control and property control; reviewing the COTR’s assessment of the Contractor’s performance; and resolving all differences between the COTR’s version and the Contractor’s version. The CS may call upon the expertise of other Government individuals as required. The Contracting Officer (CO) may appoint the CS to attend meetings with the Contractor and to act on her behalf.

3. The Contracting Officer (CO) will have overall responsibility for overseeing the Contractor’s performance. All planned additions and modifications resulting in alterations of the facility’s structure or mechanical services must be approved by the CO prior to commencement. The CO may call upon the expertise of other Government individuals as required. The CO’s procurement authorities include the following:

a. SOLE authority for any decisions which produce an increase or decrease in the scope of the contract;

b. SOLE authority for any actions subject to the "Changes" clause;

c. SOLE authority for any decisions to be rendered under the "Disputes" clause;

d. SOLE authority for negotiation and determination of indirect rates to be applied to the contract;

e. SOLE authority to approve the substitution or replacement of the PM and other key personnel;

f. SOLE authority to approve the Contractor’s invoices for payment, subject to the Limitation of Costs clause and the Limitation of Funds clause;

g. SOLE authority to monitor and enforce Department of Labor promulgated labor requirements;

h. SOLE authority to arrange for and supervise QA activities under this contract;

i. SOLE authority to approve the Contractor’s Quality Control Program;

j. To approve all Contractor purchases of equipment, supplies, and materials exceeding $2,500 are encouraged even though not required by FAR 13.106; and

k. Signatory authority for the issuance of all modifications to the contract.

Key Deliverables to be Assessed

Even though the Government through its COTR will be monitoring the Contractor’s performance on a continuing basis, the volume of tasks performed by the Contractor makes technical inspections of every task and step impractical. Accordingly, the Department of Education will use a quality-assurance review process to monitor the Contractor’s performance under this contract. Specifically, the COTR will assess the contractor’s performance across a set of tailored rating elements for each of eight (8) key deliverables:

1. Direct assessment training materials;

2. Wave 1 data analysis outline;

3. Report on wave 1 data;

4. Waves 1 and 2 data analysis outline;

5. Report on waves 1 and 2 data;

6. Waves 1, 2, and 3 data analysis outline;

7. Report on waves 1, 2, and 3 data;

8. Database on CD-ROM with documentation.

Rating Elements and Standards of Performance for Key deliverables

The Contractor’s performance shall be evaluated at the end of the preparation period with the direct assessment training materials, at the end of each of the three (3) waves of data collection with the data analysis outlines and the data analysis reports, and at the end of the project with the CD-ROM database.

1. Direct assessment training materials:

(a) Quality of the training materials

...where acceptable performance would include clarity of language used in the materials, logical sequence to the training tasks, and ease of use of the materials;

(b) Completeness of the instructions for assessment administration

...where acceptable performance would include thoroughly specified instructions that leave no room for varied interpretations and, thus, will ensure standardized test administration;

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

2. Wave 1 data analysis outline:

(a) Preliminary exploratory analysis

...where acceptable performance would include evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis;

(b) Rationale for selection of statistical procedures

...where acceptable performance would include sound, creditable, comprehensive approaches to data analysis;

...where acceptable performance would include adequately addressing key issues relevant to elementary school children in special education;

(d) Timeliness

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

3. Report on wave 1 data:

(a) Comprehensiveness, clarity, and organization of design

...where acceptable performance would include complete, clear, efficient approaches to addressing the issues, clear writing style, proper grammar/spelling, and clearly organized document format in the first draft;

(b) Responsiveness to ED’s comments and suggestions

...where acceptable performance would include thoughtful consideration to ED’s comments and suggestions and incorporation of those suggestions in the final report;

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

4. Waves 1 and 2 data analysis outline:

(a) Preliminary exploratory analysis

...where acceptable performance would include evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis;

(b) Rationale for selection of statistical procedures

...where acceptable performance would include sound, creditable, comprehensive approaches to longitudinal data analysis;

...where acceptable performance would include adequately addressing key issues relevant to elementary school children in special education;

(d) Timeliness

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

5. Report on waves 1 and 2 data:

(a) Comprehensiveness, clarity, and organization of design

(b) Responsiveness to ED’s comments and suggestions

...where acceptable performance would include thoughtful consideration to ED’s comments and suggestions and incorporation of those suggestions in the final report;

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

6. Waves 1, 2, and 3 data analysis outline:

(a) Preliminary exploratory analysis

...where acceptable performance would include evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis;

(b) Rationale for selection of statistical procedures

...where acceptable performance would include sound, creditable, comprehensive approaches to longitudinal data analysis;

...where acceptable performance would include adequately addressing key issues relevant to elementary school children in special education;

(d) Timeliness

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

7. Report on waves 1, 2, and 3 data:

(a) Comprehensiveness, clarity, and organization of design

(b) Responsiveness to ED’s comments and suggestions

...where acceptable performance would include thoughtful consideration to ED’s comments and suggestions and incorporation of those suggestions in the final report;

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

8. Database on CD-ROM with documentation:

(a) Organization of the data

...where acceptable performance would include a database that is organized in a logical format for clarity and ease of use;

(b) Accuracy and user friendliness of documentation

...where acceptable performance would include complete, clear, accurate documentation of variable names and labels, value labels, codes for missing values, descriptions of procedures used to compute analysis variables, and documentation of all edit and consistency checks used to clean the data;

...where acceptable performance would include a deliverable that is received on time or within a reasonable amount of time (i.e., within one (1) day of the date that it is due).

Process of Quality Assurance Assessment

While quality assurance is closely tied to these performance standards for deliverable content, cost is also an important consideration in the assessment of Contractor performance. The Contractor’s cost performance will be evaluated by the Department at the end of the contract.

In the event of an excusable delay (as defined in FAR 52.249-14), the Department and the Contractor shall work together to modify the contract in regard to the due dates of the deliverables. If such an event were to occur that would require a modification to the due dates of the deliverables, the Contractor’s performance, where applicable in this QASP, shall be measured by the date agreed upon in the modification.

The COTR will use the appropriate key deliverable evaluation forms (Exhibits A: Direct Assessment Training Materials; B: Wave 1 Data Analysis Outline; C: Report on Wave 1 Data; D: Waves 1 and 2 Data Analysis Outline; E: Report on Waves 1 and 2 Data; F: Waves 1, 2, and 3 Data Analysis Outline; G: Report on Waves 1, 2, and 3 Data; and H: Data on CD-ROM with Documentation) to document and evaluate the Contractor’s performance for each of the key deliverables under this contract. The COTR’s evaluation of the quality of that deliverable will serve as the overall evaluation score.

Each key deliverable will be evaluated in accordance with the following definitions of Contractor performance:

1) Unacceptable. Level of performance which is not acceptable and which fails to meet the minimum standards of performance, resulting in the Contractor receiving a reduction in targeted fee for that deliverable;

2) Acceptable. Level of performance which meets the minimum standards of performance, resulting in the Contractor receiving its targeted fee for that deliverable; or

3) Superior. Level of performance which exceeds the minimum standards of performance, resulting in a bonus over targeted fee for that deliverable.

Incentive fees for the key deliverables will be assessed as follows:

Direct assessment training materials

Superior: Target fee plus $2,000

Unacceptable: Target fee minus $4,000

Wave 1 data analysis outline

Superior: Target fee plus $3,000

Unacceptable: Target fee minus $5,000

Report on wave 1 data

Superior: Target fee plus $5,000

Unacceptable: Target fee minus $8,000

Waves 1 and 2 data analysis outline

Superior: Target fee plus $3,000

Unacceptable: Target fee minus $5,000

Report on waves 1 and 2 data

Superior: Target fee plus $5,000

Unacceptable: Target fee minus $8,000

Waves 1, 2, and 3 data analysis outline

Superior: Target fee plus $3,000

Unacceptable: Target fee minus $5,000

Report on waves 1,2, and 3 data

Superior: Target fee plus $5,000

Unacceptable: Target fee minus $8,000

Database on CD-ROM with documentation

Superior: Target fee plus $2,000

Unacceptable: Target fee minus $3,000

Total dollar amount of fee increase possible due to superior performance (deliverable quality): $28,000. Total dollar amount of fee decrease possible due to unacceptable performance (deliverable quality): $46,000.

The COTR will forward all completed QA monitoring forms according to the following schedule:

Direct assessment training materials: submitted by the close of business 20 working days from the date the final materials were received by the COTR.

Wave 1 data analysis outline: submitted by the close of business 20 working days from the date the outline is received by the COTR.

Report on wave 1 data: submitted by the close of business 20 working days from the date the final report is received by the COTR.

Waves 1 and 2 data analysis outline: submitted by the close of business 20 working days from the date the outline is received by the COTR.

Report on waves 1 and 2 data: submitted by the close of business 20 working days from the date the final report is received by the COTR.

Waves 1, 2, and 3 data analysis outline: submitted by the close of business 20 working days from the date the outline is received by the COTR.

Report on waves 1, 2, and 3 data: submitted by the close of business 20 working days from the date the final report is received by the COTR.

Database on CD-ROM with documentation: submitted by the close of business 20 working days from the date the database and documentation are received by the COTR.

For the purposes of documentation, the Contractor may respond in writing to any unacceptable evaluation within 5 working days after receipt of the form with the score; however, this does not mean that the final score will be changed.

The CO will review each key deliverable evaluation form prepared by the COTR. When appropriate, the CO may investigate the event further to determine if all the facts and circumstances surrounding the event were considered in the opinions outlined on the forms. The CO will immediately discuss every deliverable receiving an unacceptable rating with the Contractor to assure that corrective action is promptly initiated. Discussion with the Contractor of unacceptable performance on deliverables does not negate the Department’s right to terminate the Contractor for default for poor performance per FAR 52.249-6, Termination (Cost Reimbursement).

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: DIRECT ASSESSMENT TRAINING MATERIALS

DATE: ________________________________

Rating Element 1: Quality of training materials

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include unclear language used in the materials, illogical sequence of the training tasks, and difficulty in use of the materials.

Acceptable performance (5-7) would include clarity of language used in the materials, logical sequence of the training tasks, and use of use of the materials.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful approaches.

Supporting comments (required for unacceptable or superior performance ratings):

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Rating Element 2: Completeness of the instructions for assessment administration

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include incomplete instructions and ambiguous language.

Acceptable performance (5-7) would include thoroughly specified instructions that leave no room for varied interpretations.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful approaches.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include late delivery (i.e., received more than one (1) day after the date that it is due, per the schedule in the contract) for the training materials. Four (4) points should be given if the deliverable is received two (2) working days late; three (3) points should be given if the deliverable is received three (3) working days late; two (2) points should be given if the deliverable is received four (4) working days late; one (1) point should be given if the deliverable is received five (5) working days late; no points should be given if the deliverable is received six (6) or more working days late.

Acceptable performance (5-7) would include timely delivery of the training materials (i.e., within one (1) day of the date that it is due, per the schedule in the contract). Seven (7) points should be given if the deliverable is received on the date due; five (5) points should be given if the deliverable is received one (1) working day late.

Superior performance (8-10) would include early (i.e., before the date that it is due, per the schedule in the contract) delivery of the training materials. Eight (8) points should be given if the deliverable is received one (1) working day early; nine (9) points should be given if the deliverable is received two (2) to three (3) working days early; ten (10) points should be given if the deliverable is received four (4) or more days early.

Supporting comments (required for unacceptable or superior performance ratings):

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: WAVE 1 DATA ANALYSIS OUTLINE

DATE: ________________________________

Rating Element 1: Preliminary exploratory analysis

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include no evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis.

Acceptable performance (5-7) would include evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful approaches.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 2: Rationale for selection of statistical procedures

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include missing, illogical, unclear, inappropriate approaches to data analysis.

Acceptable performance (5-7) would include sound, creditable, comprehensive approaches to data analysis.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful approaches and/or methods for data analysis.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Relevance

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include missing or inadequately addressed key issues relevant to elementary school children in special education.

Acceptable performance (5-7) would include adequately addressed key issues relevant to elementary school children in special education.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include exceptional insight into the key issues relevant to elementary school children in special education.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 4: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Supporting comments (required for unacceptable or superior performance ratings):

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: REPORT ON WAVE 1 DATA

DATE: ________________________________

Rating Element 1: Comprehensiveness, clarity, and organization of design

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include unclear language, grammar/spelling errors, omission of key points, and poor organization of the text in the first draft.

Acceptable performance (5-7) would include complete, clear, efficient approaches to addressing the issues, clear writing style, proper grammar/spelling, and clearly organized document format in the first draft.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful reporting on data analysis results in the first draft.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 2: Responsiveness to ED’s comments and suggestions

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include unsubstantiated disregard for ED’s comments and suggestions for revisions throughout the editing process.

Acceptable performance (5-7) would include thoughtful consideration to ED’s comments and suggestions and incorporation of those suggestions in the final report..

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative. exceptionally skillful handling of revisions based on ED’s comments and suggestions.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Supporting comments (required for unacceptable or superior performance ratings):

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: WAVES 1 AND 2 DATA ANALYSIS OUTLINE

DATE: ________________________________

Rating Element 1: Preliminary exploratory analysis

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include no evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis.

Acceptable performance (5-7) would include evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful approaches.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 2: Rationale for selection of statistical procedures

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include missing, illogical, unclear, inappropriate approaches to longitudinal data analysis.

Acceptable performance (5-7) would include sound, creditable, comprehensive approaches to longitudinal data analysis.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful approaches and/or methods for longitudinal data analysis.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Relevance

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include missing or inadequately addressed key issues relevant to elementary school children in special education.

Acceptable performance (5-7) would include adequately addressed key issues relevant to elementary school children in special education.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 4: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Supporting comments (required for unacceptable or superior performance ratings):

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: REPORT ON WAVES 1 AND 2 DATA

DATE: ________________________________

Rating Element 1: Comprehensiveness, clarity, and organization of design

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include unclear language, grammar/spelling errors, omission of key points, and poor organization of the text in the first draft.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful reporting on data analysis results in the first draft.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 2: Responsiveness to ED’s comments and suggestions

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include unsubstantiated disregard for ED’s comments and suggestions for revisions throughout the editing process.

Acceptable performance (5-7) would include thoughtful consideration to ED’s comments and suggestions and incorporation of those suggestions in the final report..

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Supporting comments (required for unacceptable or superior performance ratings):

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: WAVES 1, 2, AND 3 DATA ANALYSIS OUTLINE

DATE: ________________________________

Rating Element 1: Preliminary exploratory analysis

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include no evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis.

Acceptable performance (5-7) would include evidence that the recommended analysis is based on patterns observed in preliminary exploratory data analysis.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful approaches.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 2: Rationale for selection of statistical procedures

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include missing, illogical, unclear, inappropriate approaches to longitudinal data analysis.

Acceptable performance (5-7) would include sound, creditable, comprehensive approaches to longitudinal data analysis.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Relevance

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include missing or inadequately addressed key issues relevant to elementary school children in special education.

Acceptable performance (5-7) would include adequately addressed key issues relevant to elementary school children in special education.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 4: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Supporting comments (required for unacceptable or superior performance ratings):

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: REPORT ON WAVES 1, 2, AND 3 DATA

DATE: ________________________________

Rating Element 1: Comprehensiveness, clarity, and organization of design

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include unclear language, grammar/spelling errors, omission of key points, and poor organization of the text in the first draft.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful reporting on data analysis results in the first draft.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 2: Responsiveness to ED’s comments and suggestions

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include unsubstantiated disregard for ED’s comments and suggestions for revisions throughout the editing process.

Acceptable performance (5-7) would include thoughtful consideration to ED’s comments and suggestions and incorporation of those suggestions in the final report..

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Supporting comments (required for unacceptable or superior performance ratings):

QUALITY ASSURANCE SURVEILLANCE PLAN

EXHIBIT A: DATABASE ON CD-ROM WITH DOCUMENTATION

DATE: ________________________________

Rating Element 1: Organization of the data

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include illogical format and poor organization.

Acceptable performance (5-7) would include organization in a logical format for clarity and ease of use.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative, exceptionally skillful organization of data.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 2: Accuracy and user friendliness of documentation

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Unacceptable performance (1-4) would include incomplete or inaccurate documentation of variable names and labels, value labels, codes for missing values, procedures used to compute analysis variables, and description of all edit and consistency checks used to clean the data.

Acceptable performance (5-7) would include complete, clear, accurate documentation of variable names and labels, value labels, codes for missing values, descriptions of procedures used to compute analysis variables, and description of all edit and consistency checks used to clean the data.

Superior performance (8-10) would meet acceptable performance standards for this rating element, AND include innovative. exceptionally skillful handling of the database documentation.

Supporting comments (required for unacceptable or superior performance ratings):

Rating Element 3: Timeliness

Circle the appropriate number for your rating:

1 2 3 4 5 6 7 8 9 10

where:

Supporting comments (required for unacceptable or superior performance ratings):

Printable view

Last Modified: 10/06/2003

How Do I Find...

Information About...