STATEMENT OF WORK WORKSTATEMENT -- QUICK-RESPONSE INFORMATION SYSTEM (QRIS) TABLE OF CONTENTS Page I. Introduction 2 A. Background/Purpose of the Procurement 2 B. Basic Design of the System 4 C. Description of the System 5 II. Nature of this Procurement and Scope of Work 6 A. Nature of Procurement 6 B. Scope of Work 7 III. Work Statement: Tasks 8 Task 1-Initial Meeting with NCES....................... 9 Task 2- Systems Clearance............................... 9 Task 3- Coordination with NCES and with Other Groups... 10 Task 4- Sample Design and Selection 14 Task 5- Questionnaire Development and Data Analysis Planning 20 Task 6- Data Collection and Follow-Up 25 Task 7- Data Processing and Analysis 28 Task 8- Report Preparation and Dissemination 30 Task 9- PEQIS Panel Evaluation 33 Task 10- System Maintenance 33 Task 11- Project Management Activities.........................34 IV. Summary of Deliverables 34 V. Other Contract Requirements 36 STATEMENT OF WORK QUICK-RESPONSE INFORMATION SYSTEM (QRIS) I. INTRODUCTION A. Background/Purpose of the Procurement The National Center for Education Statistics (NCES) regularly conducts surveys and studies in order to carry out its mission "to collect, analyze, and disseminate statistics and other data related to education in the United States and in other nations" (GEPA, Sec. 406 (b) as amended). At the elementary/secondary and postsecondary education levels many large-scale, comprehensive surveys and systems of surveys such as the Common Core of Data (CCD), the Schools and Staffing Survey (SASS), the Integrated Postsecondary Education Data System (IPEDS), and the National Postsecondary Student Aid Study (NPSAS) are conducted to meet this mandate. However, Federal education policymakers frequently make specific inquiries for data that are not collected by NCES's regular, recurring surveys. Because of the ad hoc nature of these requests, the content, the population of interest, or the periodicity of NCES's large-scale surveys are not adequate to provide data quickly to address them. Therefore, in order to address emerging policy issues and provide data on topics not covered by its ongoing large-scale surveys, NCES has established the Quick-Response Information System (QRIS). The QRIS has two components: the Fast Response Survey System (FRSS), which was established in 1976, and the Postsecondary Education Quick Information System (PEQIS), established in 1991. The FRSS and the PEQIS are similar in that the survey topics of each vary widely; they are, for the most part, one-time data collections; they are conducted quickly, usually within a year; they use generalized components and procedures as much as possible; and they impose a minimum burden on the respondents. The FRSS and the PEQIS, however, survey very different populations. For example, previous FRSS surveys (approximately 66 have been completed to date) have been directed toward State Education Agencies (SEAs) and Local Education Agencies (LEAs)--school districts, public elementary/secondary schools, private elementary/secondary schools, and public libraries. FRSS surveys are all sample surveys with well-defined universes except for surveys of SEAs, which are census surveys. Previous PEQIS surveys, on the other hand, have all been directed toward a standing panel of postsecondary education institutions. Each of the institutions on the PEQIS institutional panel has agreed to participate in the PEQIS and has appointed an institutional coordinator who facilitates data collection by identifying the appropriate campus respondent for each survey (depending on the topic) and forwarding the questionnaire to that person. Approximately 10 PEQIS surveys have been completed to date. The contractor may wish to review the various topics covered by previous PEQIS surveys by browsing the NCES Web site at: http://nces.ed.gov. The PEQIS also has the capability to do universe surveys of the State Higher Education Agencies (SHEAs) and sample surveys of less-than-two-year institutions (which are not part of the current PEQIS institutional panel). NCES seeks to retain all the features of the present QRIS, which also allows for other sample units (e.g., households and businesses), second-stage sampling (e.g., students and teachers within elementary/secondary schools and postsecondary institutions), data collection methods in addition to traditional mail questionnaires (e.g., computer-assisted telephone interviews--CATI), user-friendly products (e.g., public-use files, complete with documentation), products (files) for the NCES Web site, and even concurrent, coordinated elementary/secondary and postsecondary education surveys. This iteration of the QRIS will be a performance-based contract. That is, the work performed on this contract is subject to the technical review of the COTR and will be evaluated in terms of how well the requirements of each task are satisfied and the extent to which the work performed met the time schedule for each task. The contractor's performance will be evaluated annually in terms of the performance standards set forth in the Surveillance Plan in Section H. Only the final products from tasks that have been completed during the annual evaluation period will be evaluated. Interim products leading to final products will be evaluated in terms of their contributions to final products, but not separately. The performance standards will allow the government to make an objective assessment of the contractor's performance based on the quality, completeness, and timeliness of its technical work. B. Basic Design of the System In order to meet NCES's needs for a variety of data on a quick-turnaround basis, the QRIS must have the capability to conduct surveys at the public and private elementary/secondary education level, and at the postsecondary education level. Although mail surveys will probably continue to be the most cost-effective data collection method for the majority of quick-response surveys, the system shall have provisions for the use of other methods when timing prohibits mail out/mail back or for complex issues or certain samples that require the use of an interviewer. The QRIS system shall have the following capabilities: 1.Survey institutional respondents in all major educational sectors. 2. Survey individuals within educational institutions, households, and business establishments. 3. Provide reliable data on a variety of policy-related topics quickly, efficiently, and with minimum response burden. 4. Provide national estimates of characteristics of importance, cross-tabulations of survey items with other available items, and other analyses. 5. Combine data from the quick-response surveys with data from other NCES surveys. 6. Report and publish findings. 7. Disseminate reports to appropriate interest groups and audiences. 8. Produce public-use files and documentation. 9. Produce files for the NCES Web site. C .Description of the System Since speed is essential to the success of the project, pre-established procedures for conducting surveys shall be used as much as possible. Most QRIS procedures are common to both the elementary/secondary and postsecondary education sectors. However, conducting FRSS surveys and PEQIS surveys, with their emphasis on two different education sectors will have certain unique procedures. The procedures unique to the FRSS and the PEQIS are described in the subtasks listed under Tasks 3, 4, 6, and 8. The following activities are common to both the FRSS and the PEQIS and shall be undertaken when conducting a survey. o Meet with the survey requester o Research the topic o Identify the appropriate respondent group o Develop the survey instrument o Obtain government clearance o Collect the data o Process and analyze the data o Prepare a survey report and disseminate the findings o Produce public-usefiles and documentation o Produce files for the NCES Web site II. Nature of this Procurement and Scope of Work A. Nature of Procurement The Federal Government will award a five-year performance-based contract. Due to the uncertainty of the number of surveys that will be needed by the Department's principle Offices (including NCES) and other Federal agencies, the contractor may be required to perform a varying number of surveys each year. Conducting FRSS and PEQIS surveys involves a succession of tight deadlines, close interaction among NCES, the contractor, the data requester, the Information Management and Compliance Division (IMCD) of the Department, the Office of Management and Budget (OMB), State or institutional coordinators, and sometimes national groups representative of the respondent sector. The contract calls for an operational environment that allows for expeditious exchanges of telephone calls, letters (including some originals on NCES letterhead with the Commissioner's signature), questionnaires, clearance packages, camera-ready copies of reports for printing, documents, tabulations, and meetings called on short notice. Therefore, the contractor must have staff able to interact on short notice with NCES personnel and must be able to produce and send materials to NCES in a matter of a few hours. Year 5 is a year of scaled-down activity intended as an overlap year with a new QRIS contract. Therefore, most of the estimated fifty (55) QRIS surveys to be conducted shall be conducted during the first four years of the contract. Year 5 shall primarily be spent in finishing reports and documenting files. Year 5 shall also serve as a transition year to a new contractor to conduct the QRIS. If the new contractor is not the current contractor, the current contractor shall facilitate the transition to a new contractor by providing materials and information to the new contractor. These activities shall begin 30 days before the expiration of current contract. B. Scope of Work It is estimated that during the life of the contract, a maximum of fifty-five (55), and a minimum of thirty-three (33) surveys will be conducted (see "Instructions for Proposal Development" for details). The activities to be performed for each survey conducted are described in detail below. III. Work Statement - Tasks The following major Tasks are required for each survey that shall be conducted. Task 1 - Initial Meeting with NCES Task 2 - Systems Clearance Task 3 - Coordination Task 4 - Sample design and selection Task 5 - Questionnaire development Task 6 - Data collection Task 7 - Data processing and analysis Task 8 - Report preparation Task 9 - PEQIS Panel Assessment and Reselection Task 10 -System Maintenance Task 11 - Project Management Tasks 3 through 8 describe the activities involved in conducting a survey once the potential respondent and the topic have been determined. These activities include researching the topic, selecting the sample, designing the questionnaire, fielding the survey, collecting the data, preparing the data file, analyzing the data, writing the reports, and preparing files for the NCES Web site and public-use data files with documentation. Some of the activities described in Task 4 may be performed concurrently with those under Task 5 to expedite survey operations. Procedures for these tasks may vary slightly, depending on whether a particular survey is an FRSS survey (e.g., requests information from an elementary/secondary school) or a PEQIS survey (e.g., requests information from colleges and universities). The activities in Tasks 10 and 11 relate to the maintenance of the system in each year of the contract. Task 10 activities include: maintenance of the respondent networks and maintenance of the system files and programs. Task 11 relates to the ongoing management activities of the project in each year of the contract. The activities listed under Task 11 include time schedules and progress reports. Task 1 - Initial Meeting with NCES Within one week of contract award, the contractor's project director and other key project staff shall meet with the NCES Contracting Officer's Technical Representative (COTR) and the CPO contract specialist to review the procedures for FRSS and PEQIS tasks and to discuss issues relating to the proposed project staffing plan, management requirements, obtaining OMB clearance for the QRIS system, the contract transition, and the conduct of the work. These discussions will not alter the specifications or the design of the contract but will provide management information for use by both the contractor and NCES in monitoring the work to be performed. Task 2 -- Obtain Systems Clearance for the QRIS The first task after the contractor's initial meeting with NCES is to obtain OMB clearance for the QRIS system. The contractor shall prepare and submit to NCES an OMB clearance package for this purpose. After OMB has granted systems clearance for the QRIS (FRSS-PEQIS), it will then expedite its review of individual surveys and will provide comment/clearance in 4 weeks or less. Task 3 - Coordination with NCES and with Other Groups Prospective data requesters will be asked to complete a Survey Request Form and read a booklet titled "Information for Data Requesters" (see Attachments C & D). NCES will perform a preliminary assessment of all surveys requested in this manner. In the preliminary assessment, the contractor shall confer with NCES on such matters as the feasibility of satisfying the requester's data requirements, the scheduling of proposed surveys, and the suitability of sampling frames and possible sample designs. Based on the initial assessment, NCES will decide whether or not to proceed with the requested survey. If NCES does decide to proceed, the NCES COTR will sign the Survey Request Form and forward it to the CO for her signature, after which, work will begin. NCES receives a number of requests for quick-response surveys; however, the actual conduct of a survey (that is, data collection) is not guaranteed until the survey is cleared by the OMB. The result is that some surveys are deemed infeasible at the outset while others may involve meetings, research, multiple questionnaire drafts or other activities before it is determined whether or not the survey is feasible. About one-quarter of proposed surveys turn out to be infeasible for one reason or another, usually at a relatively early stage of development. In performing the activities required under the tasks, the contractor shall submit a number of products for NCES review and approval. The NCES COTR will communicate necessary revisions to the contractor by telephone, by written comments on a draft document, or in a meeting called for such a purpose. Generally, products will be reviewed by NCES within 1 or 2 weeks, depending on the priority of the product. Exceptions to this include the review of final drafts of the survey report. The current NCES publication review process, including review by the data requester and peer review, may take up to 8 weeks, depending on the quality of initial drafts. Conducting FRSS and PEQIS surveys expeditiously typically has involved a number of face-to-face meetings between the contractor, the data requester, and the NCES COTR. However, it is possible to conduct some of the necessary discussions through conference calls. These meetings or calls may include, among other things, discussion of draft questionnaires, clearance packages, pretest results, sampling frames, sample designs, data analyses, data files, and reports. Since numerous surveys usually will be in progress simultaneously, meetings may involve more than one survey. With input from the contractor and the data requester, the NCES COTR will set the agenda for the meetings. When meetings are necessary, they shall be held at NCES in Washington, D.C., and the contractor shall be fully prepared to discuss all agenda items. 3.1 Outside support for surveys A. Respondent Groups Solicitation of support from outside of NCES has been effective in increasing response rates for past FRSS surveys. NCES will obtain the support of appropriate organizations representing the potential survey respondents. Usually this support will come from a small group representing the respondent organization, which will expedite the coordination of FRSS surveys. For example, at the elementary/secondary education level, the support of the Fast Response Panel of the Education Information Advisory Committee (EIAC) in the Council of Chief State School Officers (CCSSO) may be requested. In previous contracts, proposed elementary/secondary survey questionnaires and related information have been discussed with the EJAC Fast Response Panel in telephone conference calls. Library surveys have been discussed with such organizations as the American Library Association and the National Commission on Libraries and Information Sciences. For any organization whose support is sought, the contractor shall arrange a telephone conference call involving NCES staff, the data requester, the contractor, and several members of the group representing the organization to obtain that organization's comments and support for the survey. Organizations representing the respondent group will be identified by NCES. As a result of the recommendations from the organization consulted, the contractor shall revise the questionnaire and cover letter, as required, incorporating the NCES approved changes. B . Elementary/Secondary Surveys: State Coordinators Each Chief State School Officer has designated a State Coordinator under the jurisdiction of the State Education Agencies (SEAs) to coordinate surveys concerning elementary/secondary education. The assistance of these coordinators in obtaining the cooperation of the sample units is essential to the success of surveys conducted under the FRSS. Data collection and follow-up require an optimum combination of tact and perseverance. It is essential to conduct these tasks according to the data collection preferences in each state. For each survey of elementary/secondary schools or districts, State Coordinators will be requested to specify their preference among three alternative procedures for mailing and returning questionnaires. The following options are available: 1. Group A - States participating in the data collection and follow-up Questionnaires are mailed to respondents by the contractor through the State Coordinators, and returned by the respondents to the contractor through the State Coordinators, who assume full responsibility for obtaining responses in their states. 2. Group B - States participating in the mailout but not in the follow-up Questionnaires are mailed to respondents by the contractor through the State Coordinators. However, respondents return the completed questionnaires directly to the contractor. 3. Group C - States not participating in the data collection procedures Questionnaires are mailed by the contractor directly to the respondents, who then return the completed forms directly to the contractor. Currently there are nine (9) states in Group A, six (6) in Group B, and 36 in Group C. The contractor shall draft, for NCES's review and approval, a letter informing the State Coordinator in each state of the scheduled survey and the respondents in the new sample. The letter shall contain a list of the districts and/or schools selected for the sample in the State Coordinator's state. The contractor shall, upon request, also prepare a letter for school districts (LEAs) informing them of the schools in their district that have been selected for the survey. In surveys of individuals, the contractor shall follow the NCES confidentiality requirements when information concerning sample selection is released. C. Postsecondary Education Surveys: Institutional Coordinators For postsecondary education surveys (i.e., PEQIS surveys), the president of each institution in the PEQIS's standing panel has designated an institutional coordinator. Since institutional respondents for PEQIS surveys will vary by the topic of the survey, the coordinators facilitate data collection by identifying the appropriate institutional respondent for a given PEQIS survey and forwarding the questionnaire to that person. Institution coordinators will also provide faculty lists when second-stage sampling is required to survey faculty within institutions. In view of the central role that institutional coordinators play in the data-collection and follow-up process, the contractor shall develop and maintain good working relationships with them. Task 4 - Sample design and selection A. National Samples In sectors in which surveys based on samples can provide accurate national estimates, the contractor shall select samples. For surveys conducted at the elementary/secondary education level, an appropriate national probability sample shall be selected for all sectors except State Education Agencies (SEAs). The SEA sector (covering the 50 States and the District of Columbia), in contrast with the other sectors of elementary/secondary education, is so small that a census is required to provide accurate results. That is also true of the State Higher Education Executive Offices (SHEEO's), who are the designated agency respondents to PEQIS state surveys of State Higher Education Agencies. B. Universes of possible interest for FRSS 1. Elementary/secondary public schools 2. Elementary/secondary private schools 3. Local education agencies (LEA) 4. State education agencies (SEA) 5. Households - with elementary/secondary participants; with postsecondary participants; general population 6. Libraries - public and school 7. Employers - commercial establishments 8. Other education providers - museums, labor unions, community centers 9. Students, faculty, or other staff in elementary/secondary schools or postsecondary institutions 10. Other education consumers 11. State higher education executive officers and agencies 12. Public and private (for profit and not-for-profit) 2- and 4-year postsecondary institutions 13. Public and private (for profit and not-for-profit) less-than-2-year postsecondary institutions After discussion with the contractor, NCES will specify the sectors requiring sample selection based on the requirements of the survey requests. The contractor shall select the samples for each elementary/secondary education survey and establish the communications channels with the new respondents. Samples for postsecondary education surveys are selected from the pre-established panel of postsecondary education institutions. Sample selection for all surveys, therefore, occurs when a new survey request is deemed feasible. In addition to the major education sectors listed in Tables A and B, future urgent requests for data may necessitate expanding the FRSS to include other populations. In all cases of sample selection, NCES will identify the appropriate universe as well as the sampling frame, if one is available. 4.1 - Sample design The contractor shall design, for review and approval by NCES, a new sample for each survey including the determination of the sample size, the selection of the stratification variables, the allocation of sample to strata, the method of selection, and the provision for easy and accurate variance estimation. The sample design should result in estimates that have a coefficient of variation for major variables of approximately 5 to 7 percent. Both estimates of totals, where the variable is continuous, and estimates of proportions, where the variable is an attribute, are often important in the same survey. The contractor shall provide a written description of the proposed sample design and indicate how it will meet the survey precision needs. In most cases, NCES will provide or specify a sufficiently complete sampling frame for sample selection. Occasionally, major data elements on one frame may require editing. The contractor shall discuss with NCES the required accuracy of these major data elements and shall edit the frame after discussion with NCES. The contractor shall also edit auxiliary variables that are added to the file to increase its utility. All editing decisions will be implemented subject to NCES approval. On occasion, information from non-computerized sources may be required, such as listings or directories. In this case the available information shall be added to the sample file rather than to the universe file. 4.2 - Sample selection The contractor shall select the sample according to the approved sample design, document the sample characteristics in a written report, and provide to NCES a listing of the selected respondents so that the accuracy of the selection process can be verified. C. Approximate sample sizes for FRSS elementary/secondary (E/S) universes Table A below provides typical sample and universe sizes for previous FRSS surveys. To limit respondent burden, the initial sample sizes in each sector shall not be more than 20 percent larger than the sizes indicated in Table A. The fixed sample sizes will result in varying precision for major variables. On occasion greater precision than these sample sizes allow may be desirable, in which case the sample size constraint may not be appropriate. For heavily used sectors, samples shall be selected so as to minimize the overlap between old and new respondents. Table A. Elementary/Secondary Universe and Samples as of Fall 1996 Sector Sample size Population size Universe source LEA 800 15,000 NCES Public School District Universe Public el/sec school 900 87,000 NCES Public School Universe Private el/sec schools 800 26,000 NCES Private School Universe Public libraries 850 9,800 NCES Public Library Universe Public school teachers 1,350 2,500,000 NCES Public School Universe D. PEQIS sample selection For PEQIS surveys, the contractor shall select a sample from the pre-established panel of 1,628 postsecondary education institutions. Table C. provides the number of institutions in the sectors of the existing postsecondary education panel (see page 19). These institutions have agreed to participate and provide responses to postsecondary education data requests quickly. This panel of institutions provides coverage for certain sectors of postsecondary education as exhibited in Tables B and C on pages 18 & 19. Less-than-2-year institutions are not currently represented on the panel. However, the contractor shall draw a sample of these institutions when they are the focus of a survey topic. The current panel of institutions provides national estimates for major segments of the postsecondary education universe, e.g., institutions of higher education, research institutions, and community colleges. The panel contains stand-alone subpanels to accommodate survey topics that are targeted toward particular segments of the postsecondary education panel. Table B --Number of institutions in the current PEQIS universe by institutional level and Control, and IHE status*: Fall 1996 IHE Status Institutional level and control Total IHE Non-IHE Total 5,353 3,706 1,674 4-year 2,787 2,244 543 Public 619 608 11 Private nonprofit 1,966 1,519 447 Private for-profit 202 117 85 2-year 2,550 1,446 1,104 Public 1,264 1,045 219 Private nonprofit 610 183 427 Private for-profit 676 218 458 Less than 2-year 16 16 0 Public 2 2 0 Private nonprofit 4 4 0 Private for-profit 10 10 0 *Institutions of higher education Table C --PEQIS institutional panel, classification by institutional level and control, and IHE status*: Fall 1996 IHE Status Institutional level and control Total IHE Non-IHE Total 1.628 1,358 270 4-year 937 845 92 Public 398 396 2 Private nonprofit . 442 394 48 Private for-profit 97 55 42 2-year 689 511 178 Public 463 418 45 Private nonprofit 110 51 59 Private for-profit 116 42 74 Less than 2-year 2 2 0 Public 1 1 0 Private nonprofit 1 1 0 Private for-profit 0 0 0 *Institutions of higher education The current panel has been established with provisions to replace the panel with new institutions for continued representation for each subpanel. The contractor shall submit a plan to NCES for approval indicating the procedures that shall be used to ensure that the transition from the current contractor for PEQIS is done efficiently, and that there will be adequate representation of all segments in the postsecondary institution panel throughout the contract period designated to include postsecondary education surveys. The plan shall outline the steps required for the postsecondary system's maintenance and shall include a draft letter to the coordinators of the current PEQIS panel informing them in detail of the transfer to a new contractor--and any other modifications to the initial agreement to participate. E. Random Digit Dialing (RDD) When requests for surveys of individuals in households are accepted, the contractor shall design, for review and approval by NCES, a sampling plan using RDD. The written description of the proposed sample design should, at a minimum, include: a description of the recommended RDD approach, along with justification for the recommended approach; a discussion of the key estimates that are being considered in the development of the design; a description of one or more designs, including sample sizes and variance estimates for the key statistics under each design; and a discussion of the advantages and disadvantages of each design. The proposed sample size shall be based on a yield of approximately 1,200 completed interviews. The selection of the final design will be based on a number of factors and determined by NCES and the data requester. Task 5 - Questionnaire Development A. Questionnaire Development/Data Analysis Plan After NCES accepts a survey request, the contractor shall develop an effective data-collection instrument as discussed below and shall coordinate its activities with NCES and the data requester through the NCES COTR. NCES must approve the instrument prior to the pretest and printing of the final survey instrument. The development of the questionnaire and a short transmittal letter encompasses a number of activities. The primary responsibilities for these activities are divided between NCES and the contractor and are listed and described below. For a given survey all the meetings described below may not be necessary, while certain additional ones may be required. For example, for a given survey the contractor may also seek the assistance of outside experts in the field, to solicit their suggestions and comments for the questionnaire, or NCES and the contractor may convene a focus group to solicit input. After NCES approves the draft of the data items, the contractor shall develop and format a draft of the questionnaire and submit it to NCES and the data requester. This draft will be reviewed to determine whether it meets the data requester's needs. Typically, more than one draft is necessary. The contractor also shall prepare a short transmittal letter to the prospective respondents and information letters to institutional coordinators, State Coordinators, and district superintendents, as required. The following describes the questionnaire-development process. This process shall be followed whether the survey is conducted through a self-administered mail questionnaire or through CATI. 1. Define Data Needs NCES will hold a preliminary meeting with the requester to ascertain the purpose and nature of the survey request. The data requester will specify the data needs and the goals or expected outcomes of the proposed survey. The data requester and the NCES COTR will then meet with the contractor to specify the data issues the survey will address. 2. Research the Topic The contractor shall research the topic of the proposed survey. This task may include such activities as literature searches, phone calls to members of the sector to be surveyed, and consultation with experts in the field. The research should clarify the issues involved in the topic, summarize any existing studies, and identify any recent surveys on the topic. 3. Meet with the Requester and Develop a Data Analysis Plan The contractor shall meet with the data requester and NCES, and shall present a brief summary of its research findings, along with a draft of data items (if necessary) for discussion. The meeting will further clarify data needs including an analysis plan and will identify possible solutions to problems. 4. Prepare Drafts of Questionnaire and Transmittal Letter Questionnaire development is particularly challenging because the PEQIS and the FRSS both limit the questionnaire's size and the respondents' burden. To be an effective survey instrument, the questions must be clear, concise, meaningfully related, unambiguous, must elicit accurate and consistent responses, and must meet the criteria listed below. The contractor shall develop and format a questionnaire applying the following criteria: * The data items will not duplicate existing data or data readily derivable from other sources; * The survey and each data item in it must be justified on the basis of intended use; * Data required to answer the survey must be readily available to the respondent with minimum dependence upon record-searching; * Questionnaires must be relatively short, usually not exceeding three single pages; and * Time to complete the questionnaire for mail surveys must be no more than about 45 minutes per respondent, and should not exceed 15 minutes per respondent for telephone surveys. For surveys of households using CATI/RDD, the contractor shall develop, in addition to the questionnaire of data items, a screener that will be used to identify eligible participants and create the sample of respondents. Questionnaires and screeners for these surveys shall have no page limitations, but response time for both screener and questionnaire will not usually exceed 15 minutes. 5. Prepare IMCD/OMB Clearance Package Congress mandated clearance of all education data collection forms through IMCD of the package for each survey questionnaire for the review and approval of NCES, IMCD, and OMB. The contractor shall draft the justification to be used in coordinating the survey with representative respondent groups and in obtaining clearance and approval from IMCD/OMB. The justification will be developed and written in the prescribed OMB format and shall contain such information as the federal policy purpose of the survey, as it relates to federal laws, bills, initiatives, or priorities, the structure of the collection instrument, a copy of the cover letter to survey respondents, the timeliness and use of the data and survey, the analysis plan for reports, the plan for public-use files, and the schedule and cost estimates of the survey in accordance with detailed specifications. NCES will review the data collection instrument and the justification package. After review by NCES, IMCD, and finally OMB, the contractor shall make revisions as necessary. 6. Discussions with Data Requester and NCES The contractor shall confer with NCES and the data requester at various stages in the development of the questionnaire until NCES concurrence on the final draft of the questionnaire is achieved. 7. Conduct the Pretest The contractor shall conduct a pretest of the survey questionnaire, either by mail/telephone or with a focus group of not more than nine (9) respondents, representative of the group to be surveyed. The contractor shall choose these respondents and submit its choice to NCES for its approval. The respondents must be eligible for inclusion in the survey sample. The purpose of the pretest is to test the instrument, particularly in regard to the clarity of the wording, the availability of the requested data, and the magnitude of the response burden. The contractor shall assure NCES that each respondent has agreed to participate in the pretest. This requirement may lead to the selection of substitute respondents for the pretest. The contractor shall also keep NCES informed about the progress of the pretest and any problems concerning the questionnaire. 8. Discussions of Changes Within two weeks from the completion of the pretest, the contractor shall submit a report on the conduct of the pretest and meet with NCES to discuss the results and evaluation of the pretest. Based upon the pretest results, the contractor shall propose to NCES, for its approval, changes to be made to the questionnaire, cover letter, and justification package. At this meeting the contractor shall discuss and review the data analysis plans. The contractor shall subsequently implement the NCES approved changes. 9. Obtain Government Clearance NCES will obtain the necessary government approvals for the documents. At times, the contractor may be requested to participate in the clearance meeting at IMCD and/or OMB. If any changes to the documents are needed as a result of this meeting, the contractor shall incorporate these changes into the documents. 10. Print questionnaire For self-administered mail questionnaires, the contractor shall prepare the data collection instrument in camera ready copy and print it after IMCD and OMB have cleared it. The contractor shall also type the cover letter on NCES letterhead and send it within one day to NCES staff to obtain the Commissioner's signature. After the Commissioner of NCES has signed the letter, the contractor shall arrange for its delivery back within one day for printing and mailout. For data collection through CATI, the contractor shall develop all CATI screens and train interviewers in the conduct of the survey. Task 6 - Data Collection and Follow-up A. Self-administered mail questionnaires The contractor shall be responsible for data collection and strive to achieve as quickly as possible the goal of a 100% response rate in universe surveys and at least 90% in sample surveys. Based on the response rate and total time for data collection, NCES will decide when to close out data collection (see Key Milestone Elapsed Times). Questionnaires are addressed to agencies, institutions, and individuals. Responses to the surveys are voluntary, and all FRSS questionnaires must contain a statement to that effect. Past survey questions generally have not requested sensitive data; however, a few surveys may contain data that have privacy implications. In all cases, special efforts are required to maintain confidentiality, and procedures and precautions shall be described. 6.1. Mailout of Data Collection Instrument The contractor shall mail the survey materials to the coordinators and prospective respondents in the most efficient way and in accordance with state data collection preferences for FRSS E/S surveys. For PEQIS surveys, the contractor shall mail the survey materials to the institutional coordinators. For universe surveys of SEAs or SHEAs, the contractor shall mail directly to the agency. The notification to prospective respondents of their selection in the sample will be included in the cover letter and shall be sent with mailout of the questionnaire. Their cooperation plays an important part in the data collections. The contractor shall draft a letter that shall contain information about NCES, FRSS or PEQIS, and the survey, the importance of the prospective respondent's participation in the survey, and the expected workload of the prospective respondent. The contractor shall submit this draft for NCES review. After NCES review, the contractor shall make any necessary revisions and shall type the letter and submit it to NCES to obtain the Commissioner's signature. It shall be mailed with the survey and shall be mailed with FRSS E/S surveys according to preferred mail-out procedures in each state. 6.2. Receipt Control The contractor shall develop receipt-control procedures to track responses through follow-up. At the conclusion of thedata-collection phase, the contractor shall submit a copy of the receipt control log to NCES. 6.3. Data Collection Until the desired response rate is achieved, the contractor shall submit a written status report once a week and report immediately any problems that would require special attention by NCES. 6.4 Data Editing and Follow-up The contractor shall edit the responses for inconsistent or incomplete returns according to the edit specifications developed by the contractor and approved by NCES. The edit specifications shall include logic checks, range checks, and other edits deemed appropriate. The contractor shall conduct a follow-up to clarify item responses, as well as follow-up for nonresponses, until a 90% response rate for mail surveys is achieved, or until directed to stop data collection. Follow-up shall be conducted according to the data collection procedures in the state for elementary/secondary education surveys. Telephone follow-up for nonresponses to PEQIS surveys shall be directed toward the institutional coordinators who would then urge the prospective respondents at their institutions to complete and return their questionnaires. Follow-up for any missing data or clarification on PEQIS surveys shall be directed toward the respondents to the survey. Follow-up shall also be directed toward prospective respondents in universe surveys of SEAs and SHEAs. B. Telephone Interviews For surveys conducted using computer-assisted telephone interviews, the data shall be entered directly into a computer file through a CATI system. Although most data edits shall occur on-line as the interviewers enter responses, additional data editing and coding may be necessary and shall be performed at the conclusion of the data collection period. These editing and coding activities shall involve both computer-assisted and manual activities. The contractor shall draft for NCES review and approval, a plan outlining the post-CATI edit checks. The contractor shall strive to achieve as quickly as possible the goal of 85% response rate for the screener, which identifies eligible respondents and creates the sample; and also, an 85% response rate for the questionnaire data items. The overall response rate is expected to be at least 72% for surveys conducted by use of random digit dialing. Based on the response rate and total time for data collection by CATI, NCES will decide when to close out data collection. Task 7 - Data Processing and Analysis The contractor shall provide facilities and establish arrangements for the data processing requirements of this contract. The contractor shall assure that all computer programs for this project are coded, well documented, and debugged by the time they are needed. 7.1. Weighting and Imputation The contractor shall develop, submit to NCES, and implement with NCES approval a procedure to compensate for unit nonresponse, shall determine the weights needed to produce valid national estimates, and shall apply these weights to the unweighted edited data. Weighted response rates of 90% or less may require nonresponse adjustments at subdomain levels. While it has not been necessary to implement imputation procedures for item specific nonresponses in past surveys, the contractor shall be prepared for such an eventuality in the future. 7.2. Data Preparation and Data Files The contractor shall accurately encode the survey responses to machine-readable format and develop edited data files. These files, containing survey data, other auxiliary information needed for cross-tabulations, and the final weights adjusted for nonresponse, shall be structured by the contractor so that other analysis can be done by NCES, the data requester, and the general public. After weights have been calculated, the contractor shall deliver to NCES the survey data files including weights and any auxiliary variables. The contractor shall also prepare public-use data files, accompanied by documentation, and files containing the final survey report, in PDF and HTML format, for NCES to place on its Web site. The file preparation for public release shall adhere to all confidentiality requirements of NCES. The documentation shall be developed in accordance with NCES standards on data files and documentation and shall include a User's Manual that provides sufficient detail to enable any user to fully understand the files. It should include: * A description of the sample * A description of the data collection procedures * A description of the response rates (unit and item) * A discussion and evaluation of the design effects of the sample * Standard errors for each survey item * A codebook that contains the unweighted and weighted frequencies 7.3. Tabulations Based on the analysis plan and joint NCES/contractor/data requester review of data, the contractor shall produce tabulations of edited unweighted and weighted responses and of weighted cross-tabulations based upon survey items. The statistics may be totals, averages, or ratios based on the universe, strata, or subdomains. A meeting with NCES to discuss the tabulations and the results of the analysis may be necessary. 7.4. Variance Estimates The contractor shall compute variance estimates to guide and support the analysis and presentation of data for the survey report. Initially, variance estimates for major statistics based on the questionnaire shall be calculated and delivered to NCES to coincide with the delivery of first tabulations. Subsequent variance estimates, to guide the selection of data for and comparisons made in the text of the survey report, shall be provided as necessary (3 weeks after the first draft of the survey report and again to coincide with the final drafts of the survey reports). In general, any table in the final report must have a corresponding table of variance estimates. The contractor must receive the approval of NCES COTR for the program used to calculate the variance estimates. 7.5. Analysis of Data The objective of analyzing the survey data is to find important (from a subject matter point of view) and statistically significant relationships among the survey items and available variables. In addition to plots and cross-tabulations with associated variances, the contractor shall explore, with NCES approval, other multivariate analyses (including regression analysis, factor analysis, log-linear analysis, discriminate analysis, principal component analysis, cluster analysis and any scaling, making transformations, and other procedures necessary to prepare the data for analysis) that might clarify relationships and increase the usefulness of FRSS reports. The contractor shall conduct analyses, produce tables, and identify important findings and highlights to be presented in reports for dissemination. All analyses conducted for NCES shall meet the standards in the NCES Statistical Standards manual. Task 8 - Report Preparation and Dissemination The contractor shall prepare and submit a survey report and a technical report for each FRSS and PEQIS survey, and, in addition, a "peer" report for each PEQIS. On rare occasions, NCES may request a short supplementary survey report. During the third option year of the contract, if the Government has exercised the option, the contractor shall also submit an evaluation report for the PEQIS institutional panel, which shall be used to help guide panel reselection. For the survey report, the contractor shall prepare and submit to the NCES COTR a draft of the report outline one week after preliminary tabulations are produced. NCES will review and comment on the proposed outline and the contractor shall revise it according to NCES specifications. The survey report requires the services of an experienced writer to attain the style, economy, and clarity of presentation required by NCES. Conclusions made in the survey reports must be based on the data and must be statistically defensible. All survey reports must be adjudicated by NCES before public release. 8.1 Survey Report The contractor shall prepare and submit three (3) drafts of the survey report for NCES review. The report shall conform in style to the types of publications NCES uses to disseminate survey data and shall adhere to the procedures outlined in OERI Publications Guides. Survey reports usually consist of highlights, a foreword or introduction, background information providing the policy relevance of the survey, text giving an overview of the results, several significant findings, figures (graphs and charts) and text tables, and a technical appendix that includes summary and variance tables. The survey report is usually about 75 pages long, including text and tables. After adjudication of the final draft, the contractor shall make all the necessary revisions and prepare the report in camera-ready form for printing in accordance with current NCES/Government Printing Office (GPO) standards. Upon approval by NCES, the GPO will print the required number of copies of the report, and the contractor shall disseminate it to the survey respondents, and state and institutional coordinators, as appropriate. In some instances the data requester may be the most knowledgeable individual on a given topic and may wish to write the survey report or to work with the contractor in writing the report. In such instances, the contractor shall facilitate this arrangement in any way possible by providing data files, tabulations, statistical advice, etc. 8.2 Technical Report of Survey Operations After each survey is completed, the contractor shall prepare and submit to NCES a report documenting the conduct of the survey. The report shall include a description of the survey operations, any operational problems, their resolutions and recommendations, completion dates and cost figures. The purpose of this report, intended for NCES use only, is to provide a complete documentation of the activities, timing, and cost of survey operations. The preparation of this report must not delay the survey report, which has higher reporting priority. 8.3. Peer Report For each PEQIS survey, the contractor shall prepare a one-page "peer" report for each institution or agency that responded to the survey. The report will provide a comparison of an institution's responses with that of its peers on selected survey items. The format of the report will be a straightforward table that contains the following information: if a selected survey item is a multiple choice (categorical) item, the respondent's choice as well as the unweighted percentage distribution of its peers will be indicated; if the questionnaire item called for a numerical response, then the respondent's answer will be presented with the unweighted arithmetic mean of its peers. For purposes of this report, "peers" will be defined as those institutions which are in the same stratification cell and in the same geographic region as the responding institution. If a responding institution has less than 5 peers in the same geographic region, then its responses will be compared with its peers (institutions in the same stratification cell) nationwide. The items selected and the detailed format for the peer report will be specified by the NCES COTR. Task 9 - PEQIS Panel Evaluation In Year 3 of the contract, the contractor shall evaluate the PEQIS panel and submit a report that details the strengths and weaknesses of the current panel of institutions in its capacity to address various issues and to represent the various sectors of postsecondary education. If the report cites any sectors of the panel as having deficiencies, the report shall outline various methods to supplement/expand/refresh the current panel through the addition/replacement of some institutions. Upon approval of the report by NCES, the contractor shall add/replace those institutions. Task 10 - System Maintenance The contractor shall maintain current lists and machine-readable files of State Coordinators for the elementary/secondary education surveys. The contractor shall also maintain a current list of the Panel of Institutional Coordinators for PEQIS surveys. The contractor shall use feedback from mailouts, NCES, coordinators, and institution presidents to update these files. The contractor shall provide to the NCES COTR, the updated lists upon request. The preferred methods of data collection for FRSS E/S surveys in each state shall also require periodic updating. The contractor shall obtain these preferences, at the direction of NCES, in timely fashion and with consideration for minimizing response burden. The contractor shall maintain complete data files for all surveys conducted through the FRSS and the PEQIS. Maintenance of data files means keeping coordinator and respondent files, sample files, and program files (along with documentation available for reproduction), so that copies can be supplied to NCES at the end of each survey. Task 11 - Project Management Activities Since the timing of survey requests and, to a lesser extent, certain facets of conducting individual surveys are somewhat unpredictable in the short run. Since numerous FRSS and PEQIS surveys are in various stages of completion at any one time, the effective management of these concurrent tasks is imperative. The contractor shall, upon the request of the NCES COTR, furnish a schedule for the completion of all active surveys. This schedule is subject to NCES approval. NCES will provide any necessary revisions within 1-2 weeks. These schedules shall be based upon NCES/data requester discussions of the priority and the timing of various surveys and tasks within surveys. IV. Summary of Deliverables The contractor shall be responsible for delivering the following items: A. Monthly Progress Reports In order to document the progress on current tasks, the contractor shall provide monthly progress reports (1 copy to the Contracting Officer and 1 copies to the NCES COTR). These reports shall document completed activities, tasks planned for the following month, products remaining to be delivered on planned or on-going activities, difficulties encountered, and a manpower breakdown by category of personnel for each survey. B. To NCES. for each survey : 1. Drafts of the questionnaire and a schedule of milestones for the survey in question. Due 3 weeks after the first meeting with NCES and the data requester (3 copies of each draft and 10 copies of final instrument prior to mail-out). 2. Clearance packages for OMB/IMCD approval - 1 week after the pretest (10 copies of final draft). 3. Final questionnaire and coordinator and transmittal letters - 1 week after OMB approval. (10 copies) 4. Weighted and unweighted tabulations, data plots, variances, and other analyses - 4 weeks after data retrieval (2 copies). 5. First draft of survey report of findings - 10 weeks after the submission of tabulations (3 copies of each draft); Adjudication draft - 2 weeks after NCES approval of third draft (20 copies) Camera-ready copies of the final report - 1 week after approval by the Office of the Commissioner (2 camera-ready copies). 6. Technical report on survey operations - 2 copies of final report. 7. Survey data public-use files and documentation and NCES internal-use files and documentation - 2 copies. 8. PEQIS peer reports (1 copy mailed with survey report to each responding institution) 9. Survey report and data files for NCES's Web site (one copy of each) - submitted with camera-ready copy of final report. 10. In Year 3: Report on the evaluation of the PEQIS Panel - 1 copy. C. To NCES, for each selected sample: 1. Description of proposed sample design - 4 weeks after NCES request (2 copies). 2. Listing of the sample as selected, if appropriate, (1 copy) and description of sample characteristics - 6 weeks after NCES request (2 copies). 3. Notification letters to coordinators and district superintendents, as required - 7 weeks after NCES request (3 copies of each draft and final) . 4. Report of the operations and problems in selecting the sample - 8 weeks after NCES request (2 copies). V. Other Contract Requirements A. Compliance with NCES Standards All work conducted for NCES by the contractor shall adhere to the NCES Statistical Standards, available on the Internet at: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=92021XXXXX B. Protection of Data Under no circumstances may the contractor release any survey information unless the NCES COTR approves such releases. Names of respondents and background information necessary to reach them for follow-up shall be maintained separately from other respondent data. Retrieval of data by personal and institutional identifiers shall require the creation of physically separate tape files, which may be used only by NCES and approved contractor personnel. Individual identifiers shall be associated with data only for purposes of data gathering, matching new data with old, establishing sample composition, authenticating data collections, or obtaining missing data. The contractor's QRIS staff shall sign affidavits of nondisclosure and shall be responsible for maintaining the strictest confidentiality for all data collected via these surveys. The contractor shall adhere to all the confidentiality requirements for NCES contractors that collect individually identifiable information. C. Confidentiality Requirements Public Law 100-297 requires that the rights of individuals responding to NCES surveys are protected and that confidentiality of responses is maintained. The contractor shall follow NCES Standard IV-0l-92, "Standard for Maintaining Confidentiality." D. NCES Review and Approval Throughout this contract the NCES COTR will review contractor procedures and products. NCES recognizes the importance of providing a timely review of these products. In the description of the contract's tasks, time periods for review by NCES have been indicated. These time periods reflect the expected time it will take for the NCES COTR to review the product, obtain initial comments from other NCES staff, and provide the contractor with comments over the phone or in writing.