Laws & Guidance GENERAL
Information Quality Guidelines

Overview | Guidelines | Information Correction Requests and Appeals | Contact

Guidelines

You are here: Guidelines > Information Quality > Objectivity > Statistical Data

Department of Education reports and data collections that draw upon sample survey data should be clearly written, and should follow these Guidelines:

  1. In formulating a data collection plan, the goals of the study should be clearly described;
  2. The subjects to be studied and the data to be collected should be clearly defined, using broadly understood concepts and definitions that are consistent with Department data definition handbooks;
  3. The research study approach or data collection techniques should be well thought out and designed to use state-of-the-art methodologies in the data collection and should also be clearly described in the study documentation;
  4. In designing the work, every effort should be made to minimize the amount of time required for study participants;
  5. The source of data should be reliable. The sample should be drawn from a complete list of items to be tested or evaluated, and the appropriate respondents should be identified, correctly sampled, and queried with survey instruments that have been properly developed and tested;
  6. Response rates should be monitored during data collection. When necessary, appropriate steps should be taken to ensure that the respondents are a representative sample;
  7. Care should be taken to ensure the confidentiality of personally identifiable data, as required by law, during research/data collection, processing, and analysis of the resulting data;
  8. Upon completion of the work, the data should be processed in a manner sufficient to ensure that the data are cleaned and edited to help ensure that the data are accurate and reliable;
  9. The findings and data collection should be properly documented and stored, and the documentation should include an evaluation of the quality of the data with a description of any limitations of the data. In particular, any known limitations of the information should be documented (e.g., missing values, amount of nonresponse);
  10. Data should be capable of being reproduced or replicated based on information included in the documentation, such as:
    1. The source(s) of the information;
    2. The date the information was current;
    3. Any known limitations on the information;
    4. The reason that the information is provided;
    5. Descriptions of any statistical techniques or mathematical operations applied to source data; and
    6. Identification of other sources of potentially corroborating or conflicting information.
  11. If secondary analysis of data is employed, the source should be acknowledged, the reliability of the data should be confirmed and documented, and any shortcomings or explicit errors should be acknowledged (e.g., the representativeness of the data, measurement error, data preparation error, processing error, sampling errors, and nonresponse errors);
  12. The analysis should be selected and implemented to ensure that the data are correctly analyzed using modern statistical techniques suitable for hypothesis testing. Techniques may vary from simple tabulations and descriptive analysis to multivariate analysis of complex interrelationships. Care should be taken to ensure that the techniques are appropriate for the data and the questions under inquiry;
  13. Reports should include the reason the information is provided, its potential uses, and cautions as to inappropriate extractions or conclusions, and the identification of other sources of potentially corroborating or conflicting information;
  14. Descriptions of the data and all analytical work should be reported in sufficient detail to ensure that the findings could be reproduced using the same data and methods of analysis; this includes the preservation of the data set used to produce the work;
  15. Prior to dissemination all reports, data, and documentation should undergo editorial and technical review to ensure accuracy and clarity. Qualified technical staff and peers both inside and outside the Department should do the technical review;
  16. All work should be conducted and released in a timely manner; and
  17. There should be established procedures to correct any identified errors. These procedures may include the publication of errata sheets, revised publications, or Web postings.
Previous Table of Contents Next

 
Print this page Printable view Send this page Share this page
Last Modified: 10/17/2005