Statement of Work

Early Childhood Longitudinal Study: Kindergarten Class of 1998-1999

(ECLS)

Data Development and Longitudinal Studies Group

I. INTRODUCTION	1

A. Purpose and Nature of the Procurement	1

B. Background	2

B.1 Enabling Legislation	2

B.2 The Need for National Data on Children's Early Education
2

B.3 NCES/ED Responsibilities and Initiatives	3

C. ECLS Design Activities	3

D. Key Study Issues and Questions	4

II. KINDERGARTEN COHORT STUDY DESIGN	6

A. Overview	6

A.1 Target Population	6

A.2 Kindergarten Cohort Sample	7

A.3 Survey Components	7

A.5 Data Collection Schedule	9

A.6 Measurement of Growth in Cognitive Dimensions	11

A.7 Measurement of socioemotional development	18

A.8 Physical and Psychomotor Assessment	19

B. Sample Design	19

B.1 Kindergarten sampling frame	20

B.2 Sample attrition and mobility	20

B.3 Oversampling of minority students	21

B.4 Freshening of sample	21

B.5 School-level estimates for later waves	22

C. Methodological Inquiries and Data Quality	23

D. Optional Special Studies and Supplements	23

D.1 Optional Special Studies	24

D.2 Optional Supplements	27

E. NCES/ED Standards	31

III. SCOPE OF WORK	32

A. Overview of Tasks	32

B. Core Tasks to Be Performed	32

Task 1. Project initiation and planning	32

Task 2. Technical review panel	33

Task 3. IMT/OMB clearance of main survey instruments	36

Task 4. Select school sample, notify sampled schools and secure cooperation
36

Task 5. Select student and teacher samples and secure cooperation	38

Task 6. Receipt control and reporting	39

Task 7. Tracking of kindergarten sample	40

Task 8. Conduct pilot test	41

Task 9. Reproduction of instruments and acquisition of materials	41

Task 10. Hiring and training data collection staff	44

Task 11. Collect kindergarten and first grade data	47

Task 12. Data processing	49

Task 13. Briefing materials.	52

Task 14. Methodology report	52

C. Optional DesignTasks to be Performed	55

Optional Task A. Develop instruments and procedures for the second through fifth grades	55

Optional Task B. Extend the EC LS Assessment Battery	57

Optional Task C. Design third through sixth followup samples	60

D.1. Optional special studies	63

D. 2 Optional Supplements	66

IV. STUDY REQUIREMENTS	68

A. Reporting	68

B. Schedule of Milestones and Deliverables	68

C. NCES/ED Review and Approval	69

D. Quality Control	69

V. PROPOSAL REQUIREMENTS	70

A. Use of Subcontractors	70

B. Period of Performance	70

C. Documents Available for Inspection	70

I. INTRODUCTION

A. Purpose and Nature of the Procurement

The National Center for Education Statistics (NCES) of the United States Department of Education (ED) requires a contract to conduct the base year and first followup of the Early Childhood Longitudinal Study (ECLS): Kindergarten Class of 1998-99. Two data collections occur during the base year of the ECLS, fall kindergarten and spring kindergarten. First followup data is scheduled to be collected in the spring of first grade.

The period of performance of the contract shall be 48 months. In addition to a set of 14 core tasks, the procurement contains 1) several optional tasks, 2) special studies and 3) supplements for additional tasks and collections within the 48 month period of performance.

The Early Childhood Longitudinal Study (ECLS) responds to the immediate and continuing need for high quality and timely data pertaining to the condition of the early education of the nation's children. The ECLS has two purposes: 1) descriptive and 2) analytic. On the one hand, it will use nationally representative samples of schools and children to describe (1) children's status at entry into school, and the characteristics of their schools and classrooms; (2) children's transition into school; and (3) their experiences and learning from kindergarten through grade 5. On the other hand, the ECLS will provide a rich data set that will enable researchers to study how a wide range of family, school, community, and individual variables affect early success in school.

For the first time, national data will be available on both public and private kindergarten programs and the children who attend them. A goal of the kindergarten data collection is to describe accurately the diversity of kindergarten children and the programs they attend. Such information is critical to establishing policies that can respond sensitively and creatively to this diversity.

Data collected during the kindergarten year will also serve as baseline data to examine how schooling shapes later individual development. The longitudinal nature of the study will enable researchers to study children's growth in critical developmental domains and to relate trajectories of growth and change to variations in children's school experiences in kindergarten and the early grades.

A contract to design the ECLS was awarded in Spring 1994. All design work that occurred since then, including the conduct of a large-scale field test (feasibility study) in school year 1996-97, constituted Phase I of the ECLS. Phase II of the ECLS begins with the award of this contract to conduct the base-year study and first followup study.

This procurement involves 1) finalizing the ECLS sample design building on the work that was done during Phase I, 2) selecting the ECLS sample of schools and kindergartners and 3) preparing for and implementing the ECLS base year and first followup data collections using the instruments and procedures that were developed during Phase I. It also includes an option to develop the instruments and procedures that will be used in later follow-ups of the ECLS, i.e., second through fifth grades and several additional options for enhancing the design of the ECLS through sample augmentations and supplements.

B. Background

B.1 Enabling Legislation

The ECLS is being undertaken in compliance with the mandate stated in section 404 of the National Education Statistics Act of 1994 (20 U.S.C. 9003):

"The duties of the Center are to collect, and analyze and disseminate statistics and other information related to education in the United States and in other nations, including ...conducting longitudinal studies, as well as regular and special surveys and data collections, necessary to report on the condition and progress of education..."

B.2 The Need for National Data on Children's Early Education

The critical importance of high-quality, equitable education for the continued development and prosperity of the country has emerged as a very high priority on the national agenda. Parents, educators, and policy makers are reconsidering the ways children are taught in schools and are looking for more effective approaches to education.

Much of the public discussion on education is focused on the early years, children's development and learning during the early childhood period from birth to age eight, their preparation for formal school, the first school experience, and the progress they make over the first years of school.1 A number of factors have contributed to the research and policy focus on children's early school experiences. These include: 1) the increased public awareness of the importance of children's early experiences, reinforced by the National Education Goal on school readiness; 2) the changing nature of children's preschool and early school experiences; 3) the increasingly diverse population of children entering school and the demands this places on schools; and 4) the expanded role that schools are expected to play in supporting and nurturing development and learning.

Vital to efforts to improve schools and the education of all the Nation's children is a research and data collection program that increases our understanding of the dynamics of school achievement, particularly of those factors that lead to the differential success of important groups of children during the early school years. The Early Childhood Longitudinal Study (ECLS), in which a cohort of entering kindergarten children will be sampled and studied intensively over time, responds directly to the concerns decision makers, educational practitioners, researchers, and parents have about our Nation's schools and American education.

B.3 NCES/ED Responsibilities and Initiatives

NCES has as its mission the collection and dissemination of data on the condition of education in the United States. It collects statistics on the status and progress of education in the Nation, analyzes and reports the meaning and significance of those statistics, and assists States and local education agencies in improving their statistical systems.

Prior to the National Household Education Survey (NHES), NCES collected little information on children below the eighth grade level, other than basic enrollment data and assessment data from NAEP and TIMSS. At present, there is no national data base that permits a thorough study of children's transition into school and their progress through elementary school. Such a study requires detailed data on children's experiences at school entry as well as data on their school environments and programs. It requires that all segments of the student population be represented, in order for NCES to examine the school experience for different segments of the population.

C. ECLS Design Activities

Design work on the ECLS began in 1989 and has involved many individuals and organizations. More intensive work on the design of the ECLS began in 1994. In April, 1994, ED/ NCES awarded a contract to the National Opinion Research Center to 1) develop the overall design for the ECLS including the sample design and data collection methodologies and procedures; 2) develop the instruments and procedures that would be used in the base-year study (kindergarten fall and spring) and the first two followup studies (spring first grade and spring second grade); 3) develop an assessment battery to measure the status of children's development in important cognitive and noncognitive domains at kindergarten entry and to measure growth in these domains as children move through the elementary school years; 4) design and conduct a large-scale field test or feasibility study of the ECLS base-year and first followup designs; and 5) finalize the designs of the base year and first followup studies based on the field test experience.

D. Key Study Issues and Questions

In general, the ECLS will focus on children's 1) transition to school, 2) schooling and performance in the early grades, and 3) the interaction of school, family, and community. While these general areas encompass a vast array of issues and research questions, certain issues have been identified as more important for the study. The four key issues to be addressed by the ECLS are 1) school readiness; 2) children's transitions to kindergarten, first grade, and beyond; 3) the relationship between children's kindergarten experience and their elementary school performance; and 4) children's cognitive growth and progress through elementary school. Each of these key issues is described within the context of the three general areas in the following sections.

School Readiness. Some children seem to adjust to their first encounter with formal school and are able to have a positive school experience while other children have problems making this adjustment. For most children, the first formal school experience is kindergarten. The nature of this kindergarten experience is quite variable. While 98 percent of children nationwide attend a kindergarten program prior to first grade, 40 percent of children attend full-day programs and 58 percent part-day programs. Attendance in public school kindergarten programs is about six times that of private school programs. Within these program types, curricula and instructional practices are expected to vary widely.

The ECLS has adopted a conceptual model of school readiness that is consistent with the one set forth by the National Education Goals Panel, Goal One Technical Planning Group.2 Thus, school readiness is conceived as being multidimensional and continuous. Children may have more or less of each of the skills, experience, and knowledge that increase their chances of having a positive school experience. Children who are relatively weak in one area may or may not be weak in others.

The ECLS will describe children's school readiness on a number of different dimensions and how each is related to different characteristics of children, their families, communities, and kindergarten programs/classes. The study is interested in the role that parents/family play in helping prepare children for school and in adjusting to the school environment. In order to understand children's preparation for school, it is critical to understand how schools prepare for and respond to children who enter their doors for the first time. The study will pay particular attention to how schools and kindergarten programs respond to the backgrounds and experiences that children bring with them. Data will be collected on a range of characteristics of children's schools, kindergarten programs, and classrooms.

Transitions to Kindergarten, First Grade, and Beyond. Children and adults continually make transitions from one status to another. Of particular interest to the ECLS, is the transition that occurs as young children go from preschool to kindergarten, and from kindergarten to first grade.

Kindergarten and Elementary School Student Performance. A central issue for the ECLS is the relationship between children's kindergarten experience and their school performance in grades 1 through 5. A goal of the program is to provide descriptive information on children as they move out of kindergarten, into first grade and through the elementary school grades. The study will focus on student achievement during these years as well as other measures of student performance. By closely monitoring student performance, the study will provide useful information on when children begin to experience problems with their school work and the circumstances surrounding these difficulties. The study will provide useful information on the longevity of problems and the response of the child's family, school, and teacher to problems.

Cognitive Growth and Student Progress. The ECLS is interested in students' performance in the primary (grades 1-3) and elementary grades (grades 4 and 5). A major goal of the study is to monitor student achievement and academic progress during children's early school careers. The ECLS seeks to understand better children's cognitive, as well as social development during these early school years.

II. KINDERGARTEN COHORT STUDY DESIGN

A. Overview

The ECLS is intended to provide information that will serve to inform educational policy. The data collected through this study and the information that is disseminated through reports prepared by NCES/ED and others based on these data will inform decision makers, educational practitioners, researchers, and parents about children's earliest school experiences and how the Nation's schools affect the lives of the children who pass through them.

While most of the components of the design have been developed during Phase I, they cannot be finalized until the 1996-97 field test is complete. Key elements of the ECLS design are outlined in the following sections.

A detailed account of the ECLS design can be found in the ECLS Study Design Report.3 This document includes: 1) the ECLS sampling plan, 2) a description of the data collection procedures and methods, 3) a plan for monitoring and evaluating the quality of the data, 4) descriptions of the contextual questionnaires, and 5) the framework for the ECLS assessment battery and the rationale for the assessment approach being used in the study. The design outlined in this document is the one that is being field tested in school year 1996- 97.

A.1	Target Population

The ECLS is concerned with the Nation's children. It is intended to focus on the educational experiences of minority and majority, disadvantaged and advantaged children alike. Thus, the target population of the ECLS is all children enrolled in U.S. public and private kindergartens.

There are many types of kindergarten programs (e.g., public school and nonprofit private school programs, as well as for-profit programs). Some programs are full-day and others are part-day. Most operate five days a week while some operate for fewer than five days. Most programs are designed as one year programs, while others are designed as two- year programs. All of these characteristics are being considered when defining the population of both kindergarten programs and kindergartners.

NCES/ED intends to be as inclusive as possible when defining kindergarten programs and including kindergarten children in the ECLS. Also, it is NCES/ED's goal to limit the numbers of children who are excluded from the ECLS as a consequence of their having special needs that require specialized data collection instrumentation and procedures. NCES/ED's desire to collect data on children from multiple sources (e.g., parents, teachers, and the children themselves) should reduce the number of children who are excluded from participation. NCES/ED has been working with the U.S. Department of Education's Office of Special Education Programs to find ways to increase the participation of children with disabilities in the ECLS, and NCES/ED is continuing to develop an approach for more effectively including children with limited proficiency in English in the ECLS assessment.

A.2	Kindergarten Cohort Sample

A nationally representative sample of approximately 23,000 children enrolled in about 1,000 kindergarten programs during the 1998-99 school year will be selected for participation in the study. These children will be selected from both public and private kindergartens, offering full-day and part-day programs. The sample will consist of children from different racial-ethnic and socioeconomic backgrounds. Presently, NCES plans to oversample private kindergartens and kindergartners. It also plans to oversample Asian and Pacific Island children, and if necessary Hispanic children. It continues to explore the feasibility of oversampling American Indian children and children with disabilities.

A.3	Survey Components

In order to address all of the key research issues, it is necessary to examine the whole of children's environments and development. The emphasis that the ECLS is placing on the whole child has critical implications for the design of the ECLS. The study will collect data about the child, the child's school, classroom, teacher, and family. These data will be collected directly from the child, the child's parents/guardians, teachers, and school administrators.

Children will be asked to participate in various activities in order to measure the extent to which they exhibit those abilities and skills deemed important to success in school. They shall be asked to participate in activities designed to measure important cognitive (e.g., general knowledge, language and quantitative skills) and noncognitive (e.g., social skills) competencies. At least through the end of first grade, direct measures of a child's cognitive skills will be obtained through a one-on-one assessment of the child. During later years of the ECLS, children's participation will be broadened to include self-reports of their school and out of school experiences.

During the kindergarten year, children will twice participate in a one-on-one assessment. In the fall of the kindergarten year, children will be administered a 30-45 minute cognitive assessment with three subtests (mathematics, language/reading, and general information). During the fall, children will also participate in a short psychomotor assessment, and possibly a short direct assessment of their social skills. In the spring of kindergarten, children will be administered a 45-50 minute cognitive assessment with the same subtests, and possibly the same direct social skills assessment. The first grade cognitive assessment will be 45-50 minutes in length and will cover the same domains. A direct measure of children's social skills may be used in first grade. No assessment of children's psychomotor development will occur after the fall of kindergarten. All of these assessments are being developed and tested during Phase I.

Parents/guardians are an important source of information about the families of the children selected for the study and about themselves. Due to the young age of the children that will be participating in the ECLS, parents/guardians are needed to act as proxy respondents for their children during the early years of the study. Furthermore, they are an important source of information on themselves and on their children's home environment, development at school entry, and neighborhood.

Because the ECLS will rely on parents for so much data, interviews with parents or guardians will be conducted at each wave of data collection—i.e., in the fall and spring of the kindergarten year and in the spring of first grade. The interview shall be conducted via telephone using CATI technology and shall be limited to 45 minutes on average. In those cases where a telephone interview is not possible, a personal interview shall be conducted using CAPI technology. The parent/guardian to be interviewed shall be the individual who is the most knowledgeable about the sampled child's care and education. In many cases, this individual will be the child's mother.

Parent/guardian interviews for fall and spring kindergarten and for spring of first grade were developed during Phase I and are being field tested in 1996-97.

Teachers, like parents, represent a valuable source of information on themselves, the children in their classrooms, and the children's learning environment, i.e., the classroom. No other respondent group is as burdened as teachers by a study such as the ECLS. Teachers of sampled children will not only be asked to provide information about their own backgrounds, teaching practices, and experience, they will be called upon to provide information on the classroom setting for the sampled children they teach during the fall and spring of the kindergarten year and in the spring thereafter.

Teachers will be asked to provide information across a range of important themes in a self- administered questionnaire. It is estimated that teachers will spend between 30-45 minutes completing this questionnaire.

Teachers will also be asked to evaluate each sampled child on a number of critical cognitive and non-cognitive domains. The information teachers provide about a sampled child's social skills and behaviors, physical activity, and performance in the curricular area domains included in the cognitive assessments broadens the range of information about the child. The three instruments developed for rating each child include: 1) a teacher checklist reporting on such things as the children's language skills, special needs, program placements, attendance, and physical activity, 2) a cognitive rating scale, and 3) a social skills rating scale.

While teachers of sampled children will not be remunerated for completing the teacher questionnaire, they will be paid for completing the three rating instruments for their pupils. Without remuneration for their role as data collectors, ECLS teachers would be subject to unusually high levels of uncompensated burden. On average, teachers will spend over 2 hours completing the three rating instruments for the sampled pupils in their class. This estimate is based on the assumption that the set of forms for each child will take 12-15 minutes to complete and that teachers will average 10 ECLS sampled students in their class.

Kindergarten (fall and spring) and first grade (spring) teacher questionnaires and rating forms were developed during Phase I and are being field tested in 1996-97.

School administrators, directors, principals, and headmasters are an important source of information on the schools that children attend. While classroom experiences may be the leading school-related factors of interest in the ECLS, school physical and organizational characteristics, school policies, and other school-level characteristics can shape, promote, or inhibit what goes on in children's classrooms.

Data will be collected from school administrators/directors annually via a self- administered questionnaire. This questionnaire will be distributed during the time of the spring assessments and will take 45 minutes on average to complete.

The kindergarten and first grade school administrator questionnaire was developed during Phase I and is being field tested in the spring of 1997.

A.5	Data Collection Schedule

Exhibit 1 shows the ECLS implementation schedule. The contractor shall to begin full-scale data collection during the fall of school year 1998-99.

The contractor shall collect data twice during the base year of the survey--once at the beginning and once near the end of the school year. The beginning of the year data collection is needed in order to get baseline information on children prior to their exposure to the influences of the school. It will provide measures of the skills, attributes, and knowledge of children as they enter school for the first time.4 The data collected at the end of the school year, together with beginning school year data, will be used to examine children's early schooling experiences and changes during the kindergarten year. Follow-up surveys will be conducted annually. These will be used to study children's growth in important cognitive domains during the critical early school years.

The contractor's data collection schedule shall take into account differences in school calendars. For schools on a traditional nine-month calendar, the goal is to collect direct child assessment data for fall kindergartners over a six week period about 6 weeks after the start of the school year and spring data should be collected over a similar period about 6 weeks before the end of the school year. Spring first grade data should be collected on the same schedule as spring kindergarten. Fall kindergarten teacher ratings of children's cognitive and social skills should be collected about 8 weeks after the start of school and spring kindergarten and first grade teacher ratings should be collected about 8 weeks before the end of school. The schools on non-traditional calendars, the goal is to collect data on schedules that are comparable to those using nine-month calendars.

Fall 1998 data collection shall consist of 1) a 45 minute direct assessment with each sampled child; 2) a 30-45 minute self-administered teacher questionnaire; 3) teacher ratings of each sampled child's cognitive and social skills, together taking approximately 10 minutes to complete; 4) a teacher checklist for each sampled child taking approximately 2-5 minutes to complete; and 5) a 45 minute CATI/CAPI parent interview. For schools operating on a traditional nine month calendar, all direct child assessment data shall be collected within a six week period. The collection of teacher questionnaire data in these schools, along with teacher's ratings of children's cognitive development and social skills, shall be collected within a four week period. Parent interviews shall begin no later than two weeks after the first direct child assessment and shall be completed within two weeks of the last direct child assessment.

Spring 1999 data collection shall consist of 1) a 45-60 minute direct assessment with each sampled child; 2) a 30-45 minute self-administered teacher questionnaire; 3) teacher ratings of each sampled child's cognitive and social skills, together taking approximately 10 minutes to complete; 4) a teacher checklist for each sampled child taking approximately 2-5 minutes to complete; 5) a 45 minute self-administered school administrator questionnaire; 6) a 45 minute CATI/CAPI parent interview; and 7) a school record review and retrieval. All direct child assessment data shall be collected within a six week period. The collection of teacher questionnaire data, including teacher's ratings of children's cognitive development and social skills, and the teacher checklist, shall be completed 8 weeks before the end of school. All teacher questionnaire data shall be collected within a four week period. The collection of school administrator questionnaire data shall be completed within a four week period. Parent interviews shall begin no later than two weeks after the first direct child assessment and shall be completed within two weeks of the last direct child assessment. The extraction of data from children's school records shall occur as late in the school year as possible to insure that the most complete academic year data are available as possible. The collection of school record data shall be completed within a four week period.

Spring 2000 data collection shall consist of 1) a 45-60 minute direct assessment with each sampled child; 2) a 30-45 minute self-administered teacher questionnaire; 3) teacher ratings of each sampled child's cognitive and social skills, together taking approximately 10 minutes to complete; 4) the teacher checklist for each sampled child taking about 3-5 minutes to complete per child; 5) a 45 minute self-administered school administrator questionnaire; 6) a 45-60 minute CATI/CAPI parent interview; and 6) a school record review and retrieval. All direct child assessment data shall be collected within a six week period. The collection of teacher questionnaire data, including teacher's ratings of children's cognitive development and social skills and the teacher checklist, shall be completed within a four week period. The collection of school administrator questionnaire data shall be completed within a four week. Parent interviews shall begin no later than two weeks after the first direct child assessment and shall be completed within two weeks of the last direct child assessment. The extraction of data from children's school records shall occur as late in the school year as possible to insure that the most complete academic year data are available as possible. The collection of school record data shall be completed within a four week period.

A.6	Measurement of Growth in Cognitive Dimensions

A.6.1 Purpose and areas of interest

Measuring children's growth in critical cognitive dimensions is an essential element of the ECLS. Like the assessment batteries used in other NCES/ED longitudinal studies, the primary purposes of the ECLS cognitive assessment battery are to obtain accurate measures of the status of children at a given point in time and their growth over time and to relate this growth to a broad set of school, family, and community variables.

The knowledge and skills assessed by the ECLS cognitive battery represent typical educational goals of the elementary school curricula. The ECLS cognitive battery includes three cognitive domain subtests that measure children's receptive, expressive, and nonverbal reasoning abilities in language/literacy, mathematics, and general knowledge (i.e., science and social studies). The focus on core academic subjects (i.e., reading, mathematics, and general knowledge) reflects the interest in precursors of children's later educational career outcomes and the impracticality of adequately measuring kindergartners' and first graders' proficiencies in writing, art, and music within the constraints of the study.

Exhibit 1.—ECLS Data Collection Schedule, by time of school year

Survey Activity	Phase I	Phase II
F '96	S '97	F '97	S '98	F '98	S '99	F '99	S '00	F '00	S '01	F '01	S '02	F '02	S '03	F '04	S '04
Field Test
Kindergarten
First Grade
Second Grade
Third Grade
Fourth Grade
Fifth Grade
Full-Scale Data Collection
Kindergarten
First Grade
Second Grade
Third Grade
Fourth Grade
Fifth Grade

F — Fall	S — Spring

Analogous instruments have been developed to collect information on children's status and growth in the core academic areas directly from the children and indirectly from their teachers during the kindergarten through second-grade years. Teacher ratings are used to augment and extend the range of cognitive data collected on the children.

A.6.2 Approach to developing the cognitive battery

Consultations with curriculum specialists and teachers guided the development of the ECLS direct cognitive assessment battery framework and item pool. In addition, the ECLS battery framework drew upon the recently completed National Assessment of Educational Progress (NAEP) frameworks and other relevant domain frameworks for reading, mathematics, science, and social studies. The overlap in the goals and skills of the NAEP and ECLS frameworks allow for comparisons between the two studies and potentially enriches what is learned from each.

While the ECLS assessment battery frameworks have much in common with the NAEP framework and others, it differs from these in the following ways. The ECLS will measure children's growth of cognitive skills and knowledge across the whole span of the study. The longitudinal design of the ECLS dictates the necessity of a vertical scale that can measure valid change scores within more focused cognitive domains. In addition, the length of time that children in the early primary grades can be realistically assessed constrains the administration of direct assessment batteries in the ECLS. This time restriction directly limits the number of items within a set of skills that can be measured in the ECLS.

For each domain subtest, a pool of items was created. The item pool includes items selected from published instruments and items developed specifically for the ECLS. Sources of items from published instruments include the Peabody Picture Vocabulary Test (PPVT-3), Peabody Individual Achievement Test (PIAT), Primary Test of Cognitive Skills, Test of Early Mathematics Ability (TEMA-R), Test of Early Reading Ability (TERA-R), and the Woodcock-Johnson Psycho-Educational Battery-Revised Achievement Tests. Teachers, subject area specialists, and test development staff have evaluated the items for their relevancy and importance to specific domains and subdomains, sensitivity to diverse populations, and clarity and validity.

Language/literacy.	The ECLS language/literacy domain test specifications are derived mainly from the 1992 and 1994 NAEP Reading Framework. Other sources of information for the reading test item pool include the Standards for the English Language Arts (1996). Curriculum specialists in early literacy and reading and focus groups of primary grade teachers reviewed the proposed framework and item pool. Items in the reading subtest measure the following skill and knowledge categories: basic skills, vocabulary, and comprehension skills. In the early primary grades, basic skills is a measure of early literacy such as print awareness and orientation, phonetics such as the knowledge of letter sound relationships, and sight word vocabulary. Vocabulary measures children's knowledge of receptive verbal concepts. Children's comprehension skills are measured with the use of listening comprehension and reading comprehension items. The types of comprehension skills measured by the listening and reading comprehension items are similar to NAEP's four types of reading comprehension skills (i.e., initial understanding, developing interpretation, personal reflection, and demonstrating a critical stance).

In the kindergarten and first grade assessment batteries, language/literacy battery items measuring relevant reading skills include items such as left-right orientation of print, letter names, word rhymes, initial and final sounds of words, and recognition of high frequency words (i.e., sight vocabulary) and sentence reading. Measures of children's language include receptive vocabulary items and expressive language use (e.g., use of complex sentence structures).

Mathematics. The mathematics domain test specifications are primarily based on the NAEP 1996 Mathematics Framework. This framework is based largely on the curriculum standards from the Commission on Standards for School Mathematics of the National Council of Teachers of Mathematics. Items were allocated across specific mathematical content strands on the advise of math curriculum specialists. The items are distributed across five specific mathematical content strands by the three main cognitive processes of conceptual knowledge, procedural knowledge, and problem solving. The five mathematics content strands include the following: 1) number sense, properties, and operations; 2) measurement; 3) geometry and spatial sense; 4) data analysis, statistics, and probability; and 5) patterns, algebra, and functions.

Mathematics battery items measuring quantitative and analytic skills in the kindergarten and first grade include recognizing numbers, counting, comparing and ordering numbers, solving word problems using simple addition and subtraction with and without the aid of illustrations or manipulatives. Other measures of mathematical concepts include recognizing and solving problems involving graphic and numeric patterns and geometric relationships. Items involving the interpretation of picture graphs measure beginning analysis and statistics skills.

General Knowledge. The ECLS general knowledge domain subtest includes items that measure knowledge and skills in the natural and social sciences. The test specifications for the general knowledge domain subtest were mostly developed from recommendations of content area specialists and are similar to the NAEP frameworks in science (1996) and history (1994) and published standards for science and social studies. The content specifications of this subtest are classified into science and social studies which more closely reflects elementary school curricula.

The ECLS science framework differentiates between two broad classes of science competencies: conceptual understanding and investigation skills. Conceptual understanding refers to both the factual knowledge base and theoretical concepts that children have developed for the reasons things occur as they do. As children move up in grade level, the emphasis in ECLS will be more on the adequacy of conceptual understandings than the grasp of discreet facts. Investigation skills refer to children's abilities to form questions about the natural world, to answer them on the basis of the tools available and the evidence collected, and to communicate one's answers and describe how one achieved the answers. Items measuring investigation skills are further organized into the following subcategories: 1) collecting data and classification, 2) communication, and 3) drawing inferences. The science items are roughly distributed equally among the earth, life, and physical science content areas.

Children's knowledge, understanding, and application of scientific concepts is measured by asking children to name something, describe a process, show a relationship, and give reasons for how or why something happens or exists. Examples include the naming of a body part, explaining the reason for the melting of ice, pointing to one of four illustrations of objects (i.e., paper clip, glass, sweater, and apple) which goes with the stimulus illustration of a magnet.

The items related to social studies are allocated across five content areas: 1) history, 2) government, 3) culture, 4) geography, and 5) economics. These are further categorized into items measuring knowledge and items measuring the ability to analyze and interpret information about the social world. The ECLS social studies items for kindergarten and first grade reflects the concepts and skills within the culture and geography domains commonly taught in these early grades. Examples of these items include the naming of historical events such as Independence Day. Children's understanding of occupations and roles of workers is measured by identifying the a dentist from four pictures or orally naming the person who delivers mail.

A.6.3 Two-stage administration

The range of cognitive skills and knowledge children bring to school varies greatly. In a longitudinal study where the measurement of growth is a primary goal, minimizing item floor and ceiling effects is especially critical. To achieve reliable growth measures of the specific cognitive skill domains, direct cognitive measures must be adaptive with item difficulties tailored to each child's present level of performance. Special procedures were designed during the development and administration of the cognitive measures to minimize floor and ceiling effects which typically distort gain scores.

Adaptation of the ECLS test battery subtests is accomplished by a two-stage procedure involving the administration of a routing test followed by one of several alternative second stage tests. At the first stage, all children are administered a short routing test of specific domain items having a broad range of complexity/difficulty levels. The routing test includes between 10 to 12 items with about half of the items in the moderate range of complexity for the particular school grade and the remaining items divided between very complex and very simple items. Performance on this test is used to determine the appropriate second stage assessment form. The items in the routing test constitute a set of common items that will also be used to put all scores regardless of which second stage test form is administered on the same vertical scale. Just as the content and skills increase in complexity and difficulty from one grade to the next, so to will the routing test items change to reflect the increasing complexity of subject domain-related material. At each grade, the easiest or simplest items will be replaced with more very difficult or complex items to allow for the vertical equating of items across three within grade alternative second stage forms and to provide the link to equate the scores across the grades. The routing test shares the majority of its items with the preceding grades' first stage test making across grade scaling more efficient. Consequently, all individuals in a given grade will take a substantial core of common items which further contributes to the efficiency and reliability of scaling across the forms and grades.

At the second stage, each child will be administered one of the alternative test forms depending on his or her performance on the routing test. Items in these second stage forms are shared across the grades. The sharing of items among forms and across grades adds stability to the vertical scaling within and across grades. The psychometric report for the National Education Longitudinal Study of 1988 describes this approach in some detail.5 The use of multilevel forms at each grade level for each domain subtest also minimizes the floor and ceiling effects resulting in more accurate estimates of children's growth.

During kindergarten and first grade, the cognitive assessment battery is individually administered with the use of a computer and an easel. The child is shown pictures on an easel, while the examiner reads directions and enters the child's responses in a computer. Although assessment is relatively untimed, 45 minutes have been allocated to administer the battery. It is expected that each domain subtest will take approximately 15 minutes to administer both the routing and alternative test forms. Items for the kindergarten and first grade are being field tested during the 1996-1997 school year to identify those items which will be included in the final forms. A pretest of the two-stage test administration is scheduled for the fall of 1997 as a part of the Phase I contract.

The battery items include open ended and multiple choice questions. Children will be asked to demonstrate their skills, knowledge, and understanding of relevant domain concepts using a variety of response formats such as responding orally, pointing to one of four possible answers that best represents a stimulus, and using manipulatives.

Some children may understand a concept or mathematical skill but be unable to demonstrate that understanding verbally. Thus, items requiring verbal and nonverbal response formats were incorporated into the item pool to assess conceptual knowledge (i.e., general knowledge) and mathematical skills separately from the ability to use the English language. Verbal items require children to name or describe something whereas nonverbal items require the matching or categorizing of objects or entities according to properties and features.

A.6.4 Group testing in the second grade

Item difficulties in the item pool of each of the direct cognitive domain subtests extend from beginning kindergarten to second grade. However, the direct cognitive assessment in second grade shall be group administered. Consequently, additional work is necessary to identify items in the pool for group administration. First, it shall be necessary to adapt items shared with the preceding grade domain subtest for group administration. Second, the items must be compared to ensure that the group administered items are on the same scale as the individually administered items. A bridge study or some other means for equating the items is critical for confirming the comparability of these items.

A second grade version of the direct individually administered cognitive assessment battery is being developed during Phase I. No work is scheduled on the group administered version. The work required to develop a group administered test is described in Optional Task C.

A.6.5 Teacher cognitive ratings

Teacher cognitive ratings have been developed for the ECLS that measure teachers' perceptions of students' academic achievement and dispositions for learning in the three cognitive subdomains (i.e., language/literacy, mathematics, and general knowledge). The primary emphasis of the teacher ratings is to measure change in a student's academic and behavioral performance within the kindergarten year and between years for grades k through 2.

The questions ask teachers to make judgments about a student's achievement of specific skills, knowledge, and behaviors that reflect a representative sampling of the most recent national, state, and local curriculum standards in the three academic domains and have also been found to be predictive of achievement in the research literature. The following criteria were used in selecting the items: 1) represent the diversity of actual classroom practices, 2) reflect developmental change across time (i.e., kindergarten through second grade), 3) have a high level of specificity, 4) easily observable by teachers, 5) reflect a balance between expressive and receptive language competencies, 6) sufficiently broad to permit fair evaluations of diverse populations, and 7) minimally obscured by language proficiency and cultural experiences. The items are designed to represent average grade level performance indicators and examples. While the performance indicators are often constant across the different years, examples within each item represent grade level distinctions and a developmental progression across the years.

A.7 Measurement of socioemotional development

Another critical element of the ECLS is the measurement of children's socio- emotional status. Children's socio-emotional development is inherently related to their cognitive development and success in school. The ECLS socio-emotional assessments focus specifically on social competence and the skills and behaviors that contribute to that competence.

The primary vehicle for measuring children's socio-emotional performance is an adaptation of the Social Skills Rating System (SSRS, Gresham and Elliot 1990). Additional items have been added to reflect behaviors indicative of approaches to learning, positive affect, and pragmatic language abilities. Both teachers and parents will rate children's prosocial behaviors and social competence skills as well as problem behaviors. In addition, teachers and parents will rate the importance of each skill or behavior in their respective environments (i.e., classroom and home). Parallel items are asked of both teacher and parent (e.g., concentration, cooperation, task orientation). While teachers will rate children's socio- emotional functioning using self-administered survey forms, parents will be asked to rate their children during a computer assisted interview (i.e., CATI or CAPI). Ratings of children's socio-emotional status will be collected from parents and teachers during the fall and spring of the kindergarten year and in the spring of succeeding years.

Information about children's social cognition (i.e., understanding of social problems and how they can be solved) will be collected from the children themselves. An adaptation of the Social Problem Solving Measure (SPSM, Weiss, Dodge, Bates, and Pettit 1992) was pilot tested in spring 1996 and is being field tested in 1996-97. The items represent problems children experience in the following settings: 1) gaining entry in social group settings, 2) taking turns, 3) responding to physical and verbal aggression, 4) responding to conflicts involving objects, and 5) responding to disrespect for personal rights. To measure children's social cognition, children are shown illustrations of the hypothetical scenarios representing each of the different problem situations and asked to generate as many behavioral solutions as possible. Two sets of identical hypothetical situations have been designed, one for girls and another for boys.

Variations of children's responses to the hypothetical measures may show the developmental growth of children's problem solving abilities. In addition, comparing the results of the SPSM measure with the ratings on the SSRS may provide some insight into the behavior ratings of the SSRS.

A.8 Physical and Psychomotor Assessment

Children's developmental status in one domain, including motor development, affects their development in the other domains. Because of the functional interrelatedness of the human body systems, it is necessary to measure children's physical and psychomotor development. In the ECLS, direct measures of children's physical and psychomotor status at kindergarten will include motor ability and coordination and visual motor skills (and optionally measures of height and weight). Children's measures of motor ability and coordination will involve balancing on one foot, hopping on one foot, skipping, and walking backwards. Copying shapes using paper and pencil, drawing a child, and block building will represent the tasks for measuring children's visual motor skills.

Measures of children's physical development will be collected in the fall and spring of the kindergarten year and in the spring of the first and second grades. Children's psychomotor skills will only be assessed in the fall and spring of the kindergarten year.

Additional information of children's physical development, physical well-being, and health will be collected from parents and teachers.	Parents will report on physical activity, nutrition, illness and disease, health behaviors, and general health and teachers will rate children's physical activity level and health.

B. Sample Design

After an extensive study in Phase I, a sample design which meets the goals, requirements and constraints of ECLS, was developed and the plans for selecting this sample are documented in the Early Childhood Longitudinal Study Design Report. The plan calls for a dual-frame (i.e., list and area frames) multi-stage stratified design be used to select the base-year sample for this study. Geographic areas (e.g., county or group of counties) will be the primary sampling units (PSUs); schools/kindergarten programs will be the secondary sampling units (SSUs); and children will be the ultimate sampling units (USUs). Possible stratification variables are: census region, level of urbanization, and state for stage 1; school size, and school type (public vs. private) for stage 2; and student race-ethnicity for stage 3. The sample will be selected with probability proportionate to size (PPS) in the first two stages and with oversampling of Asian and Pacific Islanders in stage 3. The plan recommends a sample of 23,500 children who are attending 1,000 schools with kindergartens (200 private and 800 public schools) in 100 PSUs. In follow-up surveys, children who were selected for previous waves will be selected into the current sample with given probabilities. For the school sample, only those schools that enroll sample students from previous waves are eligible for the current wave and will be selected with given probabilities. Although key sampling issues have been decided and documented in the ECLS Study Design Report, the contractor is expected to develop many of the operational details such as the sample allocation among strata that are required to implement the plan.

The sample for the ECLS shall be drawn from clustered geographical areas. The decision to use geographic areas for the first stage of sampling was based on a concern for economy in face-to-face data collection activities and the expected high rates of mobility for the ECLS sample over the six year study period. NCES/ED's current plans call for a first stage sampling at the level of county or county groupings, school districts, MSAs, or Zip Codes with the selection of at least two PSUs from each stratum.

B.1 Kindergarten sampling frame

The ECLS contractor for phase II shall develop a sampling frame of all kindergarten programs in selected PSUs. Because a complete frame of kindergarten programs from which to select such a sample does not presently exist, several different sources will have to be used to develop the frame. Public school kindergarten programs will be identified through the Common Core of Data (CCD). Some private school kindergartens may be identified through commercial data bases such as Quality Education Data (QED), and through the Private School Survey (PSS), which is conducted for NCES/ED by the U.S. Bureau of the Census.

At the current time, there is some uncertainty about the coverage of kindergarten programs in private school frames. Existing frames may not adequately cover the population of kindergarten children who attend programs operated out of private nursery schools or other early childhood programs. The 1995-96 PSS was designed to help to alleviate this problem. Special efforts were made to identify the full-range of schools and programs offering kindergarten.

A dual-frame approach will be applied to solve this problem. The private school sample selected from list frames will be supplemented by a sample selected from an area frame. An area frame sample can be produced by 1) sampling geographic areas (i.e., PSUs), 2) conducting an extensive search for eligible schools/programs within these areas, 3) matching these eligible schools/programs with those appearing on the list frame, and 4) sampling eligible schools/programs that do not appear on the list frame for participation. These are the procedure that have been successfully used by the PSS. The ECLS contractor for Phase II shall conduct all four tasks outlined above.

B.2 Sample attrition and mobility

Attrition needs to be examined carefully when designing a sample for a longitudinal survey. A moderate level of attrition for each single wave may become a severe problem over the life of the study. In addition to the well-documented factors for non-response (e.g., refusals, unable to locate, and out-of-scope) which are common to both cross-sectional and longitudinal studies, longitudinal studies are also affected by the mobility of the sample.

Sampled kindergarten children may change schools for many reasons including: 1) their families move to a new location; 2) children complete the highest grade of the schools they are currently attending; and 3) the school closes its door or school boundaries change. Such student mobility may necessitate subsampling in follow-ups in order to curb cost. Ignoring family movement, data from the CCD indicates that about 13% of public school kindergarten children change schools after they complete third grade because their schools do not serve children beyond grade 3. About 20% of public school children change schools after the completion of fourth grade for the same reason. The percentages of kindergarten children in private schools who change schools after grades 3 and 4 are 5% and 7%, respectively according to data from PSS. Transitions of this type, compounded with high family mobility of primary and elementary school-age children (e.g., data analysis of mobility for the Prospects Study suggests that family mobility rate ranges from 27.3% for five-year old children to 18.3% for ten-year old children), will result in large numbers of children attending schools other than the ones for which they were selected in the base year. It is anticipated that the numbers of schools that enroll sample children will expand as the study proceeds, with high numbers of schools enrolling just one or two sample students. Subsampling these schools (i.e., schools with few base year sample students) to reduce survey costs will result in further losses to the student sample.

Mobility rates may change over time. In addition, they differ by socio-economic, geographic, or demographic characteristics such as family income, school type, locality, and race-ethnicity. Children with certain characteristics are more likely than their peers to attend schools enrolling one or two children sampled in the previous wave. To ensure that a sufficient number of children for policy-relevant subgroups will be kept, the contractor shall take account of these factors when devising subsampling plans for follow-up surveys (e.g., first through fifth grades).

B.3 Oversampling of minority students

Asian and Pacific Islanders will be oversampled in order to meet the precision requirements of the study. Experience gained from past longitudinal studies indicates that oversampling can be accomplished through many different strategies. NELS:88 successfully implemented a strategy that utilized student rosters within selected schools. Whether this approach can be used in the ECLS is being explored in the 1996-97 field test.

B.4 Freshening of sample

Because kindergarten is not mandatory for all children, it will be necessary to freshen the kindergarten sample when this cohort enters first grade. Freshening in later grades may also be desirable. One possible way to achieve freshening is to apply the freshening procedure which was developed for the first follow-up of NELS:88.6 This procedure links the students who are eligible for freshening to the students selected in the previous wave. The selection probability of the freshened student is derived through these linkages. Other strategies to freshen students may be proposed. The contractor shall freshen the kindergarten sample at first grade.

B.5 School-level estimates for later waves

As a part of the ECLS design, school-level information will be routinely collected. Because schools sampled at the start of the study will be selected according to some known probabilities, producing school-level estimates will be rather straightforward. This will not be the case for later follow up studies. During later waves, schools are selected into the sample because they enroll base year sample children. These schools may or may not have been sampled during the base year. Sample weights for followup schools will not be as easily produced as those for the base year schools. Producing school-level estimates from data obtained from these schools will be more problematic. A possible solution to this problem involves the collection of data that allows school followup weights to be produced. The contractor is required to provide school weights for the schools included in the first followup study, i.e., first grade schools.

The situation described above was encountered in the NELS:88 study, as students moved from eighth to tenth grade. Based on three different models, the selection probabilities of the first follow-up sample schools for NELS:88 were developed and follow-up school weights were produced. The ECLS Study Design Report has recommended these three procedures for ECLS . Other procedures may exist or can be developed.

Regardless of which procedure is proposed for this study, the contractor shall specify at the sample design stage the data needed to obtain these school selection probabilities. The contractor shall evaluate the availability and accuracy of these data prior to full-scale implementation. A contingency plan for obtaining the school weights if the proposed procedure fails shall be developed. One possible strategy is to develop several procedures and collect information specified in the procedures from all base year schools. School weights would then be generated separately for each of these procedures. A decision then could be made as to the best set of weights to be used for estimation and inference. Other strategies can be developed. The impact of school weights (on estimates and corresponding developed standard errors) from all proposed strategies shall be carefully evaluated and documented.

C. Methodological Inquiries and Data Quality

The data that will be collected through the ECLS will be used by policy makers, educators, and researchers to understand children's school experience and to guide policy decisions at local, state, and national levels. It is important that the information used by these groups be based on sound research practice and that considerable attention be paid to identifying the potential sources of error, quantifying this error, and designing techniques to either reduce the error or minimize its impact on survey estimates.

The work to be carried out in support of the ECLS includes a variety of activities that are directed toward improving the quality of the survey data. Extensive testing of study procedures, instruments, and methods was done during Phase I.

The methods and procedures used during the conduct of the study shall undergo continued evaluation beyond what is done during the design stage of the study. NCES/ED hopes to be able to set aside funds for the conduct of experiments and studies designed to enhance knowledge of the effects of different design features on the quality of the data. NCES/ED is interested in studies that measure the validity and reliability of the survey data collected from children, parents, and teachers concerning the children's educational experience and learning environments. It is also especially interested in studies that assess the validity and reliability of data that are used to describe children's status on different developmental dimensions during kindergarten. During the design and conduct of the ECLS, it is important that the contractor takes appropriate steps to limit non-sampling errors as well as to limit errors that are introduced through the sample selection process.

D. Optional Special Studies and Supplements

NCES/ED is supporting the development and implementation of a core study design that will achieve the goals of the program and provide the data required to answer the major research and policy questions guiding the study. NCES/ED plans to explore other ways of enriching the study. The study may be expanded in scope or enriched through a variety of special studies and supplements. Whether or not these studies and supplements are incorporated into the final ECLS design will depend on the activities and outcomes of the Phase I. Some of these may be supported by NCES/ED depending on the availability of funds, while others may be sponsored by other parts of the U.S. Department of Education, other Federal agencies, and foundations.

NCES/ED is considering expanding the ECLS design in several ways, including: 1) oversampling specific populations of children, 2) adding questions to one or more questionnaires in order to explore more fully an area of interest, 3) the use of supplemental instruments for administration to one or more groups of respondents, 4) adding rounds of data collection (e.g., fall first grade) in order to study better children's growth over the critical early school years, and 5) including methodological studies to explore issues of validity and reliability of the data gathered through traditional survey methods.

The special studies and supplements that NCES/ED is considering are described briefly below. Other topics may surface as a result of the 1996-97 field test and as data from the early rounds of the study are analyzed. The final decision as to whether or not to incorporate one or all of these studies and supplements and their final design specifications will be at the discretion of the government.

D.1 Optional Special Studies

Contingent on the activities and outcomes of Phase I, several special studies shall be conducted in conjunction with the core ECLS design. These include: 1) a validation study of teacher reports concerning children's classrooms; 2) administering an additional questionnaire to the special education teachers of children with disabilities; and 3) verifying children's Head Start program participation.

Validation Study of Teacher Reports of Classroom Characteristics. An important goal of the proposed program of special studies is to obtain a clearer understanding of the relationship between the environments in which learning takes place and children's education outcomes. It is important that data collected on the activities and interactions that take place within the home, classroom, and school be reasonably sound and representative of some of the key constructs in the ECLS.

While NCES/ED believes that valuable information can be obtained from different respondent groups (e.g., parents and teachers) on the activities and interactions that take place in the home and outside the home, it also recognizes the value of conducting studies to improve the quality of the survey data. Of particular interest to NCES/ED is the design of activities that will collect data to confirm the validity of key classroom measures found in the teacher self-administered questionnaire.

The contractor shall design and conduct a validation study of teacher reports of classroom characteristics. This study shall focus on measures of key constructs related to instructional practices. In selecting an approach, the contractor shall consider the level of classroom intrusion/disruption and burden that the study may entail.

Special Education Teacher Questionnaire. One of the objectives of the ECLS is to provide rich data to understand children's learning environment at school. NCES/ED has already planned to collect information in the study about the children's classroom environment, experiences, and competencies. Information on these areas will be collected from school records and various teacher survey instruments.

It appears that the core sample of the ECLS will include sufficient numbers of children receiving special education services to support independent analysis of this group as a whole. And the number of children in this group is expected to increase over the life of the study as more children with disabilities are identified and receive services. A proposed activity for the ECLS is to collect additional information on the learning experiences of children with disabilities who receive special education services from special education service providers (i.e., special education teachers). This supplement will be funded by the U.S. Department of Education Office of Special Education if it is incorporated into the ECLS design.

A survey of the special education teachers of core sampled children with disabilities will be conducted at each wave of data collection. The survey shall be conducted using self administered questionnaires and shall be limited to 10 minutes on average at each wave. The frequency at which topics are covered in the questionnaire shall depend on the changing nature of the topics with some items only needing to be asked once during the initial interview while others need to be asked at each wave. Special education teachers of core sampled children with disabilities would be asked to provide information about themselves and the sample child to include the following:

•	Child specific characteristics

•	Referral history

•	Services provided

•	Placement and service delivery

•	Instructional approaches and materials, including assistive devices

•	Goals and expectations for student

•	Consultations with regular classroom teacher

•	Teacher education and experiences

Verification of Head Start Program Participation. The Administration on Children, Youth, and Families (ACYF) is proposing a supplemental study of Head Start children in the ECLS. The purpose of this study is to collect descriptive information on Head Start children's school experiences after they leave Head Start. ECLS children who attended Head Start the year prior to kindergarten would be identified. Once identified, the full complement of ECLS instruments and procedures would be used with the children in the same ways as they are with the full sample.

Three issues pertaining to the ACYF supplement of Head Start children are being explored in the 1996-97 field test: 1) can children's participation in Head Start be accurately reported by schools and parents; 2) should Head Start children be oversampled; and 3) if oversampling is necessary, what is the best strategy for oversampling Head Start children.

Because children will not be identified when they are currently enrolled in a Head Start program, but rather retrospectively after they are enrolled in kindergarten, there is potential for misidentification of Head Start children. The misidentification could be an under-representation, where children who had attended Head Start are not identified, or it could be an over-representation, where children are identified that actually did not attend Head Start.

In the fall of 1996, at the time the core field test kindergartner sample was selected, school officials in all 50 field test schools were asked to identify who among the kindergarten children in the sample attended Head Start the year before and to identify the Head Start programs they attended. During the fall field test parent data collection period, the parents of kindergartners selected for the ECLS field test were asked a series of questions about their child's possible participation in Head Start and the center children attended.

All unique Head Start programs identified by schools and parents were called to verify the attendance of children reported by schools or parents as attending Head Start the year before. The center will always serve as the definitive source for Head Start attendance against which over-and under-identification on the part of other sources will be measured. All students appearing on the center list will be designated as verified Head Start children.

The main purpose of this field test activity is to evaluate the accuracy of various sources of identification. One question is how accurately can parents and schools identify students as attending a Head Start program one year earlier? We hypothesize that parents will tend to over-report Head Start attendance, whereas schools will under-report attendance. Because parents may mistake any Head Start-like compensatory intervention for Head Start, parents will most likely tend to over-report their child as attending a Head Start program. Schools, on the other, will most likely be unaware of who attended Head Start. The Head Start center list of attendees will be compared to the parent-generated and school-generated lists of children attending Head Start programs as a means of evaluating schools and parents as sources of information on Head Start attendance. Sources--schools and parents--will be identified as either tending to over-report or under-report Head Start attendance, and the rate (or magnitude) of over and under-reporting will be calculated.

In the base year ECLS, it may be necessary to verify Head Start children's attendance. Whether or not verification is needed and the best way to accomplish this task must await the outcome of the ongoing field test trials on this issue.

Fall first grade and summer learning. The current design of the ECLS includes two data collection points (fall and spring) during the base year of the study and annual spring collections thereafter. Writing about the ECLS design and the conflict between the descriptive and explanatory goals of the study, Raudenbush (see ECLS Study Design Report) has suggested that the current design is not the best for answering certain questions about the growth in children's achievement. Noting the differences in the rates of growth during the academic year and the summer, Raudenbush suggests that two assessments per year are needed in order to disentangle the school effects and home effects on children's learning. Others have suggested that "real" school begins with first grade, and that the current design does not capture baseline information on children as they enter first grade.

At the discretion of NCES/ED, the contractor shall design and conduct a fall first grade data collection. The contractor shall administer the ECLS assessment battery (both the direct and indirect components) to the ECLS kindergarten sample (either all or a subsample of the children) during the fall of first grade, fall 1999. In order to capture information on children's summer learning experiences, the contractor shall design and conduct a 15 minute parent interview for each sampled child using the same methods being used in the core study. This interview shall concentrate on capturing information about children's summer activities and experiences that might help to explain differences in summer growth curves for advantaged and disadvantaged children. The contractor shall design and have teachers of the sampled children complete a short self-administered teacher questionnaire (15 minutes) that will be used to collect information about children's fall first grade school/classroom experience. These will be the only instruments administered in the fall of first grade.

D.2 Optional Supplements

There are several supplements to the ECLS that were designed and field tested during Phase I which may or may not be fielded during the base year and beyond. Final decisions about these supplements will be made following completion of the field test and are contingent on funding from agencies other than NCES/ED.

Oversample of American Indian Children. NCES/ED's preliminary sample specifications for the ECLS called for the oversampling of black, Hispanic, and Asian children in order to support separate analyses of these groups. During Phase I, NCES/ED supported some design work to look into the feasibility of oversampling American Indian children for the ECLS in order to support separate national estimates for this group of children. Several different methods for increasing the number of American Indian children in the ECLS sample were investigated. These included oversampling American Indian children within the core ECLS sampled schools, selecting additional schools with varying levels of American Indian concentrations, and combinations of these approaches. To date, none of these approaches has been field tested. These methods are described in the ECLS Study Design Report.

Oversampling of Head Start Children. One of the goals of NCES/ED's early childhood longitudinal program is to describe the school experience of children who have participated in different types of early care and education programs. Although the ECLS is limited in the degree to which it can achieve this goal, it provides a means for examining the early school experiences of children who have participated in one of the most comprehensive early care and education programs for low income children. Head Start has been providing comprehensive services in the areas of education, health, and social services to mostly low- income 3- and 4-year-old children and their families for more than 25 years. Despite this long history of service, there is very little systematic data at the national level available to describe the school and family experiences of children once they leave the Head Start program and enter kindergarten and elementary school. The study, if incorporated into the ECLS design, will be funded by the U.S. Department of Health and Human Services, Administration for Children, Youth, and Families, Head Start Bureau.

For the optional longitudinal study of Head Start children the contractor shall concentrate on children who participate in Head Start programs during the year prior to kindergarten. The contractor shall design the study to capture data to describe the school experiences and progress of children who participated in these programs. The study is not intended as an evaluation of the Head Start programs. The design of the Head Start longitudinal study shall include the following:

◦	A sample of kindergarten children who were enrolled in Head Start programs during the year preceding kindergarten.

◦	Head Start participants shall be sampled from the schools sampled for the ECLS.

◦	All Head Start sample participants shall be subject to the same procedures and conditions as all other base-year ECLS sample members. All instruments and procedures used with members of the ECLS main sample shall be used with the Head Start sample.

◦	The design of the longitudinal study of Head Start children requires that members of the Head Start sampled be followed for the same period of time as the main ECLS sample.

During the 1996-97 field test, two issues pertinent to the oversampling of Head Start children are being explored in more detail:

(1)	the need to oversample and the sample size needed based on the analytical purposes of the ECLS Head Start supplement (ACYF is being asked to specify the key analytic questions); and

(2)	the efficiency of various rates of oversampling; for example, if based on the analytical purposes of the study, too few core ECLS kindergarten children among the 24 who will be selected for the ECLS attended Head Start the year before, the efficiency of various rates of oversampling additional children from within the ECLS school will be examined. It is also possible, that the analytical needs of the ECLS Head Start supplement could not be met by oversampling strictly within core ECLS schools and that additional schools would need to be drawn into the study to achieve the requisite sample size for addressing the research objectives of the Head Start supplement and this also will be examined in the field test. Of course, the outcomes of the field test inquiries into the accuracies of the different sources of information on children's participation in Head Start are closely linked to the best method for oversampling Head Start participants.

Depending on the outcomes of the field test, Head Start children may be oversampled in the ECLS base year.

Oversampling Populations of Children with Disabilities. One of the benefits arising from a national study of kindergartners is the collection of data on a wide range of early learning experiences and environments for different groups of children. The core ECLS sample of kindergartners will include 1) children with disabilities who have been identified and are receiving special education related services, 2) children with disabilities who are not receiving special education related services, 3) children who are at risk but have not yet been identified, and 4) children who have received special education related services but are no longer receiving these services. As children move through kindergarten and grades one to five, the distribution of children in these different groups will change. While it appears that the core ECLS sample will include sufficient numbers of children receiving special education services to support independent analyses of this group as a whole, a sufficient number of children to support independent analyses of children with specific disabilities (i.e., blindness, deafness, orthopedic impairment) is not expected unless such children are oversampled. In addition, without augmentations to the design, other key issues related to the population of children with disabling conditions and their special education needs will not be adequately addressed in the core ECLS. The longitudinal study of children with disabling conditions, if incorporated into the ECLS study design, will be funded by the U. S. Department of Education, Office of Special Education.

The longitudinal study of children with disabling conditions shall be designed to capture data to describe the school experiences and progress of children with disabling conditions especially children with the following specific disabling conditions—blindness, deafness, and orthopedic impairment. The design of the study shall include the following:

◦	A sample of kindergarten children who have been identified and are receiving special education related services before entering kindergarten. The sample should be of sufficient size to perform the analytic analyses necessary to answer the research purposes of the Special Education students longitudinal study.

◦	While most children with disabling conditions shall be sampled from the schools sampled for the ECLS, sampling sufficient numbers of children with low incidence disabling conditions such as blindness, deafness, and orthopedic impairments shall require the selection of schools outside the ECLS schools or the use of other sampling frames (e.g., national registries).

◦	To the extent possible, members of the Special Education sample shall be subject to the same procedures and conditions as all other base-year ECLS sample members. Decision rules for inclusion developed during Phase I shall be applied to determine whether individual children with disabilities are excluded from any of the procedures and instruments used with the ECLS sample members without disabilities.

◦	When assessing children who receive special education, classroom and special education teachers shall be asked whether the ECLS direct assessment measures can be used, whether accommodations are needed to administer direct measures, and the types of accommodations needed before administering assessment. In addition, any accommodation made for the child shall be recorded at the time of the assessment, and score reports involving accommodations shall be flagged on the ECLS data files.

◦	Instruments specifically designed for the special education teachers of children with disabilities shall be used to collect more information on the special education program placement experienced by these children.

◦	The design of the optional study of children with disabilities requires that the members of this sample be followed for the same period of time as the main ECLS sample.

Food and Consumer Service. One of the purposes of the ECLS is to study the influence of children's physical characteristics and household circumstances on their behavior, cognitive performance, and school progress. Thus, a proposed area of investigation for the ECLS is an examination of the relationship of children's nutrition and participation in school nutrition programs to their behavioral and cognitive outcomes.

NCES/ED has already planned to collect information in the ECLS about children's participation in school breakfast and lunch programs and their families' participation in federal assistance programs (e.g., food stamps, WIC, AFDC). As indicators of behaviors that have been found to be linked to children's nutritional status and transient hunger, NCES/ED has also planned to capture information on children's absenteeism, tardiness, attentiveness, level of concentration, irritability, and hyperactivity from school records and the teacher questionnaire.

An additional supplement to the ECLS would be to collect measures of children's physical activity, their height and weight, and the food sufficiency situation of their families. Information on how often children engage in activities requiring physical exertion would be obtained from parent and teacher reports; children's height and weight would be measured by field interviewers at each scheduled data collection; and, data on families' abilities to get enough to eat would be acquired from the parent questionnaire.

If this additional supplement is included in the ECLS, it will be funded by the U.S. Department of Agriculture, Food and Consumer Service (FCS).

E. NCES/ED Standards

NCES/ED has developed and implemented a set of standards that set forth guidelines to insure the quality of NCES/ED's work. The contractor shall follow these standards. A copy of the publication, NCES Statistical Standards and Policies (U.S. Department of Education, 1992), can be obtained by calling the Statistical Standards and Services Group of the National Center for Education Statistics at (202) 219-1831.

III. SCOPE OF WORK

A. Overview of Tasks

This Scope of Work is written in two sections (Core and Optional Tasks). The first section describes the core tasks that are required to prepare for and conduct the ECLS kindergarten fall and spring, and first grade spring data collections. The contractor shall perform a series of tasks that will lead to 1) the final design and selection of national samples of schools, kindergartners, and first graders; 2) the collection of data from children, parents, school administrators, and teachers during the fall and spring of kindergarten and spring of first grade; and 3) processing the data and preparing it for public release. The second section describes a set of optional tasks to be performed in preparation for later ECLS follow up activities. If the government chooses to exercise these options, the contractor shall perform a series of tasks directed toward extending the design of the base year and first follow up studies to grades 2 through 5. The contractor shall give special attention to developing the instruments and procedures that will be needed to assess children's growth in critical cognitive and noncognitive domains found in earlier rounds of data collection.

B. Core Tasks to Be Performed

The core tasks described here represent activities that are required to prepare for and implement three data collections: 1) fall kindergarten, 2) spring kindergarten, and 3) spring first grade. A few of these tasks will be performed only once (e.g., task 1 and 3), while most will be repeated for each of the data collections. While the outcome of some tasks will be the same each time it is performed, the level of effort required to accomplish these outcomes will not necessarily be the same. For example, task 10 requires that a set of training materials be designed and produced and that training sessions be designed for each of the study components. However, much of material that is developed for the fall base-year training and many of the activities planned for the training sessions can be used and modified for later rounds of data collection.

Many of the materials and approaches that are needed to complete these tasks were developed during Phase I. The ECLS Study Design Report contains the methods, procedures, and instruments that are being field tested in school year 1996-97. While NCES/ED does not expect major changes to these methods, procedures, and instruments as a result of the field test, some changes are inevitable. All outcomes of the field test and their implications for the ECLS design, will not be available until late spring and summer of 1997.

Task 1. Project initiation and planning

The contractor shall work closely with the NCES COTR and other NCES/ED staff to implement the study design developed during Phase I and the approach presented in their proposal. The contractor shall keep the project on schedule and within budget.

1.1 Final study reports

Upon award of the contract, NCES/ED will provide the contractor with copies of all final design reports and documents prepared during Phase I. The ss ECLS project director and staff shall review these documents.

1.2 Meet with NCES/ED

Within two weeks of the contract's start date, the contractor shall meet with NCES/ED to review the contract's tasks and to discuss issues related to the conduct of the study. The contractor shall come prepared to this meeting to discuss any changes in its technical approach based on its ongoing review of Phase I final reports and documents. The contractor shall be prepared to identify areas of concern and to suggest ways of responding to these concerns. The contractor shall also be prepared at this meeting to recommend the members of its ECLS Technical Review Panel.

The meeting shall be held in Washington, D.C. at the offices of NCES. In addition to key contractor staff, the NCES/ED COTR and other NCES staff, it shall be attended by the Contracts Officer and Contracts Specialist. In order to get the ECLS phase II contract off to a smooth and quick start, the contractor shall invite the ECLS phase I contractor's project director and other key project staff to this meeting. The contractor shall provide NCES/ED with an agenda at least five working days in advance of the meeting.

Task 2. Technical review panel and outside review

2.1 Role of the technical review panel. Studies of this scope, complexity, and importance require input from a number of individuals and organizations in order to address the data needs of policy makers and of those performing policy studies and educational research. The contractor shall form and work with a Technical Review Panel (TRP). The contractor shall ask panelists to review and comment on such matters as technical design and implementation, administrative practices, policy and research topics that are appropriate for the ECLS, and other technical and policy matters that surface from time to time. The TRP shall play an active role in the study by reviewing and commenting on overall research priorities, identifying policy and research questions, providing input about questionnaire and assessment instrument content, proposing analytical models and methods, reviewing work plans and their implementation, and reviewing and suggesting modifications to draft reports.

The TRP shall not function as a consensus group. Instead, the contractor shall ask the individuals on the panel to review and respond to matters concerning the design of the ECLS and to offer their individual opinions and evaluations on such matters. The contractor shall compile and weigh these different opinions and recommendations.

2.2	Composition of the panel and meeting schedule. The TRP shall be composed of national authorities in those substantive and methodological areas critical to the design and implementation of the ECLS. The contractor shall involve and inform the TRP members on the progress of the ECLS through consultations, reports and meetings. A list of all persons serving on the ECLS TRPs during Phase I and others who have consulted on different aspects of the project before and during Phase I can be found in the Field Test OMB Clearance Package.

The contractor shall recommend a schedule of panel meetings in keeping with the overall plans presented in its proposal. Panelists will need one day, prior to each meeting, to review materials and to otherwise prepare for the meeting. All meetings shall be held in Washington, D.C. unless otherwise approved by the NCES/ED COTR. The TRP shall meet no more than twice a per year for the life of the contract.

2.3 Contractor's responsibilities. The contractor's responsibilities related to the TRP shall include, but not necessarily be limited to, the following:

1.	Identifying and recruiting members for the TRP. Within two weeks of the contract's start date, the contractor shall identify, in writing, individuals with the needed expertise for possible membership on the TRP. The contractor shall identify the institutional affiliation of each nominee and provide a short biographical sketch highlighting expertise and prior work related to the ECLS. Following NCES/ED review of the list of individuals nominated for membership, which shall take no more than four weeks, the contractor shall contact each individual to solicit and finalize his/her participation.

2.	Inform the TRP of the progress of the study. In addition to the regular meetings, the contractor shall summarize the progress of the study for the panelists at least on a semi-annual basis.

3.	Prepare all materials and correspondence required for TRP consideration and review. After NCES/ED review and approval, share materials with the TRP for review and comment within five days of NCES/ED approval. The entire group of TRP members or individual members might be asked to review and comment on a variety of contractor produced reports and documents (e.g., field assessor training materials, methodology report, questionnaires, and content outlines).

4.	Schedule the meetings with panel members. All meetings with the exception of the first shall be scheduled at least 9 months in advance.

5.	Arrange for TRP meetings (e.g., meeting space and accommodations) and pay all associated expenses. Non-government panelists shall be paid an honorarium plus per diem and travel expenses. Government participants and guests at the TRP meetings will usually include 4-6 staff from NCES and 1-2 staff from each of the government agencies sponsoring components of the ECLS. The contractor shall also invite the ECLS Phase I contractor's project director and other key staff to attend these meetings.

6.	The contractor shall prepare and submit an agenda for each meeting to NCES/ED at least 1 month in advance. The agenda and any other materials that will be provided to the panel members for discussion at the meetings shall be delivered to panel members at least 1 week prior to the meeting.

7.	Prepare minutes of the meetings. The contractor shall tape record the meetings and submit a written summary to NCES/ED within two weeks of each meeting. The minutes shall include a summary of the discussions (There is no requirement for a verbatim transcript) and activities that took place during the meeting, highlighting major issues that were raised and decisions made. The minutes shall identify actions (long-term and immediate) the contractor will take to respond to issues that were raised and not resolved during these meetings. The minutes of the meeting shall be distributed to panel members and invited guests within three weeks of each meeting after NCES/ED review.

8.	Arrange and pay for specific work products not tied to a meeting. For example, TRP members might be asked to review and comment on a particular draft paper. Again, non-government panelists shall be paid an honorarium. The contractor shall provide all materials to panelists at least one week prior to the meeting.

9.	Supply NCES/ED with copies of all correspondence and other materials exchanged with panelists at the same time they are given to the panel members.

2.4 Other outside review. In addition to the contractor's TRP, other review panels may be required from time to time. For example, during Phase I, groups of subject area specialists were convened to review the specifications of the ECLS assessment framework and to review drafts of the assessment battery. If such panels of consultants should be needed again, the contractor shall have the same responsibilities for these panels as are outlined for the contractor's TRP (see 2.4 above).

Task 3. IMT/OMB clearance of main survey instruments

The contractor shall prepare a forms clearance package for obtaining both IMT (Information Management Team of the Department of Education) and OMB approval of the ECLS kindergarten and first grade data collections. The final draft package shall be submitted to NCES/ED not less than 150 working days prior to the start of the kindergarten fall data collection. In preparing the clearance package, the contractor shall follow instructions for completing Standard Form 83-I and writing the supporting statement required for Paperwork Reduction Act submissions. The contractor shall expect that several iterations of the forms clearance documents will be required before a final draft will be ready for submission.

Following NCES/ED review of the draft package, the contractor shall revise the package incorporating NCES/ED's comments and suggestions and resubmit the package. This version of the package will be distributed more widely within the Department of Education for further review. The contractor shall expect additional comments and suggestions within four weeks. Following this review, the contractor shall revise the package and submit it for final NCES/ED approval. Following approval, the contractor shall provide NCES/ED with 10 copies of the Clearance Package. NCES/ED will submit the package to IMT/OMB.

Much of the material that is required to be included in this package was prepared during Phase I. During Phase I, NCES/ED drafted and submitted a Clearance Package for the 1996-97 Field Test and the base year data collection. No package has been submitted for the first follow up data collection, and the contractor shall assume that a Clearance Package will be required for the base year data collection as well. NCES/ED will provide the contractor with a final copy of the Clearance Package that was approved in October, 1996 at the project initiation meeting (task 1).

Task 4. Select school sample, notify sampled schools and secure cooperation

4.1 Select kindergarten program sample

Within 3 months of the contract's start date, the contractor shall select a sample of about 1,000 schools with kindergarten programs. The sample shall be selected according to the specifications contained in the Final Study Design Report. Any revisions to the sampling plan developed during Phase I shall be brought to the attention of the NCES COTR and documented in writing. Implementation of these revisions shall be contingent on NCES/ED approval.

Three weeks prior to the sample being drawn, the contractor shall provide NCES/ED with the computer specifications that will be used to select the sample along with copies of other written materials (e.g., instructions, specifications) that document the derivation, justification, and implementation of the operations that lead to the selection of the kindergarten program sample, including 1) selection of PSUs; 2) sample allocation; and 3) stratification.

4.2 Secure State, district, and school cooperation

A high rate of participation among the schools in the sample, along with that of the children, parents, and staff sampled through these schools, must be achieved as a prerequisite for a successful study. NCES/ED's standards for high quality data require that nonresponse bias be minimal. The contractor shall contact personnel at the sampled schools to gain their cooperation and participation. Before contacting individual school administrators, the contractor shall obtain the endorsement and cooperation of key personnel in administrative units at all higher levels (i.e., district, state).

Manuals for securing state, district (and diocese), and school participation were prepared during Phase I along with several materials (e.g., letters to districts, study brochure, information sheets) to facilitate this process. NCES/ED will provide the contractor with final copies of these materials at the project initiation meeting (task 1). The approach to securing cooperation for the 1996-97 field test is described in the Study Design Report.

4.2.1 Prepare information packet. To gain the cooperation of State agencies, school districts, and schools, each shall be provided with information about the purpose and design of the study. The contractor shall design, develop, provide, and distribute informational materials. Information packets shall include: 1) a discussion of the importance of the study, its purposes, products, scheduled data collection, and sample; 2) provisions for maintaining anonymity of study participants; 3) the organizations and persons involved in the study; and 4) the benefits to be derived from the study.

Different versions of the information packet will be needed for State education officials, local education officials, school administrators, private school associations, school staff, and other interested groups. (A different version shall be prepared for the parents of sampled children as a part of securing their and their child's participation.) The content of the various information packets must be appropriate for each of these audiences.

The complete draft information packets shall be submitted to NCES/ED for review and approval within 2 months of the contract's start date. Once approved, the contractor shall reproduce the materials needed to obtain the cooperation and participation of states, districts, schools, teachers, and parents, plus a fifteen percent overage. All materials used in securing the cooperation of state, district, and schools (including teachers and parents) shall include the ECLS logo. Reproduction shall be accomplished within two weeks after the packet has been approved by NCES/ED. The contractor must be familiar with the requirements of the Government Printing Office for such reproduction; all reproduction shall be done in accordance with government printing regulations.

NCES/ED shall provide the contractor with electronic versions of the ECLS logo. NCES/ED shall also provide the contractor with all copies of the ECLS brochure. These shall be provided within two months of the contract's start date.

4.2.2 Obtain State cooperation. The contractor shall contact State education department personnel to share information about ECLS and to obtain their cooperation in the study. It is also important that the contractor work closely with the Council of Chief State School Officers' (CCSSO) Education Information Advisory Committee (EIAC). The General Statistics and Assessment Subcommittees of EIAC are responsible for monitoring the progress of ECLS. The contractor shall contact and obtain the cooperation of state education agency personnel in advance of contacting personnel in the school districts or schools.

4.2.3 Secure participation of districts. Following state-level approval, the contractor shall contact school district superintendents and personnel in private schools soliciting their support and requesting their participation in the study. The information packets sent to school district superintendents shall list the schools selected in their districts. The contractor shall confirm the district's participation via letter.

4.2.4 Secure participation of schools. The contractor shall contact all sampled schools solicitating their participation on a strictly voluntary basis and giving them full information about the study before asking them to participate. The contractor shall send Principals/headmasters of sampled schools a letter stating the purpose of the study and the voluntary nature of their school's participation. The contractor shall ask for written confirmation of a school's agreement to participate. The contractor shall follow up by telephone as needed to secure the participation of the school.

Task 5. Select student and teacher samples and secure cooperation

5.1 Select school sample of kindergarten children. After securing the cooperation of sampled schools, the contractor shall obtain the necessary information from the schools to enable it to select the sample of kindergartners within the school who will be asked to participate in the study. The selection of children shall follow the procedures contained in the Final Study Design Report, resulting in a sample of approximately 23,500 kindergartners. Any deviations from these procedures shall be documented and submitted to NCES/ED for approval.

Four weeks prior to the sample being drawn, the contractor shall provide NCES/ED with written materials (e.g., specifications , instructions) that document the derivation, justification, and implementation of the operations that lead to the selection of the sample of kindergartners within the sampled schools. These include, but are not limited to, the forming of student lists; within school sampling rates for core sample and subgroups that are oversampled; and random starts, if any; computer specifications for within school sample selection.

5.2 Identify, select and secure cooperation of teachers of children sampled. Once the sample of kindergarten children is selected from the student roster, the contractor shall identify and select the teachers of the sampled children. The contractor shall follow the procedures found in the Study Design Report, documenting any deviations or problems.

The contractor shall approach all teachers (and the parents of sampled children) on a strictly voluntary basis and give them full information about the study at the time that they are being asked to participate. The participation of teachers is critical to the success of the study given their role as a reporter about the child and the child's classroom environment.

5.3 Secure cooperation of the children's parents. Parents of sampled children will be sent a letter describing the purpose of the study, the voluntary nature of their sons' or daughters' participation, and asking their permission for their son or daughter to participate in the study. Parental consent forms (explicit and/or implicit), developed and tested during the field test, shall be used to record a parent's consent for their child's participation. The contractor shall work with the school in securing the consent of parents. About 53 percent of field test schools required explicit consent. Public schools were more likely to require explicit versus implicit consent (55% versus 45%, respectively), and private schools were about evenly split between those requiring explicit versus implicit parental consent.

Task 6. Receipt control and reporting

The design of the ECLS requires the participation of large numbers of schools, children, teachers, and parents. The design requires that large amounts of information be collected, processed, and stored in a short period of time. A system must be developed and maintained to keep control of this information and the data collection activities associated with the ECLS. The contractor shall design, implement, and document a system to ensure that each designated sample unit (child, parent, school administrator, and teacher) is properly surveyed and that all required information is obtained, properly identified, and stored. The system shall have means for correcting materials/data found to be incomplete or inaccurate. The system shall permit project staff to monitor the flow of information and to produce weekly reports of the study's progress. Such a system is referred to here as receipt control.

6.1 Status reports. The receipt control system shall support a range of project reporting activities. The contractor shall produce weekly summary status reports on the progress of data collection activities for the NCES/ED COTR using data from the receipt control system.

The receipt control system shall be used to locate any particular instrument or respondent to determine the status of the instrument or of the respondent's participation.

Weekly summary status reports produced through the receipt control system shall contain detailed information on the initial and final status of all subjects sampled for the study. At a minimum, they shall include 1) the number of cases (schools, school administrators, children, parents, and teachers) sampled; 2) the number of cases for which contact has been attempted and established; 3) the number of cases that have been completed, that have refused to participate, that are pending, or that are incomplete; and 4) cumulative statistics on response rates and followup actions.

6.2 Responsiveness to special requests. The receipt control system shall have the capability to respond to requests for information on the status of the survey other than those that are a part of the weekly status report. At a minimum, the system shall have the capability to provide information separately on the completion status of the schools, school administrators, students, teachers, and parents for the classifications used to define the sample strata for the study.

6.3 Receipt control system specifications. At least two months prior to the start of data collection, the contractor shall have in place and operational (at their own facilities) a computer-based receipt control system to monitor the flow of data collection activities from individuals and from educational institutions. The control system data files shall contain the identification numbers of all sample members and identification codes of all schools attended by the ECLS students, as well as information about various data collection events that will be necessary for monitoring the collection of data.

Prior to beginning work on this system, the contractor shall submit its plans for receipt control to NCES/ED for review and approval. These plans shall be submitted no later than 9 months following the contract's start date.

Task 7. Tracking of kindergarten sample

One of the challenges facing the ECLS is limiting the negative impacts of the mobility of the sample of kindergartners. To help maintain a high response rate, the contractor shall develop methods and procedures for collecting locating information on each ECLS child/parent sample member. The contractor shall create and maintain a tracking system data base. At a minimum, the data base system shall include the following: 1) student identification number, 2) parent(s) names, 3) home address, and 4) names, addresses, and phone numbers of two other individuals who would always know the whereabouts of the child.

The contractor shall also have a set of proven procedures for locating sample members who change schools/addresses between data collections (e.g., fall kindergarten to spring kindergarten). The contractor shall locate all child/parent sample members who change schools/addresses between data collections using these procedures and the tracking system data base.

The contractor shall submit its specifications for the tracking system to the NCES COTR within six months of the contract's start date.

Task 8. Conduct pilot test

The contractor shall conduct a pilot test of the ECLS in the spring of 1998. This test shall be used to test and evaluate the contractor's technical approach to: 1) securing State, district, school, parent, and teacher cooperation; 2) sampling schools, children, and teachers; 3) conducting child assessments in schools; 4) training field staff; 5) managing the flow of materials and data and reporting on the progress of data collection; and 6) testing the use of any technologies that will be used in schools (e.g., CAPI sampling and assessment programs) during the base year study and for capturing and processing data.

The contractor shall include no more than 12 schools in the pilot and shall assess between 240-300 children. Twelve administrator questionnaires shall be administered as well as the appropriate number of teacher questionnaires and teacher ratings. Schools shall be selected from 4 geographic sites, one of which shall be the Washington, DC metropolitan or surrounding area. The contractor shall follow as closely as possible the design specifications of the ECLS when conducting the pilot.

The contractor shall prepare a 10-20 page report that describes the pilot study, what was learned from this activity, and identifies any recommended changes to the contractor's technical approach based on the pilot experience. This report shall be submitted within 6 weeks of the end of pilot data collection and field operations.

Task 9. Reproduction of instruments and acquisition of materials

The contractor shall reproduce or acquire all the materials, and program all the systems necessary, to support the collection of data from children, school administrators, teachers, parents, and school records. This shall include, but not be limited to the school administrator questionnaire, the teacher questionnaire, the parent questionnaire, school record retrieval, and the ECLS direct and indirect assessment instruments.

9.1 Reproduction of instruments. The contractor shall reproduce or arrange for the reproduction of the school administrator questionnaire, teacher questionnaire, teacher checklist, and school record retrieval forms. The contractor shall also have responsibility for reproducing all printed materials associated with the ECLS assessment battery (both the direct and indirect instruments). A sufficient number of copies shall be reproduced to meet the needs of each data collection, plus a 10 percent overage. The contractor must be familiar with the requirements of the Government Printing Office for such reproduction; all reproduction shall be done in accordance with government printing regulations.

The materials used in the ECLS assessment battery that require reproduction are: the assessment easels for the three subtests of the direct cognitive battery (language/literacy, mathematics, and general knowledge), instructions for conducting the psychomotor assessment battery, the scenarios found in the Dodge direct socioemotional measure, teacher ratings of children's social skills and cognitive skills. The assessment easels shall be reproduced in color. All other printed materials shall be reproduced in black and white.

The complete ECLS assessment battery for children includes

•	Cognitive assessment: 3 sets of test plates spiral bound into easels: one for each of the cognitive domains. Each easel shall be approximately 8½ inches wide by 7 inches high. The covers of the easel shall be constructed from a medium of sufficient durability to withstand multiple use. The back cover of the easel shall be constructed so that it shall support the test plates in an upright position during administration. On the front cover shall be printed the ECLS logo across the top margin and the subtest domain in the center. The easel shall include the first stage routing test and the three alternative second stage test items. Each test will be separated by tab markers. The test items shall be printed using 4 color process on 80 # white o/s smooth stock. The notice of copyrighted items shall be printed on the first fly sheet. Each test item shall be printed on separate fly sheets.

•	Practice booklet: 1 set of test plates bound into an easel containing a sample of items representing the different types of responses required in the assessment such as pointing and giving an oral or spoken answer.

•	Laptop computer pre-loaded with 1) a software package for recording assessment battery responses and 2) a communications package for transmitting completed cases to the contractor's central office computers. Battery packs and extension cords with surge protectors are required.

•	Socioemotional assessment: 1 set of test plates of the hypothetical situations printed on 8"x10" spiral bound paper with separate versions designed for girls and boys.

•	Manipulatives:

- 20 Unifix cubes or equivalent, all of the same color.

- 10 1"x1" cubes, all of the same color

- primary pencil and a sheet of 8½x11" unlined white paper

- 5 shapes printed on 3x5" card stock

- 3x5" card on which a graphic illustration of a figure made of squares to represent the cubes is printed

- a two-part box 7 ½ x 5 x 2 3/4" rigid paper board box; box top is die cut on one short end with crescent shaped 2" radius cut.

•	Student rating booklets: Separate student rating booklets for the measures of children's socioemotional and cognitive behaviors and skills shall be printed on 8½"x 11" paper. All pertinent information including ECLS logo, student and teacher identification, OMB approval, instructions, and copyright notice when appropriate shall be printed on the front cover. Text shall be printed on both sides.

9.2 Acquisition of materials and equipment. The contractor shall be responsible for acquiring all materials and supplying all equipment needed to administer the ECLS data collection instruments.

9.3 CATI/CAPI Programming. Three components of the ECLS design require the use of laptop and CATI/CAPI technologies: 1) within school sampling of children; 2) parent interview; and 3) the direct child assessment. The use of these technologies increases the quality of the samples and the data that are collected from these samples, and helps the ECLS achieve its goal of collecting baseline data early in the school year from children, their teachers, and parents.

The design of the ECLS requires that children be sampled within schools with different probabilities of selection, and that their parents and teachers be identified and sampled quickly for the parent interview and teacher survey. In the spring 1996 pilot and again in the 1996-97 field test, laptop computers were used by field staff to assist in the selection of these samples. NCES/ED intends to continue to use this technology in this way in Phase II.

Parent interviews will be conducted by telephone using CATI. The exceptions to this are the parents of ECLS children who do not have telephones. For these parents, interviews will be conducted in their homes using CAPI.

The ECLS direct assessment battery uses a CAPI-assisted approach. Child assessors use standard paper easels and other printed materials when asking children a range of questions pertaining to language/literacy, mathematics, and general knowledge in science and social studies. However, they are assisted by a laptop computer that has the entire battery programmed. Assessors read from the screen while showing the child stimulus (and response) pages on the easel or manipulating other materials. All of the children's responses are entered into the laptop program. The contractor shall continue to use this technology and approach when administering the ECLS assessment battery in the base year and first followup studies. The use of this technology will be even more critical in the main study given the use of a two-staged assessment battery.

Because of the complexity of the CATI and CAPI programming of the parent interviews and child assessments, the contractor shall develop the procedures and materials necessary to insure that all program specifications are correct and complete. In the development stage it may be helpful to create flowcharts and evaluation procedures for both the CATI and CAPI instruments.

The contractor shall prepare hard-copy versions of the CATI and CAPI monitor screens for each of the parent interview and child assessment items. At a minimum, these screens shall include:

(1)	the item number and text;

(2)	all displays that are to appear in the body of an item (e.g., names, wording alternatives);

(3)	all information that is being provided to the interviewers/assessors to assist them in conducting the interviews/assessments accurately and smoothly (e.g., respondent's name);

(4)	information that identifies the next item to be asked for each response; and,

(5)	the response range specifications.

Following review by the contractor's project staff, copies of the CATI and CAPI screens shall be submitted to NCES/ED for review no later than 17 weeks prior to the start of data collection. NCES/ED will provide comments within two weeks, and screens revised in response to comments shall be submitted with two weeks of receipt of NCES' comments.

Task 10. Hiring and training data collection staff

Several different staffs are required to implement the ECLS design. Field staff are needed to conduct preliminary visits to the sampled schools to select the children who will become a part of the ECLS longitudinal sample and to administer the parent interview when no telephone is available. Child assessors are needed to administer the ECLS assessment battery to children in their schools. Telephone interviewers are needed to conduct the parent interview, and to perhaps follow up with teachers and administrators who fail to respond by mail (see task 11). Each of these activities requires a unique set of skills, skills that are developed through a combination of prior experiences working on other similar projects and training tailored to the requirements of the ECLS.

10.1 Development of training and evaluation materials. In order to ensure that the data collection staff assigned to the ECLS have the skill levels necessary to perform successfully the demanding tasks associated with conducting preassessment visits to schools, child assessments, and parent interviews, the contractor shall design and implement comprehensive training programs taking into account the procedures and materials developed for training staff for the field test.

Prior to fully developing any training materials, the contractor shall submit an outline of its interviewer training program to NCES/ED for review. This outline shall be submitted to NCES/ED no later than 5 months after the contract's start date. This outline shall include:

(1)	a training program agenda that identifies the formats that will be used to conduct the training, the topics to be covered (e.g., study background, sampling children, overview of instruments, survey topical component), and the length of the training;

(2)	an outline of the materials that trainees will be provided;

(3)	the contractor's plan for evaluating the training program and the performance of individual trainees during training; and,

(4)	a preliminary schedule that identifies when and where the training will occur.

Following NCES/ED review and approval of the interviewer training program outlines, the contractor shall develop and submit draft copies of all training materials to NCES/ED for review at least five (5) weeks prior to the start of the first training session and six (6) weeks prior to the start of data collection. After making the changes brought up by the review, the contractor shall submit 5 copies of the final training materials to the NCES/ED COTR at least five (5) working days prior to their being distributed to trainees.

Along with the training plan, the contractor shall submit a plan which shows how it will monitor all data collection activities to assure that consistent high quality data is collected throughout the data collection period. This plan shall include:

(1)	the contractor's awareness of interviewer responsibilities under the Privacy Act of 1974 (5 U.S.C. 552a), the Privacy Act Regulations (34 CFR Part 5b), the National Center for Education Statistics Act of 1994 (P.L. 103-382, Section 9007), and NCES/ED Standards and Policies; and,

(2)	a description of the evaluation and correction methods the contractor shall use immediately after training and over time to a) assure quality control, b) maintain standards, c) identify poor performance, and d) either correct such performance or release poor performing individuals so that only high quality data collection personnel are retained and utilized.

(3)	a description of the types of technologies (e.g., video taping) that will be used to complete at least some of these evaluations.

10.2 Recruitment of data collection staff. The contractor shall recruit and hire staff to complete the data collection within the time constraints imposed by the project schedule. Because data will be collected from large numbers of young children, the contractor shall give special consideration to recruiting field staff with prior experience working with young children.

Moreover, because Hispanic and other minority children will be oversampled in the ECLS, the contractor shall have data collection staff who are qualified to interact with children and conduct telephone interviews in languages other than English. At a minimum, the contractor shall have field staff and telephone interviewers who are fluent in Spanish.

Confidentiality requirements in NCES/ED contracts, discussed in more detail in Section IV, mandate that data collectors complete a sworn Affidavit of Nondisclosure (see Appendix D). In the presence of a public notary, these affidavits shall be signed by all project staff, not just field staff and telephone interviewers, having any access to ECLS raw data.

10.3 Conducting data collection staff training. The contractor shall have responsibility for all training of field staff and telephone interviewers. All training shall take place at the contractors' facilities or at facilities arranged for by the contractor. The contractor shall be responsible for assuring that all staff assigned to the project successfully complete the training program as specified in the training plan. The contractor shall have responsibility for the production of all training materials and shall have sufficient materials available for all staff trained during the conduct of the project.

Moreover, successful conduct of the ECLS requires that all data collection staff receive standardized training. Due to the large numbers of individuals that will need to be trained, training may have to occur across multiple sites. The training program shall use video tapes and other technological capabilities, as needed, to insure that all staff receive the same training in their respective assignments.

10.4 Conducting supplemental training. Because of the complex nature of the ECLS, the contractor shall develop and implement procedures for conducting supplemental training as problems are encountered. Because of the large number of staff who will be working on the ECLS during the data collection period, it is important that any decisions that are made about the conduct of the survey be disseminated immediately and clearly to all staff. The contractor shall have procedures in place that will ensure that all parties involved with the collection of data use the same solutions to and interpretations of problems that arise during the course of conducting the study.

Task 11. Collect kindergarten and first grade data

Data collection for the ECLS base year shall occur twice, at the beginning and near the end of the school year. First follow up data shall be collected during the spring of first grade. The contractor shall conduct the base-year study and first follow up study implementing those procedures and using those instruments set forth in the ECLS Final Design Report as modified based on the outcome of task 1. Base-year and first follow up data shall be collected from the administrator of the school attended by the sampled kindergartners, their teachers, and parents. Direct assessments of children's cognitive and social development (A final decision on using a direct measure of children's social development will be based on the outcomes of the 1996-97 field test.) shall be administered individually in the schools attended by ECLS children by staff employed and trained by the contractor.

11.1 Data collection schedule. The contractor shall prepare, using critical dates specified by NCES/ED (see A.5 Data Collection Schedule ), a detailed schedule for completing all data collection associated with the ECLS base year and first follow up studies. The contractor shall pay special attention to the possible effects that the timing of data collection may have on the key study measures, and especially on various child development measures. A copy of the contractor's data collection schedule shall be submitted to NCES/ED for review and approval within six months of the contract's start date. No data collection shall begin prior to NCES/ED's approval of the schedule.

11.2 Quality control procedures. The contractor shall develop and implement a set of quality control procedures that shall assure the collection of high quality data throughout the data collection period. Quality control shall be maintained through a series of both statistical and project supervisory procedures. The contractor shall have in place procedures for identifying and correcting errors resulting from both the design and poor implementation of the design. Problems that are identified shall be addressed immediately and consistently. Procedures shall be in place from the beginning of training until the completion of data collection that will require that all parties involved with the collection of data use the same solutions to and interpretations of problems that arise during the course of the survey.

The contractor's approach to data quality may include the conduct of experiments or special studies. The contractor shall design and conduct no more than one or two experiments or special studies that would significantly enhance our knowledge of the effects of different design features on data quality and the validity and reliability of the responses given by study participants.

11.3 Progress reports. Throughout the data collection period, the contractor shall provide the NCES/ED COTR with weekly progress reports. In addition to the computer- generated reports described under Task 7, these weekly reports shall identify any problems encountered and either describe how these problems were resolved or recommend alternative ways of resolving these problems. These weekly progress reports are in addition to the monthly reporting requirements described in Section IV (Study Requirements, A. Reporting).

Any problems encountered that either have consequences for the project's budget or time schedule shall be brought to the attention of the NCES/ED COTR in these weekly progress summaries.

11.4 Nonresponse followup and refusal conversion. The ECLS, like any other study, will encounter initial refusals and other types of nonresponse. The potential for nonresponse and its effects on estimates for child, school, teacher, and parent characteristics shall be given special consideration in the contractors' conduct of the ECLS. The contractor shall develop and implement procedures for converting nonrespondents to completed interviews.

For longitudinal surveys such as the ECLS, NCES/ED has established a target response rate of 90 percent for each wave of data collection. The contractor shall develop and implement methods and procedures that shall increase the chances of meeting and possibly exceeding this rate.

All nonresponse followup and refusal conversion efforts undertaken by the contractor shall be documented. The contractor shall provide NCES/ED with the decision rules to be followed if a sample case (i.e., institution or individual) refuses to participate. Any procedures for sample replacement shall be submitted to NCES/ED for approval prior to their implementation.

11.5 Control of instruments and other data. The ECLS cognitive assessment battery was constructed by creating a pool of items from existing instruments and developing new items. NCES/ED and the contractor for Phase I of the ECLS have secured permissions from various test publishers (i.e., licensors) to use individual items under a "Conditions of Use" negotiated with each publisher (see Attachment X). One of the conditions requires that all items made available to NCES/ED by each publisher will be protected at all times. Consequently, the contractor shall protect the security of the cognitive assessment battery by restricting the number of copies published to the number needed to conduct the study and maintaining a record of the location of each copy of the assessment battery instruments. A copy of this record shall be made available to the COTR upon request. Under no circumstances shall copies of the assessment battery instruments be distributed or loaned out without the permission of the NCES/ED COTR. The contractor shall employ strict procedures for ensuring that all project staff adhere to this requirement.

Another Condition of Use negotiated with the licensors charges NCES/ED to honor all requests by the licensor to protect the licensor's proprietary rights to items at all times by publishing the items marked by a proper notice of copyright. The contractor shall include proper notice of copyright for all items from existing instruments in publishing the cognitive assessment battery instruments (see task 9).

The contractor shall not include item-level data in preparing public use data files. Reports shall only contain normative and criterion-referenced status and gain scores for ECLS participants using IRT-based formulas. Individual ECLS assessment battery data and individual item data shall not be released without the permission of NCES/ED.

The contractor shall submit annual statements of use to test publishers who are receiving royalties for the use of their items in the ECLS assessment battery. The contractor shall pay the royalties associated with this level of use as previously agreed to. This shall be done no later than January 31 of each year. The contractor shall provide NCES/ED with a copy of each statement of use and royalty payment.

Task 12. Data processing

The contractor shall have responsibility for processing all data collected for the ECLS. This shall include editing (both manual and machine) study data, correcting data that fail the edit checks, coding of open-ended responses, and converting the raw survey data to a computerized form.

12.1 Coding and editing. At least ten (10) weeks prior to the start of data collection, the contractor shall submit its plan for editing and coding the data collected as a part of the ECLS. The contractor's plan shall be submitted as a memorandum to the NCES COTR and shall clearly identify 1) the type of edits that will be performed for the different items of data, 2) the edit specifications for each data item, 3) the response that will follow failed edit checks, and 4) the approach that will be used to code the ECLS data, including coding specifications for open-ended items. Its plan shall include edit specifications for CADE, CAPI, and CATI. Approval of the plan by NCES/ED is required prior to its implementation.

Following NCES/ED approval of the plan, the contractor shall develop and test the methods and procedures needed to carry out the plan. Responses to question items that fail edit checks and require followup shall be resolved promptly according to approved procedures.

The contractor shall maintain a software system that can be used to update, recode, correct, or temporarily modify data values, and to modify edit specifications. The system shall be capable of providing the hard copy printout necessary for any respondent call-back.

Editing of the data shall be done by machine, but limited manual scan edits to check for completeness may be required. Coding of responses to all open-ended questions shall be performed by the contractor (about 5-10 open-ended items per survey form).

12.2 Data entry/conversion. The contractor shall submit a plan for converting raw data into machine readable format by no later than six weeks before the start of data collection. The plan shall describe the proposed structure and specifications for these data files for both draft and final deliverables.

Prior to the base year data collection, the contractor shall design, establish, and implement the procedures to transform the data into a computerized form that can be used by a variety of standard statistical software packages (e.g., SAS and SPSS for Windows). After data collection begins, the computerized data shall be edited and errors corrected until the computer files contain only verified and accurate records. For NCES review, the contractor shall provide separate ASCII data files with SPSS for Window code on CD-ROMs. These files and all documentation (including linking IDs) needed to access and read the data stored in each file shall be delivered within four (4) months of the end of data collection activities. The following files shall be delivered:

Base year files

1. Child file

2. Principal/School file

3. Parent file

4. Teacher file

First followup files

1. Child file (including student record data)

2. Principal/School file

3. Parent file

4. Teacher file

Final data deliverables shall consist of CD-ROMs with electronic codebooks that allow users to tag variables and create SAS and SPSS for Windows code files. For the final file deliverables, the data shall be provided as megafiles (e.g., merged child, parent, school, and teacher files) similar to those contained on the 1996 release of the NELS:88 base year through third followup release files. The plan shall not be implemented without the approval of NCES/ED. For this contract, final file deliverables include:

•	Merged fall and spring base year kindergarten files -- These merged files shall be delivered within eight (8) months of the close of spring kindergarten data collection activities.

•	Merged base year and first followup files -- These merged files shall be delivered within eight (8) months of the close of spring first grade data collection activities.

•	Each of these merged megafiles will be accompanied by user's manuals that are similar to those produced for the second follow up of NELS:88 and the 1993 and 1995 National Household Education Survey.

12.3 Quality control. The contractor shall develop and implement a set of quality control procedures that assures high quality data processing.

12.4 Data processing status report. The contractor shall allow for NCES/ED inspection of the records and results of the data editing and follow-up actions. These records shall be delivered to NCES/ED on request. A monthly status report of the data editing and corrective actions shall be provided to NCES/ED. These status reports shall include weekly and cumulative summary statistics by instrument on the volume of work completed; edit rejection rates for various types of items and respondents; and the status and method of corrective action taken.

12.5 Documentation. All data processing activities and procedures shall be clearly documented. This documentation shall become part of the final ECLS methodology report (task 14).

12.6 Base-year and first follow up documents and materials. To ensure a smooth transition from the base year and first follow up data collections to the later followup data collections, the contractor shall coordinate with the second followup contractor and provide copies of all documents and materials developed for the base year and first follow up of ECLS. These shall include, but not be limited to hard and electronic copies of instruments and other project deliverables, and data on CD-ROMs accompanied by appropriate documentation.

12.7 Public and proprietary data files. For purposes of disseminating the ECLS data, the contractor shall prepare and deliver to NCES/ED both public and proprietary data files on CD-ROMs with electronic codebooks. The construction of these files will be modeled after those in NELS:88 second follow up and the 1993 and 1995 National Household Education Survey.

The contractor shall conduct a thorough confidentiality analysis of the ECLS data when preparing the public use files. This analysis shall ensure that the confidentiality provisions contained in PL 100-297 are fully complied with. To protect the privacy of respondents as required by PL 100-297, respondents with high disclosure-risk will be identified. Data that can potentially be used to identified these respondents will be masked.

There are many different strategies that can be used to identify high disclosure-risk respondents. For a multi-stage school-based survey like ECLS, one strategy is to identify kindergartens that are considered to have high disclosure-risk. By matching data available on school universe files (e.g., CCD, PSS, QED ) with data on the ECLS school sample file, the distance between a school as it appears in the universe file and the same school as it appears in the sample file can be calculated. Based on these distance measures, sample schools with small distances are considered as having high disclosure-risk and are identified. School level data that can be used to identify these schools are masked. This method of identifying high disclosure-risk schools has been applied successfully to other NCES longitudinal studies (e.g., NELS:88).

Within the high disclosure-risk school, some individual-level variables (e.g., student, teacher) in the sample data file may provide information that can be used to identify individual respondents. These variables and the individuals with high risks of being disclosed shall be identified. Their responses shall be masked.

In addition, disclosure-risk can be increased as more information is collected during future rounds of data collection and as this information is added to the data records of the sample members. The contractor shall also consider this type of disclosure risk when planning and conducting confidentiality analysis for the ECLS.

Task 13. Briefing materials.

Throughout the course of the project, NCES/ED shall provide interested individuals and agencies with information about the nature, findings, and progress of the ECLS. The contractor shall support these activities by developing up to six sets of briefing materials over the life of the contract. These materials shall be prepared only for NCES/ED's use. Each set of briefing materials shall be no more than 10 typewritten pages (double-spaced) with no more than 10 accompanying graphs and/or tables. Requested materials may include overhead slides (transparencies or Power point files) for making presentations regarding the ECLS. Within two weeks of a request, a set of briefing materials shall be submitted to NCES/ED for review and comment. The contractor shall make necessary revisions to the materials based on this review and resubmit the materials within one week of receipt of comments.

Task 14. Methodology report

The contractor shall submit a methodology report that documents the entire project including: a description of the sample design, instrument development with particular attention to the ECLS assessment battery and data collection procedures. Because the ECLS represents an important NCES/ED initiative that is expected to receive much attention and scrutiny, it is critical that this technical report describe the study design and all procedures used to conduct the study. Special features of the study shall be described in detail and any problems encountered either during the design or implementation of the survey identified. Changes that were made to the design and/or procedures developed during Phase I in response to unanticipated problems shall be fully documented. The report shall include major findings pertaining to the methods used and/or experimented with during the conduct of the study and discuss their implications for future follow up data collection activities. Information on the design and findings of methodological studies/experiments directed toward assessing and improving the quality of the ECLS data shall be incorporated into the methodology report. The design and findings of these studies/experiments shall be presented in a manner that will allow them to be used as stand alone documents.

Much of the information that is presented in this report will be drawn from the Phase I Final Study Design Report. The task here is to update that report.

The final methodology report shall provide an in-depth description of the rationale, development, and psychometric properties of the ECLS assessment battery. A separate section of this report shall address whether the psychometric objectives of the assessment battery were met or exceeded. Using the Phase I Final Study Design Report as a starting point, the contractor shall document and expand on the following assessment topics:

Assessment Specifications

Aims and objectives of ECLS assessment

Description of the two stage testing methodology

Description of the assessment development process

Difficulty levels of items

IRT scaling for longitudinal measurement

Psychometric goals of the ECLS assessment battery

Specifications for individual assessment instruments

Matching test content to curriculum

Psychometric Analysis results

Reliability of the ECLS assessment battery

Item statistics by gender and racial/ethnic groups

Differential Item Functioning (DIF) for race-ethnicity and gender

Factor structure of the ECLS assessment battery

Performance of racial/ethnic and gender groups on the ECLS assessment battery

Proficiency level subscores by subgroups

Item Response Theory (IRT) parameters for the ECLS assessment battery

Assessment scores on both public and privileged released CD-ROMs

The contractor shall submit a draft of this report within 10 months of the end of the spring first grade data collection. The contractor shall revise the draft report based on the comments and suggestions of NCES/ED staff. A revised report shall be submitted to NCES/ED. The revised version will be extensively reviewed within the Department of Education. A final methodology report which incorporates the comments and suggestions made by NCES/ED and others within the Department of Education shall be prepared and submitted. Once NCES/ED has approved the final report, the contractor shall submit 20 copies of the report to NCES/ED. In addition to hard copies, the final methodology report shall be delivered on floppy diskette. The electronic version of the report shall be delivered to NCES/ED as a Word 7.0 or later file. No information contained in this report shall be released without NCES/ED's prior approval.

Optional Tasks, Special Studies, and Supplements

In addition to the core tasks (tasks 1 through 14), this procurement includes:

1) 3 optional design tasks (tasks A through C);

2) 4 optional special studies; and

3) 2 optional supplements.

C. Optional DesignTasks to be Performed

Design activities during Phase I were devoted to 1) developing the overall design parameters of the ECLS (kindergarten through grade 5); 2) developing the kindergarten sample design and procedures for freshening the sample at first grade; and 3) creating the specific instruments (questionnaires and assessments) and procedures that would be required for the first two data collection years, i.e., kindergarten, both fall and spring, and first grade, spring only. In addition, many of the instruments that will be needed if a second grade spring data collection is once again included in the ECLS design were developed during Phase I.

Before NCES/ED can field the later rounds of data collection, additional design work is needed on the designs for second, third, fourth, and fifth grades, extending and building upon the work that was begun in Phase I. This design work will be aimed at developing 1) a final sampling plan for later rounds of the study; 2) an ECLS assessment battery that extends beyond grade one, and reflects the content and process emphases of the ECLS assessment framework for later years; and 3) school, teacher, and parent questionnaires for later elementary school grades, and student questionnaires beginning with third grade.

NCES/ED has identified three (3) optional tasks that support this design activity. All work required under each of these tasks must be completed within 18 months of award of the option. Completing these tasks on schedule is critical, in order for them to be field tested one year prior to their full-scale implementation. Large-scale field testing of the instruments and procedures is not a part of this procurement; however, smaller pilot studies and cognitive research is expected.

Optional Task A. Develop instruments and procedures for the second through fifth grades

Extensive conceptual work has gone in to designing the survey instruments and data collection procedures for kindergarten and first grade (and second grade to a certain extent). If the option to design the survey instruments for the second through fifth grades is exercised, the contractor shall extend this work and develop survey instruments for students, parents, teachers, and school administrators for the second through fifth followups (grades two through five). This task pertains to the development of all instruments other than those comprising the ECLS assessment battery.

A.1 Identify key research and policy issues. The contractor shall review the research and policy issues previously identified for the kindergarten and first grade years (see ECLS Study Design Report). Based on this review, the contractor shall submit a memorandum which identifies both the issues that extend through to the fifth grade as well as any new research and policy issues which were not previously included but are relevant to any or all of grades two through five. Moreover, in the memorandum, the contractor shall prioritize the issues and questions to be addressed by the ECLS and clearly state the types of analyses that the ECLS will be able to support.

A.2 Review content outline. The contractor shall review the content coverage of the ECLS (see ECLS Study Design Report). Based on this review, the contractor shall provide in a memorandum any updates or modifications needed for the second through fifth grades.

A.3 Develop survey instruments for the second through fifth grades. The contractor shall develop all survey instruments and data collection forms needed for the second through fifth grades.

The survey instruments shall contain items that will lead to accurate and reliable measurement of the topics listed in the ECLS Study Design Report and extend the information collected in the kindergarten base year and first grade followup. In designing the survey instruments, the contractor shall take into account the comments and suggestions made by TRP members and NCES/ED regarding the content coverage of the instruments. When the ECLS includes topics and/or sets of items that have been previously fielded through other early childhood studies or when the survey is designed to build upon the data collection efforts of other agencies, special consideration shall be given to maintaining the comparability of items across surveys.

Draft survey instruments (e.g., questionnaires and any forms needed to support school record data acquisition) shall be submitted to NCES/ED for review and comment. Following the contractor's receipt of NCES/ED's review comments, the contractor shall revise the instruments responding to the comments and suggestions of NCES/ED, and submit a second set of revised instruments. With the approval of the COTR, the contractor shall submit this version of survey instruments to cognitive testing.

Following the cognitive laboratory research, the contractor shall submit a third draft of the instruments.

A.4 Conduct cognitive laboratory research. It is important that instruments and items developed for the ECLS are both valid and sensitive measures of a child's educationally relevant experiences. In support of these objectives, the contractor shall design and implement research directed toward understanding the extent to which questionnaire items are understood by respondents and how the answers they give might be interpreted. This research shall seek to identify any areas of ambiguity and confusion.

The contractor shall conduct cognitive laboratory research on each data collection instrument. Prior to conducting any cognitive laboratory research, the contractor shall develop and submit a memorandum to the COTR that outlines its plans for conducting such research. The plan shall describe the contractor's general approach to this work and identify the specific methods and procedures that shall be used. Justification shall be provided for the selection of certain methods over others relative to the objectives of the research. The characteristics of individuals and/or groups that will be recruited for participation in this research shall be identified and justified in light of the research objectives. The plan shall describe how the outcomes of the research will be analyzed and used to improve the overall quality of the ECLS instruments. Implementation of the plan shall be contingent upon NCES/ED approval.

After implementing the cognitive research lab activities, the contractor shall analyze results and prepare and submit a memorandum that summarizes the methodology and findings of the ECLS cognitive laboratory research. The memorandum shall include recommendations for modifying the instruments and procedures when indicated by the findings.

Optional Task B. Extend the EC LS Assessment Battery

Extensive work has been completed to design and develop the assessment battery of the ECLS during Phase I with the creation of frameworks for each of the domains of child development (i.e., cognitive, socioemotional, and physical and psychomotor) and both direct and indirect assessment instruments for kindergarten and first grade. Each framework lays out the key elements of the individual domains which are fundamental and predictive of academic achievement. The creation of each of the kindergarten and first grade assessment instruments was guided by these assessment frameworks.

All kindergarten and first grade cognitive assessment battery instruments as well as instruments for measuring the socioemotional dimension in grades k-2 were completed during Phase I. While cognitive assessment battery items have been developed for second grade, instruments for group administration have yet to be developed. An optional task during Phase II of the ECLS includes a) completing the group administered cognitive assessment battery instruments for grade two, and b) the extension of work on the assessment framework, development of items and cognitive and socioemotional assessment battery instruments for grades three to five. If the option to complete the development of the cognitive assessment battery instruments for the second grade and all assessment instruments for the third through fifth grades is exercised, the contractor shall review the work completed in Phase I and extend both the direct and indirect assessment battery instruments for the second through fifth followups (grades two through five). The contractor shall with concurrence from the NCES COTR convene a panel of curriculum specialists to review and comment on the contractor's work. The panel members shall have expertise in the relevant assessment domains and specific age and grade groups (i.e., ages seven to eleven and second to fifth grade). This group shall meet up to two times in Washington, D.C. and shall have not more than 12 members.

B.1 Extend the assessment battery to fifth grade. The contractor shall review the ECLS cognitive assessment frameworks, pool of existing items, existing kindergarten and first grade instruments, and documentation of item statistics from the 1996-97 field test. In addition, the contractor shall include the extensive work completed by the National Assessment of Educational Progress (NAEP) in developing NAEP frameworks, and relevant published curricular standards in its review. The contractor shall also review the ECLS framework for measuring socioemotional development, existing direct and indirect measures and relevant item statistics. Based on these reviews, the contractor shall provide in a memorandum any additions or modifications needed to create the direct and indirect assessment instruments for grades two through five. The contractor shall include any new domain elements or items which were not previously included but are related to the study issues and have been shown to be predictive of children's school performance in these later years in the memorandum. Furthermore, in the memorandum, the contractor shall prioritize the key elements according to their relationships to the empirical evidence establishing the importance of the element in each domain and the research questions to be addressed by the ECLS.

B.2 Operationalize new elements in each domain. After reviewing the documents pertaining to the development of the ECLS assessment battery in Phase I, the contractor shall provide in a memorandum definitions of each new element and its relationship to each of the identified dimensions of learning (i.e., cognitive and socioemotional, and physical) for grades two to five.

B.3 Extend the item pool for each element in all assessment domains. The pool of items for measuring each of the existing and new elements shall be of sufficient number to reliably measure growth across the grades. The contractor shall submit a memorandum describing the approach to be taken to expand the item pool to the COTR for approval before undertaking any work. With the approval of the COTR, the contractor shall build upon Phase I work to extend the item pool from second grade to fifth grade. Attention shall be given to maintaining the longitudinal nature of the ECLS assessment battery in extending the item pool.

Draft items shall be submitted to NCES/ED for review and comment. Following the contractor's receipt of comments and review from NCES/ED, the contractor shall respond to the comments and suggestions made by NCES/ED by revising the set of items. With the approval of the COTR, the contractor shall submit these versions of the items to curriculum and child assessment specialists for review. Copies of comments and suggestions from reviewers shall be submitted to NCES/ED. Following contractor's receipt of the COTR's approval, the contractor shall revise the items responding to reviewer comments and suggestions, and then submit a revised set of items.

B.4 Adapt items for group administration. In developing the cognitive battery assessment instruments for group administration, the contractor shall first identify the appropriate items that will form the set of common "anchor" or linking items between the preceding and current grade assessment instruments (e.g., first and second grade) and the sets of items for the second stage forms. The first stage form shall contain items that will reliably identify which second stage form will be administered. It shall also be relatively easy and fast to score. The second stage forms shall include a sufficient number of items shared in common with the first grade pool and items that are sufficiently distributed at both the upper and lower levels of difficulty. Secondly, the contractor shall identify the items suitable for adapting to group administration. The contractor shall submit the collection of items along with a memorandum delineating the approach used and justification for item selection to NCES/ED, child assessment specialists, curriculum specialists, and teachers for review and comment. Following receipt of comments from these groups, the contractor shall respond to the comments and suggestions and submit a second set of revised items. With approval of the COTR, the contractor shall submit a draft of the first and second stage forms of the instruments.

The kindergarten through second grade cognitive item pool created during Phase I includes items from several published assessment instruments and items created for the ECLS. Agreements from licensors have been obtained to copy, adapt, and use specific items in the kindergarten and first grade assessment batteries. Permission to use any of these items in the later grade batteries or any new items is needed before use. The contractor shall consult with the COTR before pursuing any permissions.

B.5 Conduct small-scale pilot. The validity and sensitivity of any items and measures of children's cognitive and socioemotional status is critical to the success of the ECLS. In support of these objectives, the contractor shall design and implement research activities to ascertain the extent to which items accurately reflect the elements measured, how the items are understood by the respondents (i.e, children and teachers) and how they might be interpreted. These activities shall seek to identify any areas of ambiguity and confusion.

Such early testing may include cognitive laboratory work similar to that described under subtask A.4. Prior to conducting any small-scale pilot research, the contractor shall develop and submit a memorandum outlining its plans for conducting the research. The plan shall describe the contractor's general approach to this work and identify the specific methods and procedures that shall be used. Justification shall be provided for the selection of certain methods over others relative to the objects of the research. The characteristics of individuals and/or groups that will be recruited for participation in this research activity shall be identified and justified in according to the research objectives. The plans shall describe how the outcomes of the research shall be analyzed and used to improve the overall quality of the ECLS assessment battery. Implementation of the plan shall be contingent upon the approval of the COTR.

All methods and procedures advanced for collecting information directly from children shall be appropriate for their age. Special care shall be taken to limit any negative impact of children's participation.

After implementing the small-scale pilot activities, the contractor shall analyze results and submit a memorandum that summarizes the methodology and findings of the ECLS research activity. The memorandum shall include recommendations for revising the instruments and procedures when indicated by the findings.

B.6 ECLS assessment plan. Completion of subtasks B.1 - B.5 shall result in a set of clear plans for and justification of the design of the second through fifth grade assessment. A report summarizing the contractor's plans for the design and conduct of the assessment shall be submitted to NCES/ED. At a minimum, the plan shall: 1) identify the key assessment domains and the critical elements within each domain to be collected; 2) describe the methods and procedures that will be used to conduct the assessment, identifying the sources of data for each domain/element; 3) include copies of any questionnaires and assessment instruments; 4) describe in detail any procedure that involves the direct assessment of young children and steps that will be taken to minimize the impact of children's direct participation; and 5) describe procedures for analyzing the data.

Optional Task C. Design third through sixth followup samples

A dual-frame (i.e., list and area frames) multi-stage stratified probability proportionate to size (PPS) design shall be used to select the base-year sample for this study. Geographic areas (e.g., county or group of counties) will be the primary sampling units (PSUs); school/kindergarten programs will be the secondary sampling units (SSUs); and children will be the ultimate sampling units (USUs). In followup surveys, children who were selected for previous waves will be selected into the current sample with given probabilities. For the school sample, only those schools that enroll sampled students from previous waves are eligible for the current wave and will be selected with given probabilities.

In most longitudinal surveys, mobility of sample units will influence their sample design. Student mobility is a major cost-driver in studies such as ECLS. High rates of mobility may require more drastic subsampling of students who transfer out of their schools, and in turn decrease sample efficiency. Mobility is an important contributor to sample attrition. Without sufficient efforts, resources, and adequate procedures to track students' movement from one school to another school, many sampled students will be lost in followup surveys due to "unable to locate". Hence, in just a few waves, the sample can deteriorate rather quickly. It is desirable to have some device in the ECLS sample design that would limit the impact of mobility to sample attrition.

C.1 Sample size requirements. The study will collect several different types of data. Estimates that will be produced by this study include: proportions (e.g., percent of children with certain characteristics) and means(e.g., mean mathematics achievement scores). The government is interested not only in detecting changes of the estimates over time but also in the precision of the cross-sectional estimates. For alpha equal to .05 with correlation of .60, the study should be able to detect a 20% change in proportions and a 5% change in means for ALL WAVES of data collection. The study also should be able to produce level estimates for which the standard error is no bigger than 10% of a proportion and 2.5% of a mean in ALL WAVES.

The above requirements shall be applied not only to the entire sample but to important subgroups. These subgroups include children who ever attended center-based early childhood programs, children who participate in public/private kindergarten programs, children who

participate in full-and part-day kindergarten programs, and children whose mothers are in the labor force and whose mothers do not participate in labor force.

Based on information on student mobility rates and sample attrition observed from other studies, a sample of 23,500 children attending 1,000 schools with kindergartens (200 private and 800 public schools) in 100 PSUs was determined to be adequate for meeting the objectives of the study for ALL WAVES. However, mobility rates may change over time, and they differ by socio-economic, geographic, and demographic characteristics such as family income, school type, locality, race-ethnicity. Students with certain characteristics are more likely than their peers to attend schools enrolling only one or two students from the previous wave. Consequently, they are more likely to be subsampled out and excluded from the current wave. It is important to have a sufficient number of students in the followup samples to support statistical analyses for these subgroups with the desired precision.

Data collected from the ECLS sample will, in addition to be used for descriptive purposes (e.g., estimate means, proportions), also be used for analytical purposes (e.g., estimate relationships). Many statistical procedures such as regression analysis and Hierarchical Linear Model (HLM) analysis have been applied to other NCES longitudinal studies data files (e.g., NELS:88). It is important that the ECLS sample data be able to support analyses using these procedures. Therefore, the contractor shall consider these factors when devising subsampling plans for followup surveys.

C.2 Mobility and tracking sample units. Sampled kindergarten children may change schools for many reasons including: 1) their families moving to a new location; 2) children completing the highest grade of the school they are currently attending; 3) the school closing; or 4) the school changing its boundaries. Such student mobility may necessitate subsampling in future waves in order to control cost.

In order to limit sample attrition due to "unable to locate" and to facilitate subsampling in followups, it is important to have some mechanism that gathers information on movers and locates them. The information that can be used to locate movers may be obtained from parents or legal guardians, and/or school staff. Information that has been collected and used successfully in locating sample units in other NCES longitudinal studies include, but are not limited to: 1) names, addresses, telephone numbers , and relationship to the child for parent /legal guardians , and two other people who would always know the whereabouts of the child. 2) names and location of the schools the child expect to attend after completing the highest grade of the school s/he is currently attending. In addition, school administration records may also provide information on school to which the student has transferred. Contractor shall maintain or enhance the procedures used in the base year and first follow up studies.

C.3 Oversampling for minorities. The study shall be able to support analyses of children by race-ethnicity (i.e., Hispanics, blacks, Asian and Pacific Islanders). In order to yield sample sizes sufficient for analytic purposes at the required level of precision, Asian and Pacific Islanders shall be oversampled in base-year. In subsequent sampling for followup, it is important to ensure that sample sizes meet the desired precision for the race-ethnicity subgroups.

C.4 Alternate sample design specifications. Even though the government envisions a dual-frame, multi-stage, stratified sample design for the base-year survey, and acknowledges the strong possibility that students and the schools in which they are currently attending will need to be subsampled for the followup surveys for ECLS, it recognizes that other designs may be as or more efficient. The final design shall be the one that best meets the objectives of the study.

C.5 Possible biases resulting from the proposed design and methods for adjusting these biases. Sample attrition may not be random. It is likely that children with certain characteristics have a higher attrition rate than others. Consequently, they are less likely to be available for subsampling in the followup surveys. This may cause bias. Other types of biases include nonresponse bias. The contractor shall develop methods for adjusting these biases and include them in its sample design plan (see subtask C.6).

C.6 Sample design plan. The contractor shall update the sample design plan found in the Final ECLS Study Design Report, presenting a sampling plan that meets the objectives and the requirements of the ECLS as stated in this RPF. The updated plan shall include, but not be limited to, specific sample sizes for each stage, the expected level of precision of survey estimates, sample allocation, stratification, sample units sort order, sample selection procedures, methods for adjusting biases, freshening, weighting procedures (including weights for followup schools), procedures for variance estimation, and a schedule for implementing the plan. All aspects of the sample design plan shall be well documented and submitted to NCES/ED as a Sample Design Report for review and approval. The Sample Design Report shall be submitted to NCES/ED within 12 months of the start of this option.

In order to assist NCES/ED staff monitoring the implementation of the sampling plan, the contractor shall provide NCES/ED with copies of all documents and specifications (e.g., computer specifications, formula for selection probability, sample size derivation and allocation, etc.) that describes and document how the sampling plan was developed.

D.1. Optional special studies

Validation Study of Teacher Reports of Classroom Characteristics. An important goal of the ECLS is to provide high quality and reliable data that will allow researchers to study the relationship between the environments in which learning takes place and children's education outcomes. The ECLS is especially interested in the classroom environment and its relationship to student achievement in core subject areas. It is important that the data collected on the structure, activities, and interactions that take place within the classroom be reasonably sound and representative of some of the key constructs of interest to the ECLS.

While NCES/ED believes that teachers represent a valuable source of information about the structure, activities, and interactions of the classroom, it also recognizes the value of conducting studies designed to monitor, evaluate, and improve the quality of the survey data. NCES/ED is especially interested in evaluating the validity of key classroom measures of instructional practices and content coverage for important curricular domains (e.g., reading, mathematics, science, and social studies) found in the teacher self-administered questionnaire.

The contractor shall design and implement a validation study of key classroom characteristics reported on the teacher questionnaire. The validation study of teacher reports of classroom characteristics shall focus on relevant measures of key constructs related to instructional practices and content coverage. In selecting and recommending an approach, the contractor shall consider the level of classroom intrusion/disruption and burden that the study may entail. The study shall consider the validity of both kindergarten and first grade teacher reports. It shall be conducted in the spring of the base year and first follow up studies.

Special Education Teacher Questionnaire. Extensive design work and field testing has gone into developing the survey instruments and data collection procedures for surveying the special education teachers of core ECLS sample children with disabilities. A survey instrument for teachers of core sample children with disabilities will be field tested during the spring of the 1996-1997 field test year. This instrument will be finalized based on the outcomes of the field test, suggestions from expert reviewers, OSEP, and NCES/ED.

The Office of Special Education Programs (OSEP) is funding this design and developmental work during Phase I. It may decide to fund this activity in the full-scale ECLS. The tasks that the contractor shall complete if this optional study is awarded are outlined in the following sections.

Reproduction of instruments. NCES/ED will provide the contractor with the final version of the special education teacher questionnaire at the project initiation meeting (task 1). The contractor shall reproduce or arrange for the printing of the special education teacher questionnaire in accordance with government regulations and requirements. A sufficient number of copies shall be produced to meet the needs of the study (see task 9).

Identify special education teachers of core sample children with disabilities. Once children with disabilities are identified by their classroom teachers and/or by school records, the contractor shall identify the special education teachers of these children. Children with multiple disabilities may receive multiple special education services from more than one special education teacher. For example, a child with multiple disabilities may receive services from a speech therapist and a learning disabilities teacher. In instances where a core sample child with disabilities receives special services from more than one provider, the contractor shall identify up to two special education service providers to include in the study. The rules that the contractor will follow in identifying and selecting these teachers will be specified in a memorandum to the NCES/ED COTR within six weeks of start date of this option.

Administer survey instrument to special education teachers of core sample children with disabilities, process survey data, and report on survey methodology. The contractor shall follow the procedures for administering the survey instruments found in the Study Design Report, documenting any deviations or problems. The contractor shall have responsibility for all tasks associated with securing the cooperation of special education teachers, administering the survey instrument, reporting on the progress of this activity, processing the survey data, and reporting on the results of these activities (see core tasks 5, 6, 10, 11, 12, and 14).

Verification of Head Start Program Participation. During the 1996-97 field test, school and parent reports of kindergartner's participation in Head Start are being evaluated. Schools and parents are being asked about children's participation in Head Start and to identify the Head Start program sampled children attended the year prior to kindergarten. All identified programs are being contacted to verify children's participation. The field test is designed to provide better information about 1) the level of misreporting (especially overreporting) of Head Start program participation, 2) the best source of information concerning children's Head Start participation, and 3) the difficulties of verifying children's past participation.

Depending on the outcomes of the field test, Head Start/ACYF might decide that it is necessary to verify children's Head Start participation in the base year ECLS and fund this activity. If Head Start/ACYF funds this activity, the contractor shall verifying the participation of all children who are identified as having attended Head Start the year prior to kindergarten. The contractor shall use information by the ECLS parent interview and/or the school in determining the identities of children whose participation requires verification. The contractor shall contact the Head Start programs these children presumably attended and verify the child's attendance. The contractor shall also collect a limited amount of information concerning the program and the child's participation (e.g., dates of attendance, length of attendance). The contractor shall utilize the outcomes of the 1996-97 field test when deciding on 1) the source of information to use to identify the group of possible Head Start participants, 2) the information that should be collected from programs about children's

Head Start participation and the program attended, and 3) the approach to contacting and collecting the information needed to verify children's participation.

Fall first grade and summer learning. NCES/ED is considering adding a fall first grade data collection to the ECLS. If a decision is made to add a fall 1999 first grade data collection, it will be necessary to develop a plan 1) for selecting a sample of first grade schools and first graders, 2) developing the instruments and procedures that are needed for this additional round of data collection, and 3) for collecting and processing these additional data. The contractor shall specify its plans in a memorandum to the NCES/ED COTR. This memorandum shall contain the contractor's recommendation regarding what portion of the ECLS sample (all or a subsample) should be included in this new wave of data collection, and how the subsample (if planned) will be selected. It shall pay particular attention to how the contractor intends to administer the ECLS assessment battery (both direct and indirect components), and shall identify any changes to the assessment battery or administration procedures. It shall also include the contractor's plan for developing and administering a 15 minute parent interview to capture information about children's summer activities and experiences, and a 15 minute self-administered teacher questionnaire to collect information about sampled children's fall first grade school/classroom experience. Approval of the plan by the NCES/ED COTR is required before the contractor shall begin to implement its approach. The memorandum outlining its approach shall be submitted within 60 working days of award of this optional data collection.

In preparing for and conducting the fall first grade data collection, the contractor shall follow the same core tasks that cut across the fall and spring kindergarten and spring first grade data collections. For example, the contractor shall include fall first grade instruments in the forms clearance package for obtaining IMT/OMB approval (task 3), select appropriate samples of schools, students, teachers, and parents, and secure their participation in the study (tasks 4 and 5), be able to regularly report on the status of data collection (task 6), arrange for the printing of the teacher questionnaires, the production of the assessment battery, and programming of CATI/CAPI instruments (task 9), prepare a detailed schedule for completing all data collection and successfully complete data collection according to this schedule (task 11), and have responsibility for processing all data collected (task 12). However, before implementing the applicable tasks specified in the core design, the contractor shall first develop the necessary parent and teacher instruments following the same set of steps outlined in optional tasks A.3 and A.4.

Prior to conducting the full-scale collection in the fall of 1999, the contractor shall conduct a small-scale pilot test of the fall first grade data collection instruments and procedures. This test shall take place in the fall of 1998. The pilot test shall involve approximately 250-300 first graders in 10 to 15 schools along with their parents and teachers. Prior to conducting the pilot test, the contractor shall develop and submit a memorandum to the NCES/ED COTR outlining its plan for both conducting the test and selecting schools, children, parents, and teachers, and for analyzing the pilot test data. A short (10-15 page) report on the results of the pilot shall be submitted to NCES/ED within 90 working days of the close of pilot data collection. At least some of the child assessments must take place in schools within the Washington, DC metropolitan area.

D. 2 Optional Supplements

Oversample of American Indian Children. If the option to oversample American Indian children is exercised, the contractor shall first review Appendix C: Issues in Oversampling American Indians and Alaskan Natives in the ECLS Study Design Report. The contractor shall then in a memorandum delineate to NCES/ED how it plans to implement either one of the design options mentioned in the appendix or its own oversampling recommendations. This memorandum shall be submitted within six weeks of start date of this option.

Upon approval of the oversampling plans by NCES/ED, the contractor shall implement its American Indian oversampling design within the steps outlined in core tasks 4 and 5 for the selection of and the securing of cooperation from schools, students, parents, and teachers. Those sampled shall then be treated the same as the core sample in all other aspects of the study (e.g., in terms of questionnaires administered, tracking procedures, data processing, etc.)

Oversampling of Head Start children. A decision as to whether or not to oversample kindergartners who attended Head Start during the year preceding kindergarten will be made during Phase I. If this option is exercised, the contractor shall select additional children with Head Start experience during the base year ECLS. Those sampled shall then be treated the same as the core sample in all other aspects of the study. That is, all instruments and procedures used with the members of the ECLS core sample of kindergartners shall be used with this sample of children.

The contractor shall proceed to implement this optional supplement following the steps outlined in tasks 1 through 14. All activities and deliverables associated with tasks 1 through 14 of the core ECLS shall apply equally to this supplement.

Oversampling populations of children with disabilities. The core ECLS sample will include sufficient numbers of children receiving special education services to support independent analyses of this group of children as a whole. Also, the core ECLS sample will include sufficient numbers of children to support independent analyses of certain high incidence disabilities (e.g., speech impairments and learning disorders). However, sufficient numbers of children to support independent analyses of certain low incidence disabilities is not expected unless these children are oversampled. OSEP is particularly interested in four groups children and will support the oversampling of these groups in the ECLS: 1) blind, 2) deaf and hearing impaired, 3) mental retardation, and 4) physical and health impairments.

The contractor shall select samples of children to support independent analyses of each of the four groups. To the extent possible, those sampled shall then be treated the same as the core sample in all other aspects of the study. That is, all instruments and procedures used with the members of the ECLS core sample of kindergartners shall be used with this sample of children, except where it is not feasible to do so. Children who cannot meaningfully participate in the core ECLS direct assessment shall participate in an appropriate alternative assessment. Alternative assessments which are appropriate for use with these samples will be identified during Phase I.

The contractor shall proceed to implement this optional supplement following the steps outlined in tasks 1 through 14. All activities and deliverables associated with tasks 1 through 14 of the core ECLS shall apply equally to this supplement.

Child Nutrition Supplement. To implement the Child Nutrition Supplement, funded by the U.S. Department of Agriculture's Food and Consumer Service (FCS), the contractor shall follow the core design as outlined in task 9. The items on how often children engage in activities requiring physical exertion and on families' abilities to get enough to eat have already been placed in the teacher and parent instruments. Thus, this component of the supplement shall be executed through the reproduction of the teacher checklist (task 9.1), programming of the parent instrument (task 9.3), and finally the administering, processing, and reporting of information from these two instruments as outlined in the core tasks 10, 11, 12, and 14 of the work statement. These data are collected in the spring of the school year.

With regards to measuring children's height and weight, the contractor shall acquire the following materials in addition to those already specified in task 9.2:

strain gauge scale for measuring weight

height measurement board with a head and foot board

All data on children's height and weight shall be collected, processed, and reported in the same manner as the measures existing in the core design (see tasks 10, 11, 12, and 14). Height and weight data shall be obtained in the fall and spring of kindergarten and in the spring of first grade.

IV. STUDY REQUIREMENTS

A. Reporting

The contractor shall submit monthly letters of progress to the Contracts Officer for the duration of the contract. Letters of progress shall describe major activities by task and accomplishments, problems and suggested solutions, significant findings and events, any decisions which may be needed from NCES/ED, and plans for the period until the next report. In addition to reporting on the progress of the contract, the letter of progress shall be used as one of the ways of keeping all parties (e.g., subcontractors, cosponsors) up to date on project activities. Given these dual purposes, the contractor shall submit a draft letter to the NCES/COTR for review prior to circulating it more widely. Progress letters shall be submitted to the Contracts Officer on the 10th business day of each month.

The contractor shall communicate regularly (several times each week) and work closely with the NCES/ED COTR on all aspects of the study. To facilitate this communication, the contractor shall set up an electronic system for transferring information via electronic mail and microcomputer. The contractor shall make frequent phone calls and up to four visits per year to NCES/ED to discuss project problems and progress. The contractor's project director shall have a regularly scheduled weekly telephone conference with the ECLS/ED COTR to discuss project progress, upcoming activities, and problems.

B. Schedule of Milestones and Deliverables

The products that the contractor is required to prepare and submit to NCES/ED are indicated in the descriptions of the 14 core tasks, 3 optional tasks (tasks A - C), and the optional studies and supplements. While there may be other possible schedules for completing the tasks associated with the ECLS, several activities must take place at specific times during the conduct of the project. The final IMT/OMB clearance package for the base year and first follow up data collections (see task 3) shall be submitted to NCES/ED at least 150 working days prior to the start of the fall 1998 data collection. Any slippage in the delivery of the IMT/OMB clearance package can seriously jeopardize the study. During Phase I it was determined when the best time was for conducting in-school child assessments and for asking teachers to report on the skills of the children they teach. The schedules for the base year and first follow up data collection components described in A.5 Schedule of Data Collection must be adhered to. Exhibit 3 summarizes the critical project milestones and deliverable dates.

C. NCES/ED Review and Approval

The contractor shall consult with the NCES/ED COTR before making any major decision concerning a deliverable product. Reports prepared by the contractor shall follow guidelines referenced by the most current version of NCES/ED' Standards (1992).7 It is a requirement that schedules for deliverables found in the contractor's proposal be kept and that NCES/ED standards be followed during all phases of project work.

An outline shall be submitted and approved by the NCES/ED COTR before preparation of specific reports and plans. Outlines are required for the following reports and plans:

•	User's manuals (task 12)

•	Methodology report (task 14)

•	Sampling plan (optional task 17)

The contractor shall expect that major reports and deliverables (e.g., user's manuals, IMT/OMB clearance package) shall require three revisions. The contractor shall expect that other deliverables (e.g., monthly progress letters, meeting agenda) shall require one revision.

The contractor shall allow sufficient time for NCES/ED and OERI review. Unless noted elsewhere, the contractor shall allow two weeks for NCES/ED review of each draft deliverable. Two weeks shall be allowed for the contractor to respond to any requests for revisions. All revisions requested by NCES/ED shall be incorporated unless justified in a cover memorandum.

D. Quality Control

The contractor is required to develop and submit a number of reports, memorandum, and other materials and products during the conduct of the ECLS. The expectation is that all such materials (even when they are labeled draft) will be of high quality and of sufficient quality to be released as is. Quality control of these materials is a contractor responsibility. All reports, memorandum, and other deliverables (including any produced by subcontractors) submitted to NCES/ED shall be accompanied by a letter of transmittal signed by the ECLS project director and the corporate official with corporate oversight responsibility for the ECLS.

V. PROPOSAL REQUIREMENTS

A. Use of Subcontractors

Proposals may include plans to subcontract parts of the work, provided that evidence is presented that any proposed subcontractor has agreed to perform and is fully capable of performing the assigned tasks, and that the offeror has effective control of the subcontractor's work in the project. Offerors are encouraged to make use of subcontractors for specialized tasks where subcontractor expertise would strengthen the offeror's proposal.

B. Period of Performance

Tasks 1 through 14 are to be completed within 48 months of contract award. At the option of the government, optional design tasks A through C, and the optional special studies and supplements are to be completed within the same 48 month period of performance, or 18 months from the start date of the options. The contract will be incrementally funded.

C. Documents Available for Inspection

Copies of all ECLS design documents and materials referenced in this RFP may be obtained by calling the National Data Resource Center, C/O Pinkerton Computer Consultants, Inc. at (703) 845-3151.