IMPORTANT NOTE:    
THIS TEXT VERSION OF THE STATEMENT OF WORK HAS BEEN CREATED SOLELY TO 
MAKE MORE INFORMATION ABOUT THIS RFP VIEWABLE ONLINE.  THIS VERSION IS 
NOT IDENTICAL TO THE RELEASED SOW BECAUSE INFORMATION/FORMATTING MAY HAVE 
BEEN LOST IN THE CONVERSION FROM WORD PERFECT 6.1 TO ASCII TEXT.  THIS TEXT 
VERSION SHOULD NOT BE USED TO PREPARE A PROPOSAL.  






                       Statement of Work
                                
Early Childhood Longitudinal Study: Kindergarten Class of 1998-1999
                             (ECLS)
                                
                                
                                
                                
                                
                                
                                
                                
                                
                                
                                
                                
                                
                                
                                
        Data Development and Longitudinal Studies Group


I.  INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . .1
     A.  Purpose and Nature of the Procurement . . . . . . . . .1
     B.  Background. . . . . . . . . . . . . . . . . . . . . . .2
          B.1   Enabling Legislation   . . . . . . . . . . . . .2
          B.2   The Need for National Data on Children's Early Education2
          B.3   NCES/ED Responsibilities and Initiatives . . . .3
     C. ECLS Design Activities . . . . . . . . . . . . . . . . .3
      D.  Key Study Issues and Questions   . . . . . . . . . . .4

II.  KINDERGARTEN COHORT STUDY DESIGN. . . . . . . . . . . . . .6
     A.  Overview. . . . . . . . . . . . . . . . . . . . . . . .6
          A.1  Target Population . . . . . . . . . . . . . . . .6
          A.2  Kindergarten Cohort Sample. . . . . . . . . . . .7
          A.3  Survey Components . . . . . . . . . . . . . . . .7
          A.5  Data Collection Schedule. . . . . . . . . . . . .9
          A.6  Measurement of Growth in Cognitive Dimensions . 11
          A.7  Measurement of socioemotional development . . . 18
           A.8 Physical and Psychomotor Assessment . . . . . . 19
     B.  Sample Design . . . . . . . . . . . . . . . . . . . . 19
          B.1  Kindergarten sampling frame . . . . . . . . . . 20
          B.2  Sample  attrition and mobility  . . . . . . . . 20
          B.3  Oversampling of minority students . . . . . . . 21
          B.4  Freshening of sample. . . . . . . . . . . . . . 21
          B.5  School-level estimates for later waves. . . . . 22
     C. Methodological Inquiries and Data Quality. . . . . . . 23
     D. Optional Special Studies and Supplements . . . . . . . 23
          D.1   Optional Special Studies . . . . . . . . . . . 24
          D.2   Optional Supplements . . . . . . . . . . . . . 27
     E.  NCES/ED Standards . . . . . . . . . . . . . . . . . . 31

III.  SCOPE OF WORK. . . . . . . . . . . . . . . . . . . . . . 32
     A.  Overview of Tasks . . . . . . . . . . . . . . . . . . 32
     B.  Core Tasks to Be Performed. . . . . . . . . . . . . . 32
          Task 1.   Project initiation and planning. . . . . . 32
          Task 2. Technical review panel . . . . . . . . . . . 33
          Task 3.  IMT/OMB clearance of main survey instruments36
          Task 4.  Select school sample, notify sampled schools and secure cooperation36
          Task 5.  Select student and teacher samples and secure cooperation38
          Task 6.  Receipt control and reporting . . . . . . . 39
          Task 7.  Tracking of kindergarten sample . . . . . . 40
          Task 8.  Conduct pilot test. . . . . . . . . . . . . 41
          Task 9.  Reproduction of instruments and acquisition of materials41
          Task 10.  Hiring and training data collection staff. 44
          Task 11.  Collect kindergarten and first grade data. 47
          Task 12.  Data processing. . . . . . . . . . . . . . 49
          Task 13.  Briefing materials.  . . . . . . . . . . . 52
          Task 14.  Methodology report . . . . . . . . . . . . 52
     C.  Optional DesignTasks to be Performed. . . . . . . . . 55
          Optional Task A.  Develop instruments and procedures for the second through
               fifth grades. . . . . . . . . . . . . . . . . . 55
          Optional Task B.  Extend the EC LS Assessment Battery57
          Optional Task C. Design third through sixth followup samples60
          D.1.  Optional special studies . . . . . . . . . . . 63
          D. 2 Optional Supplements. . . . . . . . . . . . . . 66

IV. STUDY REQUIREMENTS . . . . . . . . . . . . . . . . . . . . 68
     A.  Reporting . . . . . . . . . . . . . . . . . . . . . . 68
     B.  Schedule of Milestones and Deliverables . . . . . . . 68
     C.  NCES/ED Review and Approval . . . . . . . . . . . . . 69
     D.  Quality Control . . . . . . . . . . . . . . . . . . . 69

V. PROPOSAL REQUIREMENTS . . . . . . . . . . . . . . . . . . . 70
     A.  Use of Subcontractors . . . . . . . . . . . . . . . . 70
     B.  Period of Performance . . . . . . . . . . . . . . . . 70
     C.  Documents Available for Inspection. . . . . . . . . . 70

                         I.  INTRODUCTION


A.  Purpose and Nature of the Procurement

     The National Center for Education Statistics (NCES) of the United States Department of
Education (ED) requires a contract to conduct the base year and first followup of the Early
Childhood Longitudinal Study (ECLS): Kindergarten Class of 1998-99.  Two data collections
occur during the base year of the ECLS, fall kindergarten and spring kindergarten.  First
followup data is scheduled to be collected in the spring of first grade.

     The period of performance of the contract shall be 48 months.  In addition to a set of 14
core tasks, the procurement contains 1) several optional tasks, 2) special studies and 3)
supplements for additional tasks and collections within the 48 month period of performance.

     The Early Childhood Longitudinal Study (ECLS) responds to the immediate and
continuing need for high quality and timely data pertaining to the condition of the early
education of the nation's children.  The ECLS has two purposes: 1) descriptive and 2) analytic. 
On the one hand, it will use nationally representative samples of schools and children to
describe (1) children's status at entry into school, and the characteristics of their schools and
classrooms; (2) children's transition into school; and (3) their experiences and learning from
kindergarten through grade 5.  On the other hand, the ECLS will provide a rich data set that
will enable researchers to study how a wide range of family, school, community, and
individual variables affect early success in school.  

     For the first time, national data will be available on both public and private
kindergarten programs and the children who attend them.  A goal of the kindergarten data
collection is to describe accurately the diversity of kindergarten children and the programs
they attend.  Such information is critical to establishing policies that can respond sensitively
and creatively to this diversity.  

     Data collected during the kindergarten year will also serve as baseline data to examine
how schooling shapes later individual development.  The longitudinal nature of the study will
enable researchers to study children's growth in critical developmental domains and to relate
trajectories of growth and change to variations in children's school experiences in kindergarten
and the early grades.

     A contract to design the ECLS was awarded in Spring 1994.  All design work that
occurred since then, including the conduct of a large-scale field test (feasibility study) in school
year 1996-97, constituted Phase I of the ECLS.  Phase II of the ECLS begins with the award of
this contract to conduct the base-year study and first followup study.
    
     This procurement involves 1) finalizing the ECLS sample design building on the work
that was done during Phase I, 2) selecting the ECLS sample of schools and kindergartners and 3)
preparing for and implementing the ECLS base year and first followup data collections using the
instruments and procedures that were developed during Phase I.  It also includes an option to
develop the instruments and procedures that will be used in later follow-ups of the ECLS, i.e.,
second through fifth grades and several additional options for enhancing the design of the ECLS
through sample augmentations and supplements.

B.  Background

B.1   Enabling Legislation   

     The ECLS is being undertaken in compliance with the mandate stated in section 404 of
the National Education Statistics Act of 1994 (20 U.S.C. 9003): 

     "The duties of the Center are to collect, and analyze and disseminate statistics and other
     information related to education in the United States and in other nations, including
     ...conducting longitudinal studies, as well as regular and special surveys and data
     collections, necessary to report on the condition and progress of education..."


B.2   The Need for National Data on Children's Early Education

     The critical importance of high-quality, equitable education for the continued
development and prosperity of the country has emerged as a very high priority on the national
agenda.  Parents, educators, and policy makers are reconsidering the ways children are taught in
schools and are looking for more effective approaches to education.  

     Much of the public discussion on education is focused on the early years, children's
development and learning during the early childhood period from birth to age eight, their
preparation for formal school, the first school experience, and the progress they make over the
first years of school.   A number of factors have contributed to the research and policy focus on
children's early school experiences.  These include: 1) the increased public awareness of the
importance of children's early experiences, reinforced by the National Education Goal on school
readiness; 2) the changing nature of children's preschool and early school experiences; 3) the
increasingly diverse population of children entering school and the demands this places on
schools; and 4) the expanded role that schools are expected to play in supporting and nurturing
development and learning.

     Vital to efforts to improve schools and the education of all the Nation's children is a
research and data collection program that increases our understanding of the dynamics of school
achievement, particularly of those factors that lead to the differential success of important groups
of children during the early school years.  The Early Childhood Longitudinal Study (ECLS), in
which a cohort of entering kindergarten children will be sampled and studied intensively over
time, responds directly to the concerns decision makers, educational practitioners, researchers,
and parents have about our Nation's schools and American education.

B.3   NCES/ED Responsibilities and Initiatives

     NCES has as its mission the collection and dissemination of data on the condition of
education in the United States.  It collects statistics on the status and progress of education in the
Nation, analyzes and reports the meaning and significance of those statistics, and assists States
and local education agencies in improving their statistical systems.

     Prior to the National Household Education Survey (NHES), NCES collected little
information on children below the eighth grade level, other than basic enrollment data and
assessment data from NAEP and TIMSS.  At present, there is no national data base that
permits a thorough study of children's transition into school and their progress through
elementary school.  Such a study requires detailed data on children's experiences at school
entry as well as data on their school environments and programs.  It requires that all segments
of the student population be represented, in order for NCES to examine the school experience
for different segments of the population.

C. ECLS Design Activities

     Design work on the ECLS began in 1989 and has involved many individuals and
organizations. More intensive work on the design of the ECLS began in 1994.  In April,
1994, ED/ NCES awarded a contract to the National Opinion Research Center to 1) develop
the overall design for the ECLS including the sample design and data collection
methodologies and procedures; 2) develop the instruments and procedures that would be used
in the base-year study (kindergarten fall and spring) and the first two followup studies (spring
first grade and spring second grade); 3) develop an assessment battery to measure the status
of children's development in important cognitive and noncognitive domains at kindergarten
entry and to measure growth in these domains as children move through the elementary
school years;  4)  design and conduct a large-scale field test or feasibility study of the ECLS
base-year and first followup designs; and 5) finalize the designs of the base year and first
followup studies based on the field test experience.


D.  Key Study Issues and Questions  

     In general, the ECLS will focus on children's 1) transition to school, 2) schooling and
performance in the early grades, and 3) the interaction of school, family, and community. 
While these general areas encompass a vast array of issues and research questions, certain
issues have been identified as more important for the study.  The four key issues to be
addressed by the ECLS are 1) school readiness; 2) children's transitions to kindergarten, first
grade, and beyond; 3) the relationship between children's kindergarten experience and their
elementary school performance; and 4) children's cognitive growth and progress through
elementary school.  Each of these key issues is described within the context of the three
general areas in the following sections.

     School Readiness.  Some children seem to adjust to their first encounter with formal
school and are able to have a positive school experience while other children have problems
making this adjustment.  For most children, the first formal school experience is
kindergarten.  The nature of this kindergarten experience is quite variable.  While 98 percent
of children nationwide attend a kindergarten program prior to first grade, 40 percent of
children attend full-day programs and 58 percent part-day programs.  Attendance in public
school kindergarten programs is about six times that of private school programs.  Within
these program types, curricula and instructional practices are expected to vary widely.

     The ECLS has adopted a conceptual model of school readiness that is consistent with
the one set forth by the National Education Goals Panel, Goal One Technical Planning
Group.  Thus, school readiness is conceived as being multidimensional and continuous. 
Children may have more or less of each of the skills, experience, and knowledge that increase
their chances of having a positive school experience.  Children who are relatively weak in
one area may or may not be weak in others.

     The ECLS will describe children's school readiness on a number of different
dimensions and how each is related to different characteristics of children, their families,
communities, and kindergarten programs/classes.  The study is interested in the role that
parents/family play in helping prepare children for school and in adjusting to the school
environment.  In order to understand children's preparation for school, it is critical to
understand how schools prepare for and respond to children who enter their doors for the first
time.  The study will pay particular attention to how schools and kindergarten programs
respond to the backgrounds and experiences that children bring with them.  Data will be
collected on a range of characteristics of children's schools, kindergarten programs, and
classrooms.  

     Transitions to Kindergarten, First Grade, and Beyond.  Children and adults
continually make transitions from one status to another.  Of particular interest to the ECLS, is
the transition that occurs as young children go from preschool to kindergarten, and from
kindergarten to first grade.

     Kindergarten and Elementary School Student Performance.  A central issue for the
ECLS is the relationship between children's kindergarten experience and their school
performance in grades 1 through 5.  A goal of the program is to provide descriptive
information on children as they move out of kindergarten, into first grade and through the
elementary school grades.  The study will focus on student achievement during these years as
well as other measures of student performance.  By closely monitoring student performance,
the study will provide useful information on when children begin to experience problems
with their school work and the circumstances surrounding these difficulties.  The study will
provide useful information on the longevity of problems and the response of the child's
family, school, and teacher to problems.

     Cognitive Growth and Student Progress.  The ECLS is interested in students'
performance in the primary (grades 1-3) and elementary grades (grades 4 and 5).  A major
goal of the study is to monitor student achievement and academic progress during children's
early school careers.  The ECLS seeks to understand better children's cognitive, as well as
social development during these early school years.


              II.  KINDERGARTEN COHORT STUDY DESIGN

A.  Overview

     The ECLS is intended to provide information that will serve to inform educational
policy.  The data collected through this study and the information that is disseminated
through reports prepared by NCES/ED and others based on these data will inform decision
makers, educational practitioners, researchers, and parents about children's earliest school
experiences and how the Nation's schools affect the lives of the children who pass through
them.  

     While most of the components of the design have been developed during Phase I,
they cannot be finalized until the 1996-97 field test is complete. Key elements of the ECLS
design are outlined in the following sections.

     A detailed account of the ECLS design can be found in the ECLS Study Design
Report.  This document includes: 1) the ECLS sampling plan, 2) a description of the data
collection procedures and methods, 3) a plan for monitoring and evaluating the quality of
the data, 4) descriptions of the contextual questionnaires, and 5) the framework for the
ECLS assessment battery and the rationale for the assessment approach being used in the
study.  The design outlined in this document is the one that is being field tested in school
year 1996-97.

A.1  Target Population

     The ECLS is concerned with the Nation's children.  It is intended to focus on the
educational experiences of minority and majority, disadvantaged and advantaged children
alike.  Thus, the target population of the ECLS is all children enrolled in U.S. public and
private kindergartens. 

     There are many types of kindergarten programs (e.g., public school and nonprofit
private school programs, as well as for-profit programs).  Some programs are full-day and
others are part-day.  Most operate five days a week while some operate for fewer than five
days.  Most programs are designed as one year programs, while others are designed as two-year programs.  All of these characteristics are being considered when defining the
population of both kindergarten programs and kindergartners.  

     NCES/ED intends to be as inclusive as possible when defining kindergarten programs
and including kindergarten children in the ECLS.  Also, it is NCES/ED's goal to limit the
numbers of children who are excluded from the ECLS as a consequence of their having
special needs that require specialized data collection instrumentation and procedures.  
NCES/ED's desire to collect data on children from multiple sources (e.g., parents, teachers,
and the children themselves) should reduce the number of children who are excluded from
participation.  NCES/ED has been working with the U.S. Department of Education's Office
of Special Education Programs to find ways to increase the participation of children with
disabilities in the ECLS, and NCES/ED is continuing to develop an approach for more
effectively including children with limited proficiency in English in the ECLS assessment.

A.2  Kindergarten Cohort Sample

     A nationally representative sample of approximately 23,000 children enrolled in
about 1,000 kindergarten programs during the 1998-99 school year will be selected for
participation in the study.  These children will be selected from both public and private
kindergartens, offering full-day and part-day programs.  The sample will consist of
children from different racial-ethnic and socioeconomic backgrounds.  Presently, NCES
plans to oversample private kindergartens and kindergartners.  It also plans to oversample
Asian and Pacific Island children, and if necessary Hispanic children.  It continues to
explore the feasibility of oversampling American Indian children and children with
disabilities.
 
A.3  Survey Components

     In order to address all of the key research issues,  it is necessary to examine the whole
of children's environments and development.  The emphasis that the ECLS is placing on the
whole child has critical implications for the design of the ECLS.  The study will collect data
about the child, the child's school, classroom, teacher, and family.  These data will be
collected directly  from the child, the child's parents/guardians, teachers, and school
administrators.

     Children will be asked to participate in various activities in order to measure the
extent to which they exhibit those abilities and skills deemed important to success in school. 
They shall be asked to participate in activities designed to measure important cognitive (e.g.,
general knowledge, language and quantitative skills) and noncognitive (e.g., social skills)
competencies.  At least through the end of first grade, direct measures of a child's cognitive
skills will be obtained through a one-on-one assessment of the child.  During later years of
the ECLS, children's participation will be broadened to include self-reports of their school
and out of school experiences.

     During the kindergarten year, children will twice participate in a one-on-one
assessment.  In the fall of the kindergarten year, children will be administered a 30-45 minute
cognitive assessment with three subtests (mathematics, language/reading, and general
information).  During the fall, children will also participate in a short psychomotor
assessment, and possibly a short direct assessment of their social skills.  In the spring of
kindergarten, children will be administered a 45-50 minute cognitive assessment with the
same subtests, and possibly the same direct social skills assessment.  The first grade
cognitive assessment will be 45-50 minutes in length and will cover the same domains.  A
direct measure of children's social skills may be used in first grade.  No assessment of
children's psychomotor development will occur after the fall of kindergarten.  All of these
assessments are being developed and tested during Phase I.
 
     Parents/guardians are an important source of information about the families of the
children selected for the study and about themselves. Due to the young age of the children
that will be participating in the ECLS, parents/guardians are needed to act as proxy
respondents for their children during the early years of the study.  Furthermore, they are an
important source of information on themselves and on their children's home environment,
development at school entry, and neighborhood.  

     Because the ECLS will rely on parents for so much data, interviews with parents or
guardians will be conducted at each wave of data collection i.e., in the fall and spring of the
kindergarten year and in the spring of first grade.  The interview shall be conducted via
telephone using CATI technology and shall be limited to 45 minutes on average.  In those
cases where a telephone interview is not possible, a personal interview shall be conducted
using CAPI technology.  The parent/guardian to be interviewed shall be the individual who is
the most knowledgeable about the sampled child's care and education.  In many cases, this
individual will be the child's mother.

     Parent/guardian interviews for fall and spring kindergarten and for spring of first
grade were developed during Phase I and are being field tested in 1996-97. 

     Teachers, like parents, represent a valuable source of information on themselves, the
children in their classrooms, and the children's learning environment, i.e., the classroom.  No
other respondent group is as burdened as teachers by a study such as the ECLS. Teachers of
sampled children will not only be asked to provide information about their own backgrounds,
teaching practices, and experience, they will be called upon to provide information on the
classroom setting for the sampled children they teach during the fall and spring of the
kindergarten year and in the spring thereafter. 

     Teachers will be asked to provide information across a range of important themes in a
self- administered questionnaire.  It is estimated that teachers will spend between 30-45
minutes completing this questionnaire.

     Teachers will also be asked to evaluate each sampled child on a number of critical
cognitive and non-cognitive domains.  The information teachers provide about a sampled
child's social skills and behaviors, physical activity, and performance in the curricular area
domains included in the cognitive assessments broadens the range of information about the
child.  The three instruments developed for rating each child include: 1) a teacher checklist
reporting on such things as the children's language skills, special needs, program placements,
attendance, and physical activity,  2) a cognitive rating scale, and 3) a social skills rating
scale.

     While teachers of sampled children will not be remunerated for completing the
teacher questionnaire, they will be paid for completing the three rating instruments for their
pupils.  Without remuneration for their role as data collectors, ECLS teachers would be
subject to unusually high levels of uncompensated burden.  On average, teachers will spend
over 2 hours completing the three rating instruments for the sampled pupils in their class. 
This estimate is based on the assumption that the set of forms for each child will take 12-15
minutes to complete and that teachers will average 10 ECLS sampled students in their class.

     Kindergarten (fall and spring) and first grade (spring) teacher questionnaires and
rating forms were developed during Phase I and are being field tested in 1996-97.

     School administrators, directors, principals, and headmasters are an important
source of information on the schools that children attend.  While classroom experiences may
be the leading school-related factors of interest in the ECLS, school physical and
organizational characteristics, school policies, and other school-level characteristics can
shape, promote, or inhibit what goes on in children's classrooms.  

     Data will be collected from school administrators/directors annually via a self-administered questionnaire.  This questionnaire will be distributed during the time of the
spring assessments and will take 45 minutes on average to complete.  

     The kindergarten and first grade school administrator questionnaire was developed
during Phase I and is being field tested in the spring of 1997.

A.5  Data Collection Schedule

     Exhibit 1 shows the ECLS implementation schedule.  The contractor shall to begin
full-scale data collection during the fall of school year 1998-99.

     The contractor shall collect data twice during the base year of the survey--once at the
beginning and once near the end of the school year.  The beginning of the year data collection
is needed in order to get baseline information on children prior to their exposure to the
influences of the school.  It will provide measures of the skills, attributes, and knowledge of
children as they enter school for the first time.  The data collected at the end of the school
year, together with beginning school year data, will be used to examine children's early
schooling experiences and changes during the kindergarten year.  Follow-up surveys will be
conducted annually.  These will be used to study children's growth in important cognitive
domains during the critical early school years. 

     The contractor's data collection schedule shall take into account differences in school
calendars. For schools on a traditional nine-month calendar, the goal is to collect direct child
assessment data for fall kindergartners over a six week period about 6 weeks after the start of
the school year and spring data should be collected over a similar period about 6 weeks
before the end of the school year.  Spring first grade data should be collected on the same
schedule as spring kindergarten.  Fall kindergarten teacher ratings of children's cognitive and
social skills should be collected about 8 weeks after the start of school and spring
kindergarten and first grade teacher ratings should be collected about 8 weeks before the end
of school.  The schools on non-traditional calendars, the goal is to collect data on schedules
that are comparable to those using nine-month calendars.  

     Fall 1998 data collection shall consist of 1) a 45 minute direct assessment with each
sampled child; 2) a 30-45 minute self-administered teacher questionnaire; 3) teacher ratings
of each sampled child's cognitive and social skills, together taking approximately 10 minutes
to complete; 4) a teacher checklist for each sampled child taking approximately 2-5 minutes
to complete; and 5) a 45 minute CATI/CAPI parent interview.  For schools operating on a
traditional nine month calendar, all direct child assessment data shall be collected within a six
week period.  The collection of teacher questionnaire data in these schools, along with
teacher's ratings of children's cognitive development and social skills, shall be collected
within a four week period.  Parent interviews shall begin no later than two weeks after the
first direct child assessment and shall be completed within two weeks of the last direct child
assessment.  

     Spring 1999 data collection shall consist of 1) a 45-60 minute direct assessment with
each sampled child; 2) a 30-45 minute self-administered teacher questionnaire; 3) teacher
ratings of each sampled child's cognitive and social skills, together taking approximately 10
minutes to complete; 4) a teacher checklist for each sampled child taking approximately 2-5
minutes to complete; 5) a 45 minute self-administered school administrator questionnaire; 6)
a 45 minute CATI/CAPI parent interview; and 7) a school record review and retrieval.  All
direct child assessment data shall be collected within a six week period.  The collection of
teacher questionnaire data, including teacher's ratings of children's cognitive development
and social skills, and the teacher checklist, shall be completed 8 weeks before the end of
school.  All teacher questionnaire data shall be collected within a four week period.  The
collection of school administrator questionnaire data shall be completed within a four week
period.  Parent interviews shall begin no later than two weeks after the first direct child
assessment and shall be completed within two weeks of the last direct child assessment.  The
extraction of data from children's school records shall occur as late in the school year as
possible to insure that the most complete academic year data are available as possible.  The
collection of school record data shall be completed within a four week period.

     Spring 2000 data collection shall consist of 1) a 45-60 minute direct assessment with
each sampled child; 2) a 30-45 minute self-administered teacher questionnaire; 3) teacher
ratings of each sampled child's cognitive and social skills, together taking approximately 10
minutes to complete; 4) the teacher checklist for each sampled child taking about 3-5 minutes
to complete per child; 5) a 45 minute self-administered school administrator questionnaire; 6)
a 45-60 minute CATI/CAPI parent interview; and 6) a school record review and retrieval. 
All direct child assessment data shall be collected within a six week period.  The collection of
teacher questionnaire data, including teacher's ratings of children's cognitive development
and social skills and the teacher checklist, shall be completed within a four week period.  The
collection of school administrator questionnaire data shall be completed within a four week. 
Parent interviews shall begin no later than two weeks after the first direct child assessment
and shall be completed within two weeks of the last direct child assessment.  The extraction
of data from children's school records shall occur as late in the school year as possible to
insure that the most complete academic year data are available as possible.  The collection of
school record data shall be completed within a four week period.

A.6  Measurement of Growth in Cognitive Dimensions

A.6.1  Purpose and areas of interest

     Measuring children's growth in critical cognitive dimensions is an essential element
of the ECLS.  Like the assessment batteries used in other NCES/ED longitudinal studies, the
primary purposes of the ECLS cognitive assessment battery are to obtain accurate measures
of the status of children at a given point in time and their growth over time and to relate this
growth to a broad set of school, family, and community variables.  

     The knowledge and skills assessed by the ECLS cognitive battery represent typical
educational goals of the elementary school curricula.  The ECLS cognitive battery includes
three cognitive domain subtests that measure children's receptive, expressive, and nonverbal
reasoning  abilities in language/literacy, mathematics, and general knowledge (i.e., science
and social studies).  The focus on core academic subjects  (i.e., reading, mathematics, and
general knowledge) reflects the interest in precursors of children's later educational career
outcomes and the impracticality of adequately measuring kindergartners' and first graders'
proficiencies in writing, art, and music within the constraints of the study.  


Exhibit 1. ECLS Data Collection Schedule, by time of school year


Survey Activity
Phase I
Phase II



F  96
S  97
F  97
S  98
F  98
S  99
F  99
S  00
F  00
S  01
F  01
S  02
F  02
S  03
F  04
S  04


Field Test


   Kindergarten


















   First Grade


















   Second Grade


















   Third Grade


















   Fourth Grade


















   Fifth Grade


















Full-Scale Data Collection


   Kindergarten


















   First Grade


















   Second Grade


















   Third Grade


















   Fourth Grade


















   Fifth Grade


















F   Fall  S   Spring     Analogous instruments have been developed to collect information on children's
status and growth in the core academic areas directly from the children and indirectly from
their teachers during the kindergarten through second-grade years.  Teacher ratings are used
to augment and extend the range of cognitive data collected on the children.

A.6.2 Approach to developing the cognitive battery

     Consultations with curriculum specialists and teachers guided the development of the
ECLS direct cognitive assessment battery framework and item pool.  In addition, the ECLS
battery framework drew upon the recently completed National Assessment of Educational
Progress (NAEP) frameworks and other relevant domain frameworks for reading,
mathematics, science, and social studies.   The overlap in the goals and skills of the NAEP
and ECLS frameworks allow for comparisons between the two studies and potentially
enriches what is learned from each.

     While the ECLS assessment battery frameworks have much in common with the
NAEP framework and others, it differs from these in the following ways.  The ECLS will
measure children's growth of cognitive skills and knowledge across the whole span of the
study.  The longitudinal design of the ECLS dictates the necessity of a vertical scale that can
measure valid change scores within more focused cognitive domains.  In addition, the length
of time that children in the early primary grades can be realistically assessed constrains the
administration of direct assessment batteries in the ECLS.  This time restriction directly
limits the number of items within a set of skills that can be measured in the ECLS.

     For each domain subtest, a pool of items was created.  The item pool includes items
selected from published instruments and items developed specifically for the ECLS.  Sources
of items from published instruments include the Peabody Picture Vocabulary Test (PPVT-3),
Peabody Individual Achievement Test (PIAT), Primary Test of Cognitive Skills, Test of
Early Mathematics Ability (TEMA-R), Test of Early Reading Ability (TERA-R), and the
Woodcock-Johnson Psycho-Educational Battery-Revised Achievement Tests. Teachers,
subject area specialists, and test development staff have evaluated the items for their
relevancy and importance to specific domains and subdomains, sensitivity to diverse
populations, and clarity and validity.

     Language/literacy.  The ECLS language/literacy domain test specifications are
derived mainly from the 1992 and 1994 NAEP Reading Framework.  Other sources of
information for the reading test item pool include the Standards for the English Language
Arts (1996).  Curriculum specialists in early literacy and reading and focus groups of primary
grade teachers reviewed the proposed framework and item pool.  Items in the reading subtest
measure the following skill and knowledge categories: basic skills, vocabulary, and
comprehension skills.  In the early primary grades, basic skills is a measure of early literacy
such as print awareness and orientation, phonetics such as the knowledge of letter sound
relationships, and sight word vocabulary.  Vocabulary measures children's knowledge of
receptive verbal concepts.  Children's comprehension skills are measured with the use of
listening comprehension and reading comprehension items.  The types of comprehension
skills measured by the listening and reading comprehension items are similar to NAEP's four
types of reading comprehension skills (i.e., initial understanding, developing interpretation,
personal reflection, and demonstrating a critical stance).

     In the kindergarten and first grade assessment batteries, language/literacy battery
items measuring relevant reading skills include items such as left-right orientation of print,
letter names, word rhymes, initial and final sounds of words, and recognition of high
frequency words (i.e., sight vocabulary) and sentence reading.  Measures of children's
language include receptive vocabulary items and expressive language use (e.g., use of
complex sentence structures).

     Mathematics.  The mathematics domain test specifications are primarily based on the
NAEP 1996 Mathematics Framework.  This framework is based largely on the curriculum
standards from the Commission on Standards for School Mathematics of the National
Council of Teachers of Mathematics.  Items were allocated across specific mathematical
content strands on the advise of math curriculum specialists.  The items are distributed across
five specific mathematical content strands by the three main cognitive processes of
conceptual knowledge, procedural knowledge, and problem solving.  The five mathematics
content strands include the following: 1) number sense, properties, and operations; 2)
measurement; 3) geometry and spatial sense; 4) data analysis, statistics, and probability; and
5) patterns, algebra, and functions.

     Mathematics battery items measuring quantitative and analytic skills in the
kindergarten and first grade include recognizing numbers, counting, comparing and ordering
numbers, solving word problems using simple addition and subtraction with and without the
aid of illustrations or manipulatives.  Other measures of mathematical concepts include
recognizing and solving problems involving graphic and numeric patterns and geometric
relationships.  Items involving the interpretation of picture graphs measure beginning
analysis and statistics skills.

     General Knowledge.  The ECLS general knowledge domain subtest includes items
that measure knowledge and skills in the natural and social sciences.  The test specifications
for the general knowledge domain subtest were mostly developed from recommendations of
content area specialists and are similar to the NAEP frameworks in science (1996) and
history (1994) and published standards for science and social studies.  The content
specifications of this subtest are classified into science and social studies which more closely
reflects elementary school curricula.  

     The ECLS science framework differentiates between two broad classes of science
competencies: conceptual understanding and investigation skills.  Conceptual understanding
refers to both the factual knowledge base and theoretical concepts that children have
developed for the reasons things occur as they do.  As children move up in grade level, the
emphasis in ECLS will be more on the adequacy of conceptual understandings than the grasp
of discreet facts.  Investigation skills refer to children's abilities to form questions about the
natural world, to answer them on the basis of the tools available and the evidence collected,
and to communicate one's answers and describe how one achieved the answers.  Items
measuring investigation skills are further organized into the following subcategories: 1)
collecting data and classification, 2) communication, and 3) drawing inferences.  The science
items are roughly distributed equally among the earth, life, and physical science content
areas.

     Children's knowledge, understanding, and application of scientific concepts is
measured by asking children to name something, describe a process, show a relationship, and
give reasons for how or why something happens or exists.  Examples include the naming of a
body part, explaining the reason for the melting of ice, pointing to one of four illustrations of
objects (i.e., paper clip, glass, sweater, and apple) which goes with the stimulus illustration of
a magnet. 

     The items related to social studies are allocated across five content areas: 1) history,
2) government, 3) culture, 4) geography, and 5) economics.  These are further categorized
into items measuring knowledge and items measuring the ability to analyze and interpret
information about the social world.  The ECLS social studies items for kindergarten and first
grade reflects the concepts and skills within the culture and geography domains commonly
taught in these early grades.  Examples of these items include the naming of historical events
such as Independence Day.  Children's understanding of occupations and roles of workers is
measured by identifying the a dentist from four pictures or orally naming the person who
delivers mail. 

A.6.3 Two-stage administration     

     The range of cognitive skills and knowledge children bring to school varies greatly. 
In a longitudinal study where the measurement of growth is a primary goal, minimizing item
floor and ceiling effects is especially critical.  To achieve reliable growth measures of the
specific cognitive skill domains, direct cognitive measures must be adaptive with item
difficulties tailored to each child's present level of performance.  Special procedures were
designed during the development and administration of the cognitive measures to minimize
floor and ceiling effects which typically distort gain scores. 

     Adaptation of the ECLS test battery subtests is accomplished by a two-stage
procedure involving the administration of a routing test followed by one of several alternative
second stage tests.  At the first stage, all children are administered a short routing test of
specific domain items having a broad range of complexity/difficulty levels.  The routing test
includes between 10 to 12 items with about half of the items in the moderate range of
complexity for the particular school grade and the remaining items divided between very
complex and very simple items.  Performance on this test is used to determine the appropriate
second stage assessment form.  The items in the routing test constitute a set of common items
that will also be used to put all scores regardless of which second stage test form is
administered on the same vertical scale.  Just as the content and skills increase in complexity
and difficulty from one grade to the next, so to will the routing test items change to reflect the
increasing complexity of subject domain-related material.  At each grade, the easiest or
simplest items will be replaced with more very difficult or complex items to allow for the
vertical equating of items across three within grade alternative second stage forms and to
provide the link to equate the scores across the grades.  The routing test shares the majority of
its items with the preceding grades' first stage test making across grade scaling more
efficient.  Consequently, all individuals in a given grade will take a substantial core of
common items which further contributes to the efficiency and reliability of scaling across the
forms and grades. 

     At the second stage, each child will be administered one of the alternative test forms
depending on his or her performance on the routing test.  Items in these second stage forms
are shared across the grades.  The sharing of items among forms and across grades adds
stability to the vertical scaling within and across grades.  The psychometric report for the
National Education Longitudinal Study of 1988 describes this approach in some detail.  The
use of multilevel forms at each grade level for each domain subtest also minimizes the floor
and ceiling effects resulting in more accurate estimates of children's growth. 

     During kindergarten and first grade, the cognitive assessment battery is individually
administered with the use of a computer and an easel.  The child is shown pictures on an
easel, while the examiner reads directions and enters the child's responses in a computer. 
Although assessment is relatively untimed, 45 minutes have been allocated to administer the
battery.  It is expected that each domain subtest will take approximately 15 minutes to
administer both the routing and alternative test forms.  Items for the kindergarten and first
grade are being field tested during the 1996-1997 school year to identify those items which
will be included in the final forms.  A pretest of the two-stage test administration is scheduled
for the fall of 1997 as a part of the Phase I contract.

     The battery items include open ended and multiple choice questions.  Children will be
asked to demonstrate their skills, knowledge, and understanding of relevant domain concepts
using a variety of response formats such as responding orally, pointing to one of four
possible answers that best represents a stimulus, and using manipulatives.

     Some children may understand a concept or mathematical skill but be unable to
demonstrate that understanding verbally.  Thus, items requiring verbal and nonverbal
response formats were incorporated into the item pool to assess conceptual knowledge (i.e.,
general knowledge) and mathematical skills separately from the ability to use the English
language.  Verbal items require children to name or describe something whereas nonverbal
items require the matching or categorizing of objects or entities according to properties and
features.

A.6.4  Group testing in the second grade     

     Item difficulties in the item pool of each of the direct cognitive domain subtests
extend from beginning kindergarten to second grade.  However, the direct cognitive
assessment in second grade shall be group administered.  Consequently, additional work is
necessary to identify items in the pool for group administration.  First, it shall be necessary to
adapt items shared with the preceding grade domain subtest for group administration. 
Second, the items must be compared to ensure that the group administered items are on the
same scale as the individually administered items.  A bridge study or some other means for
equating the items is critical for confirming the comparability of these items.   

     A second grade version of the direct individually administered cognitive assessment
battery is being developed during Phase I.  No work is scheduled on the group administered
version.   The work required to develop a group administered test is described in Optional
Task C. 
 
A.6.5  Teacher cognitive ratings

     Teacher cognitive ratings have been developed for the ECLS that measure teachers'
perceptions of students' academic achievement and dispositions for learning in the three
cognitive subdomains (i.e., language/literacy, mathematics, and general knowledge).  The
primary emphasis of the teacher ratings is to measure change in a student's academic and
behavioral performance within the kindergarten year and between years for grades k through
2.

     The questions ask teachers to make judgments about a student's achievement of
specific skills, knowledge, and behaviors that reflect a representative sampling of the most
recent national, state, and local curriculum standards in the three academic domains and have
also been found to be predictive of achievement in the research literature.  The following
criteria were used in selecting the items: 1) represent the diversity of actual classroom
practices, 2) reflect developmental change across time (i.e., kindergarten through second
grade), 3) have a high level of specificity, 4) easily observable by teachers, 5) reflect a
balance between expressive and receptive language competencies, 6) sufficiently broad to
permit fair evaluations of diverse populations, and 7) minimally obscured by language
proficiency and cultural experiences.  The items are designed to represent average grade level
performance indicators and examples.  While the performance indicators are often constant
across the different years, examples within each item represent grade level distinctions and a
developmental progression across the years.

A.7  Measurement of socioemotional development

     Another critical element of the ECLS is the measurement of children's socio-emotional status.  Children's socio-emotional development is inherently related to their
cognitive development and success in school.  The ECLS socio-emotional assessments focus
specifically on social competence and the skills and behaviors that contribute to that
competence.  

      The primary vehicle for measuring children's socio-emotional performance is an
adaptation of the Social Skills Rating System (SSRS, Gresham and Elliot 1990).  Additional
items have been added to reflect behaviors indicative of approaches to learning, positive
affect, and pragmatic language abilities.  Both teachers and parents will rate children's
prosocial behaviors and social competence skills as well as problem behaviors.  In addition,
teachers and parents will rate the importance of each skill or behavior in their respective
environments (i.e., classroom and home).  Parallel items are asked of both teacher and parent
(e.g., concentration, cooperation, task orientation).  While teachers will rate children's socio-emotional functioning using self-administered survey forms, parents will be asked to rate
their children during a computer assisted interview (i.e., CATI or CAPI).  Ratings of
children's socio-emotional status will be collected from parents and teachers during the fall
and spring of the kindergarten year and in the spring of succeeding years.   

     Information about children's social cognition (i.e., understanding of social problems
and how they can be solved) will be collected from the children themselves.  An adaptation
of the Social Problem Solving Measure (SPSM, Weiss, Dodge, Bates, and Pettit 1992) was
pilot tested in spring 1996 and is being field tested in 1996-97.  The items represent problems
children experience in the following settings: 1) gaining entry in social group settings, 2)
taking turns, 3) responding to physical and verbal aggression, 4) responding to conflicts
involving objects, and 5) responding to disrespect for personal rights.  To measure children's
social cognition, children are shown illustrations of the hypothetical scenarios representing
each of the different problem situations and asked to generate as many behavioral solutions
as possible.  Two sets of identical hypothetical situations have been designed, one for girls
and another for boys.  

     Variations of children's responses to the hypothetical measures may show the
developmental growth of children's problem solving abilities.  In addition, comparing the
results of the SPSM measure with the ratings on the SSRS may provide some insight into the
behavior ratings of the SSRS.


A.8 Physical and Psychomotor Assessment

     Children's developmental status in one domain, including motor development, affects
their development in the other domains.  Because of the functional interrelatedness of the
human body systems, it is necessary to measure children's physical and psychomotor
development.  In the ECLS, direct measures of children's physical and psychomotor status at
kindergarten will include motor ability and coordination and visual motor skills (and
optionally measures of height and weight).  Children's measures of motor ability and
coordination will involve balancing on one foot, hopping on one foot, skipping, and walking
backwards.  Copying shapes using paper and pencil, drawing a child, and block building will
represent the tasks for measuring children's visual motor skills.

Measures of children's physical development will be collected in the fall and spring of the
kindergarten year and in the spring of the first and second grades.  Children's psychomotor
skills will only be assessed in the fall and spring of the kindergarten year. 

     Additional information of children's physical development, physical well-being, and
health will be collected from parents and teachers.    Parents will report on physical activity,
nutrition, illness and disease, health behaviors, and general health and teachers will rate
children's physical activity level and health.

B.  Sample Design

     After an extensive study in Phase I,  a sample design which meets the goals,
requirements and constraints of ECLS, was developed and the plans for selecting this sample
are documented in the Early Childhood Longitudinal Study Design Report. The plan calls
for a dual-frame (i.e., list and area frames) multi-stage stratified design  be used to select the
base-year sample for this study. Geographic areas (e.g., county or group of counties) will be
the primary sampling units (PSUs); schools/kindergarten programs will be  the secondary
sampling units (SSUs); and children will be the ultimate sampling units (USUs). Possible
stratification variables are: census region, level of urbanization, and state for stage 1; school
size, and school type (public vs. private) for stage 2;  and student race-ethnicity for stage 3.
The sample will be selected with probability proportionate to size (PPS) in the first two
stages and with oversampling of Asian and Pacific Islanders in stage 3.  The plan
recommends a sample of 23,500 children who are attending 1,000 schools with kindergartens
(200 private and 800 public schools) in 100 PSUs.  In follow-up surveys, children who were
selected for previous waves will be selected into the current sample with given probabilities.
For the school sample, only those schools that enroll sample students from previous waves
are eligible for the current wave and will be selected with given probabilities.  Although key
sampling issues have been decided and documented in the ECLS Study Design Report, the
contractor is expected to develop many of the operational details such as the sample
allocation among strata that are required to implement the plan.

     The sample for the ECLS shall be drawn from clustered geographical areas.  The
decision to use geographic areas for the first stage of sampling was based on a concern for
economy in face-to-face data collection activities and the expected high rates of mobility for
the ECLS sample over the six year study period.  NCES/ED's current plans call for a first
stage sampling at the level of county or county groupings, school districts, MSAs, or Zip
Codes with the selection of at least two PSUs from each stratum.
     
B.1  Kindergarten sampling frame

     The ECLS contractor for phase II shall develop a sampling frame of all kindergarten
programs in selected PSUs.  Because a complete frame of kindergarten programs from which
to select such a sample does not presently exist, several different sources will have to be used
to develop the frame. Public school kindergarten programs will be identified through the
Common Core of Data (CCD).  Some private school kindergartens may be identified through
commercial data bases such as Quality Education Data (QED), and through the Private
School Survey (PSS), which is conducted for NCES/ED by the U.S. Bureau of the Census.
     
     At the current time, there is some uncertainty about the coverage of kindergarten
programs in private school frames.  Existing frames may not adequately cover the population
of kindergarten children who attend programs operated out of private nursery schools or other
early childhood programs.  The 1995-96 PSS was designed to help to alleviate this problem. 
Special efforts were made to identify the full-range of schools and programs offering
kindergarten.
     
     A dual-frame approach will be applied to solve this problem. The private school
sample selected from list frames will be supplemented by a sample selected from an area
frame. An area frame sample can be produced by  1) sampling  geographic areas (i.e., PSUs),
2) conducting an extensive search for eligible schools/programs within these areas, 3)
matching these eligible schools/programs with those appearing on the list frame, and 4)
sampling eligible schools/programs that do not appear on the list frame for participation.
These are the procedure that have been successfully used by the PSS. The ECLS contractor
for Phase II shall conduct all four tasks outlined above.
     
B.2  Sample  attrition and mobility 
     
     Attrition needs to be examined carefully when designing a sample for a longitudinal
survey. A moderate level of attrition for each single wave may become a severe problem over
the life of the study.  In addition to the well-documented factors for non-response (e.g.,
refusals, unable to locate, and out-of-scope) which are common to both cross-sectional and
longitudinal studies, longitudinal studies are also affected by the mobility of the sample.
     
      Sampled kindergarten children may change schools for many reasons including: 1)
their families move to a new location; 2) children complete the highest grade of the schools
they are currently attending; and 3) the school closes its door or school boundaries change.
Such student mobility may necessitate subsampling in follow-ups in order to curb cost.
Ignoring family movement, data from the CCD indicates that about 13% of public school
kindergarten children change schools after they complete third grade because their schools do
not serve children beyond grade 3. About 20% of public school children change schools after
the completion of fourth grade for the same reason. The percentages of kindergarten children
in private schools who change schools after grades 3 and 4 are 5% and 7%, respectively
according to data from PSS. Transitions of this type, compounded with high family mobility
of primary and elementary school-age children (e.g., data analysis of mobility for the
Prospects Study suggests that family mobility rate ranges from 27.3% for five-year old
children to 18.3% for ten-year old children), will result  in large numbers of children
attending schools other than the ones for which they were selected in the base year. It is
anticipated that the numbers of schools that enroll sample children will expand as the study
proceeds, with high numbers of schools enrolling just one or two sample students.
Subsampling these schools (i.e., schools with few base year sample students) to reduce
survey costs will result in further losses to the student sample.
     
     Mobility rates may change over time. In addition, they differ by socio-economic,
geographic, or demographic characteristics such as family income, school type, locality, and
race-ethnicity. Children with certain characteristics are more likely than their peers to attend 
schools enrolling one or two children sampled in the previous wave. To ensure that a
sufficient number of children for policy-relevant subgroups will be kept, the contractor shall
take account of these factors when devising subsampling plans for follow-up surveys (e.g.,
first through fifth grades).
     
B.3  Oversampling of minority students
     
     Asian and Pacific Islanders will be oversampled in order to meet the precision
requirements of the study.  Experience gained from past longitudinal studies indicates that
oversampling can be accomplished through many different strategies. NELS:88 successfully
implemented a strategy that utilized student rosters within selected schools. Whether this
approach can be used in the ECLS is being explored in the 1996-97 field test.
     
B.4  Freshening of sample
     
     Because kindergarten is not mandatory for all children, it will be necessary to freshen
the kindergarten sample when this cohort enters first grade. Freshening in later grades may
also be desirable. One possible way to achieve freshening is to apply the freshening
procedure which was developed for the first follow-up of NELS:88. This procedure links the
students who are eligible for freshening to the students selected in the previous wave. The
selection probability of the freshened student is derived through these linkages. Other
strategies to freshen students may be proposed.  The contractor shall freshen the kindergarten
sample at first grade.
     
B.5  School-level estimates for later waves
     
     As a part of the ECLS design, school-level information will be routinely collected.
Because schools sampled at the start of the study will be selected according to some known
probabilities, producing school-level estimates will be rather straightforward. This will not be
the case for later follow up studies. During later waves, schools are selected into the sample
because they enroll base year sample children. These schools may or may not have been
sampled during the base year. Sample weights for followup schools will not be as easily
produced as those for the base year schools. Producing school-level estimates from data
obtained from these schools will be more problematic.  A possible solution to this problem
involves the collection of data that allows school followup weights to be produced.  The
contractor is required to provide school weights for the schools included in the first followup
study, i.e., first grade schools.
     
     The situation described above was encountered in the NELS:88 study, as students
moved from eighth to tenth grade. Based on three different models, the selection probabilities
of the first follow-up sample schools for NELS:88 were developed and follow-up school
weights were produced. The ECLS Study Design Report has recommended these three
procedures for ECLS . Other procedures may exist or can be developed.
     
     Regardless of which procedure is proposed for this study, the contractor shall specify
at the sample design stage the data needed to obtain these school selection probabilities. The
contractor shall evaluate the availability and accuracy of these data prior to full-scale
implementation. A contingency plan for obtaining the school weights if the proposed
procedure fails shall be developed. One possible strategy is to develop several procedures and
collect information specified in the procedures from all base year schools. School weights
would then be generated separately for each of these procedures. A decision then could be
made as to the best set of weights to be used for estimation and inference. Other strategies
can be developed. The impact of school weights (on estimates and corresponding developed
standard errors) from all proposed strategies shall be carefully evaluated and documented.

C. Methodological Inquiries and Data Quality

     The data that will be collected through the ECLS will be used by policy makers,
educators, and researchers to understand children's school experience and to guide policy
decisions at local, state, and national levels.  It is important that the information used by these
groups be based on sound research practice and that considerable attention be paid to
identifying the potential sources of error, quantifying this error, and designing techniques to
either reduce the error or minimize its impact on survey estimates.  

     The work to be carried out in support of the ECLS includes a variety of activities that
are directed toward improving the quality of the survey data.  Extensive testing of study
procedures, instruments, and methods was done during Phase I.

     The methods and procedures used during the conduct of the study shall undergo
continued evaluation beyond what is done during the design stage of the study.  NCES/ED
hopes to be able to set aside funds for the conduct of experiments and studies designed to
enhance knowledge of the effects of different design features on the quality of the data. 
NCES/ED is interested in studies that measure the validity and reliability of the survey data
collected from children, parents, and teachers concerning the children's educational
experience and learning environments.  It is also especially interested in studies that assess
the validity and reliability of data that are used to describe children's status on different
developmental dimensions during kindergarten.   During the design and conduct of the
ECLS, it is important that the contractor takes appropriate steps to limit non-sampling errors
as well as to limit errors that are introduced through the sample selection process.  

D. Optional Special Studies and Supplements 

     NCES/ED is supporting the development and implementation of a core study design
that will achieve the goals of the program and provide the data required to answer the major
research and policy questions guiding the study.  NCES/ED plans to explore other ways of
enriching the study.  The study may be expanded in scope or enriched through a variety of
special studies and supplements.  Whether or not these studies and supplements are
incorporated into the final ECLS design will depend on the activities and outcomes of the
Phase I.  Some of these may be supported by NCES/ED depending on the availability of
funds, while others may be sponsored by other parts of the U.S. Department of Education,
other Federal agencies, and foundations.

     NCES/ED is considering expanding the ECLS design in several ways, including:  1)
oversampling specific populations of children, 2) adding questions to one or more
questionnaires in order to explore more fully an area of interest, 3) the use of supplemental
instruments for administration to one or more groups of respondents, 4) adding rounds of
data collection (e.g., fall first grade) in order to study better children's growth over the
critical early school years, and 5) including methodological studies to explore issues of
validity and reliability of the data gathered through traditional survey methods.

     The special studies and supplements that NCES/ED is considering are described
briefly below.  Other topics may surface as a result of the 1996-97 field test and as data from
the early rounds of the study are analyzed.  The final decision as to whether or not to
incorporate one or all of these studies and supplements and their final design specifications
will be at the discretion of the government.  

D.1   Optional Special Studies

     Contingent on the activities and outcomes of Phase I, several special studies shall be
conducted in conjunction with the core ECLS design.  These include:  1) a validation study of
teacher reports concerning children's classrooms; 2) administering an additional
questionnaire to the special education teachers of children with disabilities; and 3) verifying
children's Head Start program participation.

     Validation Study of Teacher Reports of Classroom Characteristics.  An important
goal of the proposed program of special studies is to obtain a clearer understanding of the
relationship between the environments in which learning takes place and children's education
outcomes.  It is important that data collected on the activities and interactions that take place
within the home, classroom, and school be reasonably sound and representative of some of
the key constructs in the ECLS.

     While NCES/ED believes that valuable information can be obtained from different
respondent groups (e.g., parents and teachers) on the activities and interactions that take place
in the home and outside the home, it also recognizes the value of conducting studies to
improve the quality of the survey data.  Of particular interest to NCES/ED is the design of
activities that will collect data to confirm the validity of key classroom measures found in the
teacher self-administered questionnaire.

     The contractor shall design and conduct a validation study of teacher reports of
classroom characteristics.  This study shall focus on measures of key constructs related to
instructional practices.  In selecting an approach, the contractor shall consider the level of
classroom intrusion/disruption and burden that the study may entail.

     Special Education Teacher Questionnaire.  One of the objectives of the ECLS is to
provide rich data to understand children's learning environment at school.  NCES/ED has
already planned to collect information in the study about the children's classroom
environment, experiences, and competencies.  Information on these areas will be collected
from school records and various teacher survey instruments.  

     It appears that the core sample of the ECLS will include sufficient numbers of
children receiving special education services to support independent analysis of this group as
a whole.  And the number of children in this group is expected to increase over the life of the
study as more children with disabilities are identified and receive services.  A proposed
activity for the ECLS is to collect additional information on the learning experiences of
children with disabilities who receive special education services from special education
service providers (i.e., special education teachers).  This supplement will be funded by the
U.S. Department of Education Office of Special Education if it is incorporated into the ECLS
design.

     A survey of the special education teachers of core sampled children with disabilities
will be conducted at each wave of data collection.  The survey shall be conducted using self
administered questionnaires and shall be limited to 10 minutes on average at each wave.  The
frequency at which topics are covered in the questionnaire shall depend on the changing
nature of the topics with some items only needing to be asked once during the initial
interview while others need to be asked at each wave.  Special education teachers of core
sampled children with disabilities would be asked to provide information about themselves
and the sample child to include the following:

     Child specific characteristics 
     Referral history 
     Services provided
     Placement and service delivery
     Instructional approaches and materials, including assistive devices
     Goals and expectations for student
     Consultations with regular classroom teacher 
     Teacher education and experiences

     Verification of Head Start Program Participation.  The Administration on
Children, Youth, and Families (ACYF) is proposing a supplemental study of Head Start
children in the ECLS.  The purpose of this study is to collect descriptive information on Head
Start children's school experiences after they leave Head Start.  ECLS children who attended
Head Start the year prior to kindergarten would be identified.  Once identified, the full
complement of ECLS instruments and procedures would be used with the children in the
same ways as they are with the full sample. 

     Three issues pertaining to the ACYF supplement of Head Start children are being
explored in the 1996-97 field test: 1) can children's participation in Head Start be accurately
reported by schools and parents; 2) should Head Start children be oversampled; and 3) if
oversampling is necessary, what is the best strategy for oversampling Head Start children.

     Because children will not be identified when they are currently enrolled in a Head
Start program, but rather retrospectively after they are enrolled in kindergarten, there is
potential for misidentification of Head Start children.  The misidentification could be an
under-representation, where children who had attended Head Start are not identified, or it
could be an over-representation, where children are identified that actually did not attend
Head Start. 

     In the fall of 1996, at the time the core field test kindergartner sample was selected,
school officials in all 50 field test schools were  asked to identify who among the
kindergarten children in the sample attended Head Start the year before and to identify the
Head Start programs they attended. During the fall field test parent data collection period, 
the parents of kindergartners selected for the ECLS field test were asked a series of questions
about their child's possible participation in Head Start and the center children attended.  

     All unique Head Start programs identified by schools and parents were called to
verify the attendance of children reported by schools or parents as attending Head Start the
year before. The center will always serve as the definitive source for Head Start attendance
against which over-and under-identification on the part of other sources will be measured. 
All students appearing on the center list will be designated as verified Head Start children.  

     The main purpose of this field test activity is to evaluate the accuracy of various
sources of identification.  One question is how accurately can parents and schools identify
students as attending a Head Start program one year earlier?  We hypothesize that parents
will tend to over-report Head Start attendance, whereas schools will under-report attendance. 
Because parents may mistake any Head Start-like compensatory intervention for Head Start,
parents will most likely tend to over-report their child as attending a Head Start program. 
Schools, on the other, will most likely be unaware of who attended Head Start.  The Head
Start center list of attendees will be compared to the parent-generated and school-generated
lists of children attending Head Start programs as a means of evaluating schools and parents
as sources of information on Head Start attendance.  Sources--schools and parents--will be
identified as either tending to over-report or under-report Head Start attendance, and the rate
(or magnitude) of over and under-reporting will be calculated.  

     In the base year ECLS, it may be necessary to verify Head Start children's attendance. 
Whether or not verification is needed and the best way to accomplish this task must await the
outcome of the ongoing field test trials on this issue.

     Fall first grade and summer learning.  The current design of the ECLS includes
two data collection points (fall and spring) during the base year of the study and annual
spring collections thereafter.  Writing about the ECLS design and the conflict between the
descriptive and explanatory goals of the study, Raudenbush (see ECLS Study Design Report)
has suggested that the current design is not the best for answering certain questions about the
growth in children's achievement. Noting the differences in the rates of growth during the
academic year and the summer, Raudenbush suggests that two assessments per year are
needed in order to disentangle the school effects and home effects on children's learning. 
Others have suggested that "real" school begins with first grade, and that the current design
does not capture baseline information on children as they enter first grade.

     At the discretion of NCES/ED, the contractor shall design and conduct a fall first
grade data collection.  The contractor shall administer the ECLS assessment battery (both the
direct and indirect components) to the ECLS kindergarten sample (either all or a subsample
of the children) during the fall of first grade, fall 1999.  In order to capture information on
children's summer learning experiences, the contractor shall design and conduct a 15 minute
parent interview for each sampled child using the same methods being used in the core study. 
This interview shall concentrate on capturing information about children's summer activities
and experiences that might help to explain differences in summer growth curves for
advantaged and disadvantaged children.  The contractor shall design and have teachers of the
sampled children complete a short self-administered teacher questionnaire (15 minutes) that
will be used to collect information about children's fall first grade school/classroom
experience.  These will be the only instruments administered in the fall of first grade.  

D.2   Optional Supplements

     There are several supplements to the ECLS that were designed and field tested during
Phase I which may or may not be fielded during the base year and beyond.   Final decisions
about these supplements will be made following completion of the field test and are
contingent on funding from agencies other than NCES/ED.

     Oversample of American Indian Children.  NCES/ED's preliminary sample
specifications for the ECLS called for the oversampling of black, Hispanic, and Asian
children in order to support separate analyses of these groups.  During Phase I, NCES/ED
supported some design work to look into the feasibility of oversampling American Indian
children for the ECLS in order to support separate national estimates for this group of
children.  Several different methods for increasing the number of American Indian children in
the ECLS sample were investigated.  These included oversampling American Indian children
within the core ECLS sampled schools, selecting additional schools with varying levels of
American Indian concentrations, and combinations of these approaches.  To date, none of
these approaches has been field tested.  These methods are described in the ECLS Study
Design Report.  
   
     Oversampling of Head Start Children.  One of the goals of NCES/ED's early
childhood longitudinal program is to describe the school experience of children who have
participated in different types of early care and education programs.  Although the ECLS is
limited in the degree to which it can achieve this goal, it provides a means for examining the
early school experiences of children who have participated in one of the most comprehensive
early care and education programs for low income children.  Head Start has been providing
comprehensive services in the areas of education, health, and social services to mostly low-income 3- and 4-year-old children and their families for more than 25 years.  Despite this
long history of service, there is very little systematic data at the national level available to
describe the school and family experiences of children once they leave the Head Start
program and enter kindergarten and elementary school.  The study, if incorporated into the
ECLS design, will be funded by the U.S. Department of Health and Human Services,
Administration for Children, Youth, and Families, Head Start Bureau.  

     For the optional longitudinal study of Head Start children the contractor shall
concentrate on children who participate in Head Start programs during the year prior to
kindergarten.  The contractor shall design the study to capture data to describe the school
experiences and progress of children who participated in these programs.  The study is not
intended as an evaluation of the Head Start programs.  The design of the Head Start
longitudinal study shall include the following:

     	    A sample of kindergarten children who were enrolled in Head Start programs
          during the year preceding kindergarten.

     	    Head Start participants shall be sampled from the schools sampled for the
          ECLS.

     	    All Head Start sample participants shall be subject to the same procedures and
          conditions as all other base-year ECLS sample members.  All instruments and
          procedures used with members of the ECLS main sample shall be used with
          the Head Start sample.

     	    The design of the longitudinal study of Head Start children requires that
          members of the Head Start sampled be followed for the same period of time as
          the main ECLS sample.

     During the 1996-97 field test, two issues pertinent to the oversampling of Head Start
children are being explored in more detail: 
     
     (1)  the need to oversample and the sample size needed based on the analytical
          purposes of the ECLS Head Start supplement (ACYF is being asked to specify
          the key analytic questions); and 

     (2)  the efficiency of various rates of oversampling; for example, if based on the
          analytical purposes of the study, too few core ECLS  kindergarten children
          among the 24 who will be selected for the ECLS attended Head Start the year
          before, the efficiency of various rates of oversampling additional children
          from within the ECLS school will be examined.  It is also possible, that the
          analytical needs of the ECLS Head Start supplement could not be met by
          oversampling strictly within core ECLS schools and that additional schools
          would need to be drawn into the study to achieve the requisite sample size for
          addressing the research objectives of the Head Start supplement and this also
          will be examined in the field test.  Of course, the outcomes of the field test
          inquiries into the accuracies of the different sources of information on
          children's participation in Head Start are closely linked to the best method for
          oversampling Head Start participants.

     Depending on the outcomes of the field test, Head Start children may be oversampled
in the ECLS base year. 

     Oversampling Populations of Children with Disabilities.  One of the benefits
arising from a national study of kindergartners is the collection of data on a wide range of
early learning experiences and environments for different groups of children.  The core ECLS
sample of kindergartners will include 1) children with disabilities who have been identified
and are receiving special education related services, 2) children with disabilities who are not
receiving special education related services, 3) children who are at risk but have not yet been
identified, and 4) children who have received special education related services but are no
longer receiving these services.  As children move through kindergarten and grades one to
five, the distribution of children in these different groups will change.  While it appears that
the core ECLS sample will include sufficient numbers of children receiving special education
services to support independent analyses of this group as a whole, a sufficient number of
children to support independent analyses of children with specific disabilities (i.e., blindness,
deafness, orthopedic impairment) is not expected unless such children are oversampled.  In
addition, without augmentations to the design, other key issues related to the population of
children with disabling conditions and their special education needs will not be adequately
addressed in the core ECLS.  The longitudinal study of children with disabling conditions, if
incorporated into the ECLS study design, will be funded by the U. S. Department of
Education, Office of Special Education.

     The longitudinal study of children with disabling conditions shall be designed to
capture data to describe the school experiences and progress of children with disabling
conditions especially children with the following specific disabling conditions blindness,
deafness, and orthopedic impairment.  The design of the study shall include the following:

     	    A sample of kindergarten children who have been identified and are receiving
          special education related services before entering kindergarten.  The sample
          should be of sufficient size to perform the analytic analyses necessary to
          answer the research purposes of the Special Education students longitudinal
          study.

     	    While most children with disabling conditions shall be sampled from the
          schools sampled for the ECLS, sampling sufficient numbers of children with
          low incidence disabling conditions such as blindness, deafness, and orthopedic
          impairments shall require the selection of schools outside the ECLS schools or
          the use of other sampling frames (e.g., national registries). 

     	    To the extent possible, members of the Special Education sample shall be
          subject to the same procedures and conditions as all other base-year ECLS
          sample members.  Decision rules for inclusion developed during Phase I shall
          be applied to determine whether individual children with disabilities are
          excluded from any of the procedures and instruments used with the ECLS
          sample members without disabilities.

     	    When assessing children who receive special education, classroom and special
          education teachers shall be asked whether the ECLS direct assessment
          measures can be used, whether accommodations are needed to administer
          direct measures, and the types of accommodations needed before
          administering assessment.  In addition, any accommodation made for the child
          shall be recorded at the time of the assessment, and score reports involving
          accommodations shall be flagged on the ECLS data files.

     	    Instruments specifically designed for the special education teachers of
          children with disabilities shall be used to collect more information on the
          special education program placement experienced by these children.

     	    The design of the optional study of children with disabilities requires that the
          members of this sample be followed for the same period of time as the main
          ECLS sample.

     Food and Consumer Service.  One of the purposes of the ECLS is to study the
influence of children's physical characteristics and household circumstances on their
behavior, cognitive performance, and school progress.  Thus, a proposed area of investigation
for the ECLS is an examination of the relationship of children's nutrition and participation in
school nutrition programs to their behavioral and cognitive outcomes.  

     NCES/ED has already planned to collect information in the ECLS about children's
participation in school breakfast and lunch programs and their families' participation in
federal assistance programs (e.g., food stamps, WIC, AFDC).  As indicators of behaviors that
have been found to be linked to children's nutritional status and transient hunger, NCES/ED
has also planned to capture information on children's absenteeism, tardiness, attentiveness,
level of concentration, irritability, and hyperactivity from school records and the teacher
questionnaire.  

     An additional supplement to the ECLS would be to collect measures of children's
physical activity, their height and weight, and the food sufficiency situation of their families. 
Information on how often children engage in activities requiring physical exertion would be
obtained from parent and teacher reports; children's height and weight would be measured by
field interviewers at each scheduled data collection; and, data on families' abilities to get
enough to eat would be acquired from the parent questionnaire.

     If this additional supplement is included in the ECLS, it will be funded by the U.S.
Department of Agriculture, Food and Consumer Service (FCS).

E.  NCES/ED Standards    

     NCES/ED has developed and implemented a set of standards that set forth guidelines
to insure the quality of NCES/ED's work.  The contractor shall follow these standards. A
copy of the publication, NCES Statistical Standards and Policies (U.S. Department of
Education, 1992), can be obtained by calling the Statistical Standards and Services Group of
the National Center for Education Statistics at (202) 219-1831.

     

                      III.  SCOPE OF WORK

A.  Overview of Tasks

     This Scope of Work is written in two sections (Core and Optional Tasks).  The first
section describes the core tasks that are required to prepare for and conduct the ECLS
kindergarten fall and spring, and first grade spring data collections.  The contractor shall
perform a series of tasks that will lead to 1) the final design and selection of national samples
of schools, kindergartners, and first graders; 2) the collection of data from children, parents,
school administrators, and teachers during the fall and spring of kindergarten and spring of
first grade; and 3) processing the data and preparing it for public release. The second section
describes a set of optional tasks to be performed in preparation for later ECLS follow up
activities.  If the government chooses to exercise these options, the contractor shall perform a
series of tasks directed toward extending the design of the base year and first follow up
studies to grades 2 through 5.  The contractor shall give special attention to developing the
instruments and procedures that will be needed to assess children's growth in critical
cognitive and noncognitive domains found in earlier rounds of data collection.

B.  Core Tasks to Be Performed

     The core tasks described here represent activities that are required to prepare for and
implement three data collections: 1) fall kindergarten, 2) spring kindergarten, and 3) spring
first grade.  A few of these tasks will be performed only once (e.g., task 1 and 3), while most
will be repeated for each of the data collections.  While the outcome of some tasks will be the
same each time it is performed, the level of effort required to accomplish these outcomes will
not necessarily be the same.  For example, task 10 requires that a set of training materials be
designed and produced and that training sessions be designed for each of the study
components.  However, much of material that is developed for the fall base-year training and
many of the activities planned for the training sessions can be used and modified for later
rounds of data collection. 

     Many of the materials and approaches that are needed to complete these tasks were
developed during Phase I.  The ECLS Study Design Report contains the methods,
procedures, and instruments that are being field tested in school year 1996-97.  While
NCES/ED does not expect major changes to these methods, procedures, and instruments as a
result of the field test, some changes are inevitable. All outcomes of the field test and their
implications for the ECLS design, will not be available until late spring and summer of 1997.

Task 1.   Project initiation and planning

     The contractor shall work closely with the NCES COTR and other NCES/ED staff to
implement the study design developed during Phase I and the approach presented in their
proposal.  The contractor shall keep the project on schedule and within budget.  

     1.1 Final study reports

     Upon award of the contract, NCES/ED will provide the contractor with copies of all
final design reports and documents prepared during Phase I.  The ss ECLS project director
and staff shall review these documents.
 
     1.2 Meet with NCES/ED

     Within two weeks of the contract's start date, the contractor shall meet with
NCES/ED to review the contract's tasks and to discuss issues related to the conduct of the
study.  The contractor shall come prepared to this meeting to discuss any changes in its
technical approach based on its ongoing review of Phase I final reports and documents.   The
contractor shall be prepared to identify areas of concern and to suggest ways of responding to
these concerns.  The contractor shall also be prepared at this meeting to recommend the
members of its ECLS Technical Review Panel.

     The meeting shall be held in Washington, D.C. at the offices of NCES.  In addition to
key contractor staff, the NCES/ED COTR and other NCES staff, it shall be attended by the
Contracts Officer and Contracts Specialist.  In order to get the ECLS phase II contract off to a
smooth and quick start, the contractor shall invite the ECLS phase I contractor's project
director and other key project staff to this meeting. The contractor shall provide NCES/ED
with an agenda at least five working days in advance of the meeting.

Task 2. Technical review panel and outside review

     2.1  Role of the technical review panel.  Studies of this scope, complexity, and
importance require input from a number of individuals and organizations in order to address
the data needs of  policy makers and of those performing policy studies and educational
research.  The contractor shall form and work with a Technical Review Panel (TRP).  The
contractor shall ask panelists to review and comment on such matters as technical design and
implementation, administrative practices, policy and research topics that are appropriate for
the ECLS, and other technical and policy matters that surface from time to time.   The TRP
shall play an active role in the study by reviewing and commenting on overall research
priorities, identifying policy and research questions, providing input about questionnaire and
assessment instrument content, proposing analytical models and methods, reviewing work
plans and their implementation, and reviewing and suggesting modifications to draft reports. 

     The TRP shall not function as a consensus group.  Instead, the contractor shall ask the
individuals on the panel to review and respond to matters concerning the design of the ECLS
and to offer their individual opinions and evaluations on such matters.  The contractor shall
compile and weigh these different opinions and recommendations.  

     2.2  Composition of the panel and meeting schedule.  The TRP shall be composed
of national authorities in those substantive and methodological areas critical to the design and
implementation of the ECLS.  The contractor shall involve and inform the TRP members on
the progress of the ECLS through consultations, reports and meetings.  A list of all persons
serving on the ECLS TRPs during Phase I and others who have consulted on different aspects
of the project before and during Phase I can be found in the Field Test OMB Clearance
Package.

     The contractor shall recommend a schedule of panel meetings in keeping with the
overall plans presented in its proposal.   Panelists will need one day, prior to each meeting, to
review materials and to otherwise prepare for the meeting.  All meetings shall be held in
Washington, D.C. unless otherwise approved by the NCES/ED COTR.   The TRP shall meet
no more than twice a per year for the life of the contract.  

     2.3  Contractor's responsibilities.  The contractor's responsibilities related to the TRP
shall include, but not necessarily be limited to, the following:

          1.   Identifying and recruiting members for the TRP.  Within two weeks of
               the  contract's start date, the contractor shall identify, in writing,
               individuals with the needed expertise for possible membership on the
               TRP.  The contractor shall identify the institutional affiliation of each
               nominee and provide a short biographical sketch highlighting expertise
               and prior work related to the ECLS.  Following NCES/ED review of
               the list of individuals nominated for membership, which shall take no
               more than four weeks, the contractor shall contact each individual to
               solicit and finalize his/her participation.

          2.   Inform the TRP of the progress of the study.  In addition to the regular
               meetings, the contractor shall summarize the progress of the study for
               the panelists at least on a semi-annual basis.

          3.   Prepare all materials and correspondence required for TRP
               consideration and review.  After NCES/ED review and approval, share
               materials with the TRP for review and comment within five days of
               NCES/ED approval.  The entire group of TRP members or individual
               members might be asked to review and comment on a variety of
               contractor produced reports and documents (e.g., field assessor
               training materials, methodology report, questionnaires, and content
               outlines).

          4.   Schedule the meetings with panel members.  All meetings with the
               exception of the first shall be scheduled at least 9 months in advance.

          5.   Arrange for TRP meetings (e.g., meeting space and accommodations)
               and pay all associated expenses.  Non-government panelists shall be
               paid an honorarium plus per diem and travel expenses.   Government
               participants and guests at the TRP meetings will usually include 4-6
               staff from NCES and 1-2 staff from each of the government agencies
               sponsoring components of the ECLS.  The contractor shall also invite
               the ECLS Phase I contractor's project director and other key staff to
               attend these meetings.   

          6.   The contractor shall prepare and submit an agenda for each meeting to
               NCES/ED at least 1 month in advance.  The agenda and any other
               materials that will be provided to the panel members for discussion at
               the meetings shall be delivered to panel members at least 1 week prior
               to the meeting.

          7.   Prepare minutes of the meetings.  The contractor shall tape record the
               meetings and submit a written summary to NCES/ED within two
               weeks of each meeting.  The minutes shall include a summary of the
               discussions (There is no requirement for a verbatim transcript) and
               activities that took place during the meeting, highlighting major issues
               that were raised and decisions made.  The minutes shall identify
               actions (long-term and immediate) the contractor will take to respond
               to issues that were raised and not resolved during these meetings.  The
               minutes of the meeting shall be distributed to panel members and
               invited guests within three weeks of each meeting after NCES/ED
               review.

            8. Arrange and pay for specific work products not tied to a meeting.  For
               example, TRP members might be asked to review and comment on a
               particular draft paper.  Again, non-government panelists shall be paid
               an honorarium.  The contractor shall provide all materials to panelists
               at least one week prior to the meeting. 

            9. Supply NCES/ED with copies of all correspondence and other
               materials exchanged with panelists at the same time they are given to
               the panel members.

     2.4 Other outside review.  In addition to the contractor's TRP, other review panels
may be required from time to time.  For example, during Phase I, groups of subject area
specialists were convened to review the specifications of the ECLS assessment framework
and  to review drafts of the assessment battery.  If such panels of consultants should be
needed again, the contractor shall have the same responsibilities for these panels as are
outlined for the contractor's TRP (see 2.4 above).

Task 3.  IMT/OMB clearance of main survey instruments

     The contractor shall prepare a forms clearance package for obtaining both IMT
(Information Management Team of the Department of Education) and OMB approval of the
ECLS kindergarten and first grade data collections.  The final draft package shall be
submitted to NCES/ED not less than 150 working days prior to the start of the kindergarten
fall data collection.  In preparing the clearance package, the contractor shall follow
instructions for completing Standard Form 83-I and writing the supporting statement required
for Paperwork Reduction Act submissions.  The contractor shall expect that several iterations
of the forms clearance documents will be required before a final draft will be ready for
submission.

     Following NCES/ED review of the draft package, the contractor shall revise the
package incorporating NCES/ED's comments and suggestions and resubmit the package. 
This version of the package will be distributed more widely within the Department of
Education for further review.  The contractor shall expect additional comments and
suggestions within four weeks.  Following this review, the contractor shall revise the package
and submit it for final NCES/ED approval.  Following approval, the contractor shall provide
NCES/ED with 10 copies of the Clearance Package.  NCES/ED will submit the package to
IMT/OMB.

     Much of the material that is required to be included in this package was prepared
during Phase I.  During Phase I, NCES/ED drafted and submitted a Clearance Package for
the 1996-97 Field Test and the base year data collection.  No package has been submitted for
the first follow up data collection, and the contractor shall assume that a Clearance Package
will be required for the base year data collection as well.  NCES/ED will provide the
contractor with a final copy of the Clearance Package that was approved in October, 1996 at
the project initiation meeting (task 1).  

Task 4.  Select school sample, notify sampled schools and secure cooperation  

     4.1  Select kindergarten program sample

     Within 3 months of the contract's start date, the contractor shall select a sample of
about 1,000 schools with kindergarten programs.  The sample shall be selected according to
the specifications contained in  the Final Study Design Report. Any revisions to the sampling
plan developed during Phase I shall be brought to the attention of the NCES COTR and
documented in writing.  Implementation of these revisions shall be contingent on NCES/ED
approval.

     Three weeks prior to the sample being drawn, the contractor shall provide NCES/ED
with the computer specifications that will be used to select the sample along with copies of
other written materials (e.g., instructions, specifications) that document the derivation,
justification, and implementation of the operations that lead to the selection of the
kindergarten program sample, including 1) selection of PSUs; 2) sample allocation; and 3)
stratification.

     4.2   Secure State, district, and school cooperation

     A high rate of participation among the schools in the sample, along with that of the
children, parents, and staff sampled through these schools, must be achieved as a prerequisite
for a successful study.  NCES/ED's standards for high quality data require that nonresponse
bias be minimal.  The contractor shall contact personnel at the sampled schools to gain their
cooperation and participation.  Before contacting individual school administrators, the
contractor shall obtain the endorsement and cooperation of key personnel in administrative
units at all higher levels (i.e., district, state).

     Manuals for securing state, district (and diocese), and school participation were
prepared during Phase I along with several materials (e.g., letters to districts, study brochure,
information sheets) to facilitate this process.  NCES/ED will provide the contractor with final
copies of these materials at the project initiation meeting (task 1).  The approach to securing
cooperation for the 1996-97 field test is described in the Study Design Report.

     4.2.1 Prepare information packet. To gain the cooperation of State agencies, school
districts, and schools, each shall be provided with information about the purpose and design
of the study.  The contractor shall design, develop, provide, and distribute informational
materials.  Information packets shall include: 1) a discussion of the importance of the study,
its purposes, products, scheduled data collection, and sample; 2) provisions for maintaining
anonymity of study participants; 3) the organizations and persons involved in the study; and
4) the benefits to be derived from the study. 

     Different versions of the information packet will be needed for State education
officials, local education officials, school administrators, private school associations, school
staff, and other interested groups.  (A different version shall be prepared for the parents of
sampled children as a part of securing their and their child's participation.)  The content of the
various information packets must be appropriate for each of these audiences.

     The complete draft information packets shall be submitted to NCES/ED for review
and approval within 2 months of the contract's start date.  Once approved, the contractor
shall reproduce the materials needed to obtain the cooperation and participation of states,
districts, schools, teachers, and parents, plus a fifteen percent overage.  All materials used in
securing the cooperation of state, district, and schools (including teachers and parents) shall
include the ECLS logo.  Reproduction shall be accomplished within two weeks after the
packet has been approved by NCES/ED.  The contractor must be familiar with the
requirements of the Government Printing Office for such reproduction; all reproduction shall
be done in accordance with government printing regulations.  

     NCES/ED shall provide the contractor with electronic versions of the ECLS logo. 
NCES/ED shall also provide the contractor with all copies of the ECLS brochure.  These
shall be provided within two months of the contract's start date.

     4.2.2  Obtain State cooperation.  The contractor shall contact State education
department personnel to share information about ECLS and to obtain their cooperation in the
study.  It is also important that the contractor work closely with the Council of Chief State
School Officers' (CCSSO) Education Information Advisory Committee (EIAC).  The General
Statistics and Assessment Subcommittees of EIAC are responsible for monitoring the
progress of ECLS.  The contractor shall contact and obtain the cooperation of state education
agency personnel in advance of contacting personnel in the school districts or schools.

     4.2.3  Secure participation of districts.  Following state-level approval, the contractor
shall contact school district superintendents and personnel in private schools soliciting their
support and requesting their participation in the study.  The information packets sent to
school district superintendents shall list the schools selected in their districts.  The contractor
shall confirm the district's participation via letter.

     4.2.4  Secure participation of schools.  The contractor shall contact all sampled
schools solicitating their participation on a strictly voluntary basis and giving them full
information about the study before asking them to participate.  The contractor shall send
Principals/headmasters of sampled schools a letter stating the purpose of the study and the
voluntary nature of their school's participation.  The contractor shall ask for written
confirmation of a school's agreement to participate.  The contractor shall follow up by
telephone as needed to secure the participation of the school.

Task 5.  Select student and teacher samples and secure cooperation

     5.1  Select school sample of kindergarten children.  After securing the cooperation of
sampled schools, the contractor shall obtain the necessary information from the schools to
enable it to select the sample of kindergartners within the school who will be asked to
participate in the study.  The selection of children shall follow the procedures contained in
the Final Study Design Report, resulting in a sample of approximately 23,500 kindergartners. 
Any deviations from these procedures shall be documented and submitted to NCES/ED for
approval.

     Four weeks prior to the sample being drawn, the contractor shall provide NCES/ED
with written materials (e.g., specifications , instructions) that document the derivation,
justification, and implementation of the operations that lead to the selection of the sample of
kindergartners within the sampled schools. These include, but are not limited to,  the forming
of student lists; within school sampling rates  for core sample and subgroups that are
oversampled; and random starts, if any;  computer specifications for within school sample 
selection.

     5.2  Identify, select and secure cooperation of teachers of children sampled. Once the
sample of kindergarten children is selected from the student roster, the contractor shall
identify and select the teachers of the sampled children.  The contractor shall follow the
procedures found in the Study Design Report, documenting any deviations or problems.  

     The contractor shall approach all teachers (and the parents of sampled children) on a
strictly voluntary basis and give them full information about the study at the time that they
are being asked to participate.  The participation of teachers is critical to the success of the
study given their role as a reporter about the child and the child's classroom environment.

     5.3  Secure cooperation of the children's parents.  Parents of sampled children will be
sent a letter describing the purpose of the study, the voluntary nature of their sons' or
daughters' participation, and asking their permission for their son or daughter to participate in
the study.  Parental consent forms (explicit and/or implicit), developed and tested during the
field test, shall be used to record a parent's consent for their child's  participation.  The
contractor shall work with the school in securing the consent of parents.  About 53 percent of
field test schools required explicit consent.  Public schools were more likely to require
explicit versus implicit consent (55% versus 45%, respectively), and private schools were
about evenly split between those requiring explicit versus implicit parental consent.

Task 6.  Receipt control and reporting

     The design of the ECLS requires the participation of large numbers of schools,
children, teachers, and parents.  The design requires that large amounts of information be
collected, processed, and stored in a short period of time.  A system must be developed and
maintained to keep control of this information and the data collection activities associated
with the ECLS.  The contractor shall design, implement, and document a system to ensure
that each designated sample unit (child, parent, school administrator, and teacher) is properly
surveyed and that all required information is obtained, properly identified, and stored.  The
system shall have means for correcting materials/data found to be incomplete or inaccurate. 
The system shall permit project staff to monitor the flow of information and to produce
weekly reports of the study's progress.  Such a system is referred to here as receipt control.

     6.1 Status reports.  The receipt control system shall support a range of project
reporting activities.  The contractor shall produce weekly summary status reports on the
progress of data collection activities for the NCES/ED COTR using data from the receipt
control system.
The receipt control system shall be used to locate any particular instrument or respondent to
determine the status of the instrument or of the respondent's participation.

     Weekly summary status reports produced through the receipt control system shall
contain detailed information on the initial and final status of all subjects sampled for the
study.  At a minimum, they shall include 1) the number of cases (schools, school
administrators, children, parents, and teachers) sampled; 2) the number of cases for which
contact has been attempted and established; 3) the number of cases that have been completed,
that have refused to participate, that are pending, or that are incomplete; and 4) cumulative
statistics on response rates and followup actions. 

     6.2  Responsiveness to special requests.  The receipt control system shall have the
capability to respond to requests for information on the status of the survey other than those
that are a part of the weekly status report.  At a minimum, the system shall have the
capability to provide information separately on the completion status of the schools, school
administrators, students, teachers, and parents for the classifications used to define the
sample strata for the study.

     6.3  Receipt control system specifications.  At least two months prior to the start of
data collection, the contractor shall have in place and operational (at their own facilities) a
computer-based receipt control system to monitor the flow of data collection activities from
individuals and from educational institutions.  The control system data files shall contain the
identification numbers of all sample members and identification codes of all schools attended
by the ECLS students, as well as information about various data collection events that will be
necessary for monitoring the collection of data.  

     Prior to beginning work on this system, the contractor shall submit its plans for
receipt control to NCES/ED for review and approval.  These plans shall be submitted no later
than 9 months following the contract's start date.

Task 7.  Tracking of kindergarten sample
  
     One of the challenges facing the ECLS is limiting the negative impacts of the
mobility of the sample of kindergartners.  To help maintain a high response rate, the
contractor shall develop methods and procedures for collecting locating information on each
ECLS child/parent sample member.  The contractor shall create and maintain a tracking
system data base.  At a minimum, the data base system shall include the following: 1) student
identification number, 2) parent(s) names, 3) home address, and 4) names, addresses, and
phone numbers of two other individuals who would always know the whereabouts of the
child.  

     The contractor shall also have a set of proven procedures for locating sample
members who change schools/addresses between data collections (e.g., fall kindergarten to
spring kindergarten).  The contractor shall locate all child/parent sample members who
change schools/addresses between data collections using these procedures and the tracking
system data base.
  
     The contractor shall submit its specifications for the tracking system to the NCES
COTR within six months of the contract's start date. 

Task 8.  Conduct pilot test

     The contractor shall conduct a pilot test of the ECLS in the spring of 1998.  This test
shall be used to test and evaluate the contractor's technical approach to: 1) securing State,
district, school, parent, and teacher cooperation; 2) sampling schools, children, and teachers;
3) conducting child assessments in schools; 4) training field staff; 5) managing the flow of
materials and data and reporting on the progress of data collection; and 6) testing the use of
any technologies that will be used in schools (e.g., CAPI sampling and assessment programs)
during the base year study and for capturing and processing data.

     The contractor shall include no more than 12 schools in the pilot and shall assess
between 240-300 children.  Twelve administrator questionnaires shall be administered as
well as the appropriate number of teacher questionnaires and teacher ratings.  Schools shall
be selected from 4 geographic sites, one of which shall be the Washington, DC metropolitan
or surrounding area.  The contractor shall follow as closely as possible the design
specifications of the ECLS when conducting the pilot.

     The contractor shall prepare a 10-20 page report that describes the pilot study, what
was learned from this activity, and identifies any recommended changes to the contractor's
technical approach based on the pilot experience.  This report shall be submitted within 6
weeks of the end of pilot data collection and field operations.

Task 9.  Reproduction of instruments and acquisition of materials

     The contractor shall reproduce or acquire all the materials, and program all the
systems necessary,  to support the collection of data from children, school administrators,
teachers, parents, and school records.  This shall include, but not be limited to the school
administrator questionnaire, the teacher questionnaire, the parent questionnaire, school record
retrieval, and the ECLS direct and indirect assessment instruments.  

     9.1 Reproduction of instruments.  The contractor shall reproduce or arrange for the
reproduction of the school administrator questionnaire, teacher questionnaire, teacher
checklist, and school record retrieval forms.  The contractor shall also have responsibility for
reproducing all printed materials associated with the ECLS assessment battery (both the
direct and indirect instruments).  A sufficient number of copies shall be reproduced to meet
the needs of each data collection, plus a 10 percent overage.  The contractor must be familiar
with the requirements of the Government Printing Office for such reproduction; all
reproduction shall be done in accordance with government printing regulations.  

     The materials used in the ECLS assessment battery that require reproduction are: the
assessment easels for the three subtests of the direct cognitive battery  (language/literacy,
mathematics, and general knowledge), instructions for conducting the psychomotor
assessment battery, the scenarios found in the Dodge direct socioemotional measure,  teacher
ratings of children's social skills and cognitive skills.  The assessment easels shall be
reproduced in color.  All other printed materials shall be reproduced in black and white.       

     The complete ECLS assessment battery for children includes 

         Cognitive assessment:  3 sets of test plates spiral bound into easels: one for
          each of the cognitive domains.  Each easel shall be approximately 8« inches
          wide by 7 inches high.  The covers of the easel shall be constructed from a
          medium of sufficient durability to withstand multiple use.  The back cover
          of the easel shall be constructed so that it shall support the test plates in an
          upright position during administration.  On the front cover shall be printed
          the ECLS logo across the top margin and the subtest domain in the center. 
          The easel shall include the first stage routing test and the three alternative
          second stage test items.  Each test will be separated by tab markers.  The
          test items shall be printed using 4 color process on 80 # white o/s smooth
          stock.  The notice of copyrighted items shall be printed on the first fly
          sheet.  Each test item shall be printed on separate fly sheets. 



         Practice booklet: 1 set of test plates bound into an easel containing a sample
          of  items representing the different types of responses required in the
          assessment such as pointing and giving an oral or spoken answer.

         Laptop computer pre-loaded with 1) a software package for recording
          assessment battery responses and 2) a communications package for
          transmitting completed cases to the contractor's central office computers. 
          Battery packs and extension cords with surge protectors are required.
     
         Socioemotional assessment: 1 set of test plates of the hypothetical situations
          printed on 8"x10" spiral bound paper with separate versions designed for
          girls and boys.
     
         Manipulatives:
          - 20 Unifix cubes or equivalent, all of the same color.
          - 10 1"x1" cubes, all of the same color
          - primary pencil and a sheet of 8«x11" unlined white paper
          - 5 shapes printed on 3x5" card stock
          - 3x5" card on which a graphic illustration of a figure made of squares to
          represent the cubes is printed
          - a two-part box 7 « x 5 x 2 3/4" rigid paper board box; box top is die cut
          on one short end with crescent shaped 2" radius cut.

         Student rating booklets: Separate student rating booklets for the measures of
          children's socioemotional and cognitive behaviors and skills shall be printed
          on 8«"x 11" paper.  All pertinent information including ECLS logo,
          student and teacher identification, OMB approval, instructions, and
          copyright notice when appropriate shall be printed on the front cover.  Text
          shall be printed on both sides.

     9.2 Acquisition of materials and equipment.  The contractor shall be responsible for
acquiring all materials and supplying all equipment needed to administer the ECLS data
collection instruments. 

     9.3 CATI/CAPI Programming. Three components of the ECLS design require the use
of laptop and CATI/CAPI technologies: 1) within school sampling of children; 2) parent
interview; and 3) the direct child assessment.  The use of these technologies increases the
quality of the samples and the data that are collected from these samples, and helps the ECLS
achieve its goal of collecting baseline data early in the school year from children, their
teachers, and parents.

     The design of the ECLS requires that children be sampled within schools with
different probabilities of selection, and that their parents and teachers be identified and
sampled quickly for the parent interview and teacher survey.  In the spring 1996 pilot and
again in the 1996-97 field test, laptop computers were used by field staff to assist in the
selection of  these samples.  NCES/ED intends to continue to use this technology in this way
in Phase II. 

     Parent interviews will be conducted by telephone using CATI.  The exceptions to this
are the parents of ECLS children who do not have telephones.  For these parents, interviews
will be conducted in their homes using CAPI.  

     The ECLS direct assessment battery uses a CAPI-assisted approach.  Child assessors
use standard paper easels and other printed materials when asking children a range of
questions pertaining to language/literacy, mathematics, and general knowledge in science and
social studies.  However, they are assisted by a laptop computer that has the entire battery
programmed.  Assessors read from the screen while showing the child stimulus (and
response) pages on the easel or manipulating other materials.   All of the children's responses
are entered into the laptop program.   The contractor shall continue to use this technology and
approach when administering the ECLS assessment battery in the base year and first
followup studies.  The use of this technology will be even more critical in the main study
given the use of a two-staged assessment battery.
  
     Because of the complexity of the CATI and CAPI programming of the parent
interviews and child assessments, the contractor shall develop the procedures and materials
necessary to insure that all program specifications are correct and complete.  In the
development stage it may be helpful to create flowcharts and evaluation procedures for both
the CATI and CAPI instruments.

     The contractor shall prepare hard-copy versions of the CATI and CAPI monitor
screens for each of the parent interview and child assessment items.  At a minimum, these
screens shall include:

(1)  the item number and text;
(2)  all displays that are to appear in the body of an item (e.g., names, wording
     alternatives);
(3)  all information that is being provided to the interviewers/assessors to assist them in
     conducting the interviews/assessments accurately and smoothly (e.g., respondent's
     name);
(4)  information that identifies the next item to be asked for each response; and,
(5)  the response range specifications.

Following review by the contractor's project staff, copies of the CATI and CAPI screens
shall be submitted to NCES/ED for review no later than 17 weeks prior to the start of data
collection.  NCES/ED will provide comments within two weeks, and screens revised in
response to comments shall be submitted with two weeks of receipt of NCES' comments.

Task 10.  Hiring and training data collection staff   

     Several different staffs are required to implement the ECLS design.  Field staff are
needed to conduct preliminary visits to the sampled schools to select the children who will
become a part of the ECLS longitudinal sample and to administer the parent interview when
no telephone is available.  Child assessors are needed to administer the ECLS assessment
battery to children in their schools.  Telephone interviewers are needed to conduct the parent
interview, and to perhaps follow up with teachers and administrators who fail to respond by
mail (see task 11).  Each of these activities requires a unique set of skills, skills that are
developed through a combination of prior experiences working on other similar projects and
training tailored to the requirements of the ECLS.

     10.1  Development of training and evaluation materials.  In order to ensure that the
data collection staff assigned to the ECLS have the skill levels necessary to perform
successfully the demanding tasks associated with conducting preassessment visits to schools,
child assessments, and parent interviews, the contractor shall design and implement
comprehensive training programs taking into account the procedures and materials developed
for training staff for the field test.

     Prior to fully developing any training materials, the contractor shall submit an outline
of its interviewer training program to NCES/ED for review.  This outline shall be submitted
to NCES/ED no later than 5 months after the contract's start date.  This outline shall include:

(1)  a training program agenda that identifies the formats that will be used to conduct the
     training, the topics to be covered (e.g., study background, sampling children,
     overview of instruments, survey topical component), and the length of the training;

(2)  an outline of the materials that trainees will be provided;

(3)  the contractor's plan for evaluating the training program and the performance
     of individual trainees during training; and,

(4)  a preliminary schedule that identifies when and where the training will occur.

Following NCES/ED review and approval of the interviewer training program outlines, the
contractor shall develop and submit draft copies of all training materials to NCES/ED for
review at least five (5) weeks prior to the start of the first training session and six (6) weeks
prior to the start of data collection.  After making the changes brought up by the review, the
contractor shall submit 5 copies of the final training materials to the NCES/ED COTR at least
five (5) working days prior to their being distributed to trainees.

     Along with the training plan, the contractor shall submit a plan which shows how it
will monitor all data collection activities to assure that consistent high quality data is
collected throughout the data collection period.  This plan shall include:

(1)  the contractor's awareness of interviewer responsibilities under the Privacy Act of
     1974 (5 U.S.C. 552a), the Privacy Act Regulations (34 CFR Part 5b), the National
     Center for Education Statistics Act of 1994 (P.L. 103-382, Section 9007), and
     NCES/ED Standards and Policies; and,

(2)  a description of the evaluation and correction methods the contractor shall use
     immediately after training and over time to a) assure quality control, b) maintain
     standards, c) identify poor performance, and d) either correct such performance or
     release poor performing individuals so that only high quality data collection personnel
     are retained and utilized.

(3)  a description of the types of technologies (e.g., video taping) that will be used to
     complete at least some of these evaluations.

     10.2  Recruitment of data collection staff.  The contractor shall recruit and hire staff
to complete the data collection within the time constraints imposed by the project schedule. 
Because data will be collected from large numbers of young children, the contractor shall
give special consideration to recruiting field staff with prior experience working with young
children.  

     Moreover, because Hispanic and other minority children will be oversampled in the
ECLS, the contractor shall have data collection staff who are qualified to interact with
children and conduct telephone interviews in languages other than English.  At a minimum,
the contractor shall have field staff and telephone interviewers who are fluent in Spanish.

     Confidentiality requirements in NCES/ED contracts, discussed in more detail in
Section IV, mandate that data collectors complete a sworn Affidavit of Nondisclosure (see
Appendix D).  In the presence of a public notary, these affidavits shall be signed by all
project staff, not just field staff and telephone interviewers, having any access to ECLS raw
data.

     10.3  Conducting data collection staff training.  The contractor shall have
responsibility for all training of field staff and telephone interviewers.  All training shall take
place at the contractors' facilities or at facilities arranged for by the contractor.  The
contractor shall be responsible for assuring that all staff assigned to the project successfully
complete the training program as specified in the training plan.  The contractor shall have
responsibility for the production of all training materials and shall have sufficient materials
available for all staff trained during the conduct of the project.  

     Moreover, successful conduct of the ECLS requires that all data collection staff
receive standardized training.  Due to the large numbers of individuals that will need to be
trained, training may have to occur across multiple sites.  The training program shall use
video tapes and other technological capabilities, as needed, to insure that all staff receive the
same training in their respective assignments.

     10.4  Conducting supplemental training.  Because of the complex nature of the ECLS,
the contractor shall develop and implement procedures for conducting supplemental training
as problems are encountered.  Because of the large number of staff who will be working on
the ECLS during the data collection period, it is important that any decisions that are made
about the conduct of the survey be disseminated immediately and clearly to all staff.  The
contractor shall have procedures in place that will ensure that all parties involved with the
collection of data use the same solutions to and interpretations of problems that arise during
the course of conducting the study.

Task 11.  Collect kindergarten and first grade data

     Data collection for the ECLS base year shall occur twice, at the beginning and near
the end of the school year.  First follow up data shall be collected during the spring of first
grade. The contractor shall conduct the base-year study and first follow up study
implementing those procedures and using those instruments set forth in the ECLS Final
Design Report as modified based on the outcome of task 1. Base-year and first follow up data
shall be collected from the administrator of the school attended by the sampled
kindergartners, their teachers, and parents.  Direct assessments of children's cognitive and
social development (A final decision on using a direct measure of children's social
development will be based on the outcomes of the 1996-97 field test.) shall be administered 
individually in the schools attended by ECLS children by staff employed and trained by the
contractor. 

     11.1  Data collection schedule.  The contractor shall prepare, using critical dates
specified by NCES/ED (see A.5 Data Collection Schedule ),  a detailed schedule for
completing all data collection associated with the ECLS base year and first follow up studies.
The contractor shall pay special attention to the possible effects that the timing of data
collection may have on the key study measures, and especially on various child development
measures.  A copy of the contractor's data collection schedule shall be submitted to
NCES/ED for review and approval within six months of the contract's start date.  No data
collection shall begin prior to NCES/ED's approval of the schedule.

     11.2  Quality control procedures.  The contractor shall develop and implement a set of
quality control procedures that shall assure the collection of high quality data throughout the
data collection period.  Quality control shall be maintained through a series of both statistical
and project supervisory procedures.  The contractor shall have in place procedures for
identifying and correcting errors resulting from both the design and poor implementation of
the design.  Problems that are identified shall be addressed immediately and consistently. 
Procedures shall be in place from the beginning of training until the completion of data
collection that will require that all parties involved with the collection of data use the same
solutions to and interpretations of problems that arise during the course of the survey.

     The contractor's approach to data quality may include the conduct of experiments or
special studies.  The contractor shall design and conduct no more than one or two
experiments or special studies that would significantly enhance our knowledge of the effects
of different design features on data quality and the validity and reliability of the responses
given by study participants.

     11.3  Progress reports.  Throughout the data collection period, the contractor shall
provide the NCES/ED COTR with weekly progress reports.  In addition to the computer-generated reports described under Task 7, these weekly reports shall identify any problems
encountered and either describe how these problems were resolved or recommend alternative
ways of resolving these problems.  These weekly progress reports are in addition to the
monthly reporting requirements described in Section IV (Study Requirements, A. Reporting).

     Any problems encountered that either have consequences for the project's budget or
time schedule shall be brought to the attention of the NCES/ED COTR in these weekly
progress summaries.

     11.4  Nonresponse followup and refusal conversion.  The ECLS, like any other study,
will encounter initial refusals and other types of nonresponse.  The potential for nonresponse
and its effects on estimates for child, school, teacher, and parent characteristics shall be given
special consideration in the contractors' conduct of the ECLS.  The contractor shall develop
and implement procedures for converting nonrespondents to completed interviews.   

     For longitudinal surveys such as the ECLS, NCES/ED has established a target
response rate of 90 percent for each wave of data collection.  The contractor shall develop
and implement  methods and procedures that shall increase the chances of meeting and
possibly exceeding this rate.  

     All nonresponse followup and refusal conversion efforts undertaken by the contractor
shall be documented.  The contractor shall provide NCES/ED with the decision rules to be
followed if a sample case (i.e., institution or individual) refuses to participate.  Any
procedures for sample replacement shall be submitted to NCES/ED for approval prior to their
implementation.  

     11.5 Control of instruments and other data.  The ECLS cognitive assessment battery
was constructed by creating a pool of items from existing instruments and developing new
items.  NCES/ED and the contractor for Phase I of the ECLS have secured permissions from
various test publishers (i.e., licensors) to use individual items under a "Conditions of Use"
negotiated with each publisher (see Attachment X).  One of the conditions requires that all
items made available to NCES/ED by each publisher will be protected at all times. 
Consequently, the contractor shall protect the security of the cognitive assessment battery by
restricting the number of copies published to the number needed to conduct the study and
maintaining a record of the location of each copy of the assessment battery instruments.  A
copy of this record shall be made available to the COTR upon request.  Under no
circumstances shall copies of the assessment battery instruments be distributed or
loaned out without the permission of the NCES/ED COTR.  The contractor shall
employ strict procedures for ensuring that all project staff adhere to this requirement.

     Another Condition of Use negotiated with the licensors charges NCES/ED to honor
all requests by the licensor to protect the licensor's proprietary rights to items at all times by
publishing the items marked by a proper notice of copyright.  The contractor shall include
proper notice of copyright for all items from existing instruments in publishing the cognitive
assessment battery instruments (see task 9).

     The contractor shall not include item-level data in preparing public use data files. 
Reports shall only contain normative and criterion-referenced status and gain scores for
ECLS participants using IRT-based formulas.  Individual ECLS assessment battery data and
individual item data shall not be released without the permission of NCES/ED. 

     The contractor shall submit annual statements of use to test publishers who are
receiving royalties for the use of their items in the ECLS assessment battery.  The contractor
shall pay the royalties associated with this level of use as previously agreed to.  This shall be
done no later than January 31 of each year.  The contractor shall provide NCES/ED with a
copy of each statement of use and royalty payment.

Task 12.  Data processing

     The contractor shall have responsibility for processing all data collected for the
ECLS.  This shall include editing (both manual and machine) study data, correcting data that
fail the edit checks, coding of open-ended responses, and converting the raw survey data to a
computerized form.  

     12.1  Coding and editing.  At least ten (10) weeks prior to the start of data collection,
the contractor shall submit its plan for editing and coding the data collected as a part of the
ECLS.  The contractor's plan shall be submitted as a memorandum to the NCES COTR and
shall clearly identify 1) the type of edits that will be performed for the different items of data,
2) the edit specifications for each data item, 3) the response that will follow failed edit
checks, and 4) the approach that will be used to code the ECLS data, including coding
specifications for open-ended items.  Its plan shall include edit specifications for CADE,
CAPI, and CATI.  Approval of the plan by NCES/ED is required prior to its implementation.
  
     Following NCES/ED approval of the plan, the contractor shall develop and test the
methods and procedures needed to carry out the plan.  Responses to question items that fail
edit checks and require followup shall be resolved promptly according to approved
procedures.

     The contractor shall maintain a software system that can be used to update, recode,
correct, or temporarily modify data values, and to modify edit specifications.  The system
shall be capable of providing the hard copy printout necessary for any respondent call-back.  
  
     Editing of the data shall be done by machine, but limited manual scan edits to check
for completeness may be required.  Coding of responses to all open-ended questions shall be
performed by the contractor (about 5-10 open-ended items per survey form).

     12.2 Data entry/conversion. The contractor shall submit a plan for converting raw
data into machine readable format by no later than six weeks before the start of data
collection.  The plan shall describe the proposed structure and specifications for these data
files for both draft and final deliverables.

     Prior to the base year data collection, the contractor shall design, establish, and
implement the procedures to transform the data into a computerized form that can be used by
a variety of standard statistical software packages (e.g., SAS and SPSS for Windows).  After
data collection begins, the computerized data shall be edited and errors corrected until the
computer files contain only verified and accurate records.  For NCES review, the contractor
shall provide separate ASCII data files with SPSS for Window code on CD-ROMs.  These
files and all documentation (including linking IDs) needed to access and read the data stored
in each file shall be delivered within four (4) months of the end of data collection activities. 
The following files shall be delivered:
  
     Base year files
     
     1.  Child file 
     2.  Principal/School file
     3.  Parent file
          4.  Teacher file 

     First followup files

     1.  Child file (including student record data)
     2.  Principal/School file
     3.  Parent file
     4.  Teacher file

     Final data deliverables shall consist of CD-ROMs with electronic codebooks that
allow users to tag variables and create SAS and SPSS for Windows code files.   For the final
file deliverables, the data shall be provided as megafiles (e.g., merged child, parent, school,
and teacher files) similar to those contained on the 1996 release of the NELS:88 base year
through third followup release  files.  The plan shall not be implemented without the approval
of NCES/ED.  For this contract, final file deliverables include:

          Merged fall and spring base year kindergarten files --  These merged files shall
          be delivered within eight (8) months of the close of spring kindergarten data
          collection activities.

          Merged base year and first followup files --  These merged files shall be
          delivered within eight (8) months of the close of spring first grade data
          collection activities.

          Each of these merged megafiles will be accompanied by user's manuals that
          are similar to those produced for the second follow up of NELS:88 and the
          1993 and 1995 National Household Education Survey.

     12.3  Quality control.  The contractor shall develop and implement a set of quality
control procedures that assures high quality data processing.  

     12.4  Data processing status report.  The contractor shall allow for NCES/ED
inspection of the records and results of the data editing and follow-up actions.  These records
shall be delivered to NCES/ED on request.  A monthly status report of the data editing and
corrective actions shall be provided to NCES/ED.  These status reports shall include weekly
and cumulative summary statistics by instrument on the volume of work completed; edit
rejection rates for various types of items and respondents; and the status and method of
corrective action taken. 

     12.5  Documentation.  All data processing activities and procedures shall be clearly
documented.  This documentation shall become part of the final ECLS methodology report
(task 14). 

     12.6  Base-year and first follow up documents and materials.  To ensure a smooth
transition from the base year and first follow up data collections to the later followup data
collections, the contractor shall coordinate with the second followup contractor and provide
copies of all documents and materials developed for the base year and first follow up of
ECLS. These shall include, but not be limited to hard and electronic copies of instruments
and other project deliverables, and data on CD-ROMs accompanied by appropriate
documentation. 

     12.7  Public and proprietary data files.  For purposes of disseminating the ECLS data,
the contractor shall prepare and deliver to NCES/ED both public and proprietary data files on
CD-ROMs with electronic codebooks. The construction of these files will be modeled after
those in NELS:88 second follow up and the 1993 and 1995 National Household Education
Survey.

     The contractor shall conduct a thorough confidentiality analysis of the ECLS data
when preparing the public use files.  This analysis shall ensure that the confidentiality
provisions contained in PL 100-297 are fully complied with. To protect the privacy of
respondents as required by PL 100-297, respondents with high disclosure-risk will be
identified.  Data that can potentially be used to identified these respondents will be masked.

     There are many different strategies that can be used to identify high disclosure-risk
respondents. For a multi-stage school-based survey like ECLS, one strategy is to identify
kindergartens that are considered to have high disclosure-risk. By matching data available on
school universe files (e.g., CCD, PSS, QED ) with data on the ECLS school sample file, the
distance between a school as it appears in the universe file and the same school as it appears
in the sample file can be calculated. Based on these distance measures, sample schools with
small distances are considered as having high disclosure-risk and are identified.  School level
data that can be used to identify these schools are masked. This method of identifying high
disclosure-risk schools has been applied successfully to other NCES longitudinal studies
(e.g., NELS:88). 

     Within the high disclosure-risk school, some individual-level variables (e.g., student,
teacher) in the sample data file may provide information that can be used to identify
individual respondents. These variables and the individuals with high risks of being disclosed
shall be identified. Their responses shall be masked. 

     In addition, disclosure-risk can be increased as more information is collected during
future rounds of data collection and as this information is added to the data records of the
sample members. The contractor shall also consider this type of disclosure risk when
planning and conducting confidentiality analysis for the ECLS.

Task 13.  Briefing materials. 

       Throughout the course of the project, NCES/ED shall provide interested individuals
and agencies with information about the nature, findings, and progress of the ECLS.  The
contractor shall support these activities by developing up to six sets of briefing materials over
the life of the contract.  These materials shall be prepared only for NCES/ED's use.  Each set
of briefing materials shall be no more than 10 typewritten pages (double-spaced) with no
more than 10 accompanying graphs and/or tables.  Requested materials may include
overhead slides (transparencies or Power point files) for making presentations regarding the
ECLS.  Within two weeks of a request, a set of briefing materials shall be submitted to
NCES/ED for review and comment.  The contractor shall make necessary revisions to the
materials based on this review and resubmit the materials within one week of receipt of
comments. 

Task 14.  Methodology report

     The contractor shall submit a methodology report that documents the entire project
including: a description of the sample design, instrument development with particular
attention to the ECLS assessment battery and data collection procedures.  Because the ECLS
represents an important NCES/ED initiative that is expected to receive much attention and
scrutiny, it is critical that this technical report describe the study design and all procedures
used to conduct the study.  Special features of the study shall be described in detail and any
problems encountered either during the design or implementation of the survey identified. 
Changes that were made to the design and/or procedures developed during Phase I in
response to unanticipated problems shall be fully documented.  The report shall include
major findings pertaining to the methods used and/or experimented with during the conduct
of the study and discuss their implications for future follow up data collection activities. 
Information on the design and findings of methodological studies/experiments directed
toward assessing and improving the quality of the ECLS data shall be incorporated into the
methodology report.  The design and findings of these studies/experiments shall be presented
in a manner that will allow them to be used as stand alone documents.

     Much of the information that is presented in this report will be drawn from the Phase
I Final Study Design Report.  The task here is to update that report.

     The final methodology report shall provide an in-depth description of the  rationale,
development, and psychometric properties of the ECLS assessment battery.  A separate
section of this report shall address  whether the psychometric objectives of the assessment
battery were met or exceeded.   Using the Phase I Final Study Design Report as a starting
point, the contractor shall document and expand on the following assessment topics:

Assessment Specifications

Aims and objectives of ECLS assessment

À"À Description of the two stage testing methodology
À"À Description of the assessment development process
À"À Difficulty levels of items
À"À IRT scaling for longitudinal measurement
À"À Psychometric goals of the ECLS  assessment battery
À"À Specifications for individual assessment instruments
À"À Matching test content to curriculum

Psychometric Analysis results

Reliability of the ECLS assessment battery

À"À Item statistics by gender and racial/ethnic groups
À"À Differential Item Functioning (DIF) for race-ethnicity and gender
À"À Factor structure of the ECLS assessment battery
À"À Performance of racial/ethnic and gender groups on the ECLS  assessment battery
À"À Proficiency level subscores by subgroups
À"À Item Response Theory (IRT) parameters for the ECLS assessment battery
À"À Assessment scores on both public and privileged released CD-ROMs

  The contractor shall submit a draft of this report within 10 months of the end of the
spring first grade data collection.  The contractor shall revise the draft report based on the
comments and suggestions of NCES/ED staff.  A revised report shall be submitted to
NCES/ED.  The revised version will be extensively reviewed within the Department of
Education.  A final methodology report which incorporates the comments and suggestions
made by NCES/ED and others within the Department of Education shall be prepared and
submitted.  Once NCES/ED has approved the final report, the contractor shall submit 20
copies of the report to NCES/ED.  In addition to hard copies, the final methodology report
shall be delivered on floppy diskette.  The electronic version of the report shall be delivered
to NCES/ED as a Word 7.0 or later file.  No information contained in this report shall be
released without NCES/ED's prior approval. 
Optional Tasks, Special Studies, and Supplements

  In addition to the core tasks (tasks 1 through 14), this procurement includes: 

  1) 3 optional design tasks (tasks A through C);
  2) 4 optional special studies; and 
  3) 2 optional supplements. 

C.  Optional DesignTasks to be Performed

  Design activities during Phase I were devoted to 1) developing the overall design
parameters of the ECLS (kindergarten through grade 5); 2) developing the kindergarten
sample design and procedures for freshening the sample at first grade; and 3) creating the
specific instruments (questionnaires and assessments) and procedures that would be required
for the first two data collection years, i.e., kindergarten, both fall and spring, and first grade,
spring only.  In addition, many of the instruments that will be needed if a second grade spring
data collection is once again included in the ECLS design were developed during Phase I.  

  Before NCES/ED can field the later rounds of data collection, additional design work
is needed on the designs for second, third, fourth, and fifth grades, extending and building
upon the work that was begun in Phase I.  This design work will be  aimed at developing 1) a
final sampling plan for later rounds of the study; 2) an ECLS assessment battery that extends
beyond grade one,  and reflects the content and process emphases of the ECLS assessment
framework for later years; and 3) school, teacher, and parent questionnaires for later
elementary school grades, and student questionnaires beginning with third grade.  

  NCES/ED has identified three (3) optional tasks that support this design activity.  All
work required under each of these tasks must be completed within 18 months of award of the
option.  Completing these tasks on schedule is critical, in order for them to be field tested one
year prior to their full-scale implementation.  Large-scale field testing of the instruments and
procedures is not a part of this procurement; however, smaller pilot studies and cognitive
research is expected.

Optional Task A.  Develop instruments and procedures for the second through fifth grades

  Extensive conceptual work has gone in to designing the survey instruments and data
collection procedures for kindergarten and first grade (and second grade to a certain extent). 
If the option to design the survey instruments for the second through fifth grades is exercised,
the contractor shall extend this work and develop survey instruments for students, parents,
teachers, and school administrators for the second through fifth followups (grades two
through five).  This task pertains to the development of all instruments other than those
comprising the ECLS assessment battery.

  A.1  Identify key research and policy issues.  The contractor shall review the research
and policy issues previously identified for the kindergarten and first grade years (see ECLS
Study Design Report).   Based on this review, the contractor shall submit a memorandum
which identifies both the issues that extend through to the fifth grade as well as any new
research and policy issues which were not previously included but are relevant to any or all
of grades two through five.  Moreover, in the memorandum, the contractor shall prioritize the
issues and questions to be addressed by the ECLS and clearly state the types of analyses that
the ECLS will be able to support.  

  A.2  Review content outline.  The contractor shall review the content coverage of the
ECLS (see ECLS Study Design Report).  Based on this review, the contractor shall provide
in a memorandum any updates or modifications needed for the second through fifth grades.

  A.3  Develop survey instruments for the second through fifth grades.  The contractor
shall develop all survey instruments and data collection forms needed for the second through
fifth grades.

  The survey instruments shall contain items that will lead to accurate and reliable
measurement of the topics listed in the ECLS Study Design Report and extend the
information collected in the kindergarten base year and first grade followup.  In designing the
survey instruments, the contractor shall take into account the comments and suggestions
made by TRP members and NCES/ED regarding the content coverage of the instruments. 
When the ECLS includes topics and/or sets of items that have been previously fielded
through other early childhood studies or when the survey is designed to build upon the data
collection efforts of other agencies, special consideration shall be given to maintaining the
comparability of items across surveys.

  Draft survey instruments (e.g., questionnaires and any forms needed to support school
record data acquisition) shall be submitted to NCES/ED for review and comment.  Following
the contractor's receipt of NCES/ED's review comments, the contractor shall revise the
instruments responding to the comments and suggestions of NCES/ED, and submit a second
set of revised instruments.  With the approval of the COTR, the contractor shall submit this
version of survey instruments to cognitive testing.

  Following the cognitive laboratory research, the contractor shall submit a third draft
of the instruments.  

  A.4  Conduct cognitive laboratory research.  It is important that instruments and items
developed for the ECLS are both valid and sensitive measures of a child's educationally
relevant experiences.  In support of these objectives, the contractor shall design and
implement research directed toward understanding the extent to which questionnaire items
are understood by respondents and how the answers they give might be interpreted.  This
research shall seek to identify any areas of ambiguity and confusion.

  The contractor shall conduct cognitive laboratory research on each data collection
instrument.  Prior to conducting any cognitive laboratory research, the contractor shall
develop and submit a memorandum to the COTR that outlines its plans for conducting such
research.  The plan shall describe the contractor's general approach to this work and identify
the specific methods and procedures that shall be used.  Justification shall be provided for the
selection of certain methods over others relative to the objectives of the research.  The
characteristics of individuals and/or groups that will be recruited for participation in this
research shall be identified and justified in light of the research objectives.  The plan shall
describe how the outcomes of the research will be analyzed and used to improve the overall
quality of the ECLS instruments.  Implementation of the plan shall be contingent upon
NCES/ED approval.

  After implementing the cognitive research lab activities, the contractor shall analyze
results and prepare and submit a memorandum that summarizes the methodology and
findings of the ECLS cognitive laboratory research.  The memorandum shall include
recommendations for modifying the instruments and procedures when indicated by the
findings.

Optional Task B.  Extend the EC LS Assessment Battery

  Extensive work has been completed to design and develop the assessment battery of
the ECLS during Phase I with the creation of frameworks for each of the domains of child
development (i.e., cognitive, socioemotional, and physical and psychomotor) and both direct
and indirect assessment instruments for kindergarten and first grade.  Each framework lays
out the key elements of the individual domains which are fundamental and predictive of
academic achievement.  The creation of each of the kindergarten and first grade assessment
instruments was guided by these assessment frameworks.  

  All kindergarten and first grade cognitive assessment battery instruments as well as
instruments for measuring the socioemotional dimension in grades k-2 were completed
during Phase I.  While cognitive assessment battery items have been developed for second
grade, instruments for group administration have yet to be developed.  An optional task
during Phase II of the ECLS includes a) completing the group administered cognitive
assessment battery instruments for grade two, and b) the extension of work on the assessment
framework, development of items and cognitive and socioemotional assessment battery
instruments for grades three to five.  If the option to complete the development of the
cognitive assessment battery instruments for the second grade and all assessment instruments
for the third through fifth grades is exercised, the contractor shall review the work completed
in Phase I and extend both the direct and indirect assessment battery instruments for the
second through fifth followups (grades two through five).  The contractor shall with
concurrence from the NCES COTR convene a panel of curriculum specialists to review and
comment on the contractor's work.  The panel members shall have expertise in the relevant
assessment domains and specific age and grade groups (i.e., ages seven to eleven and second
to fifth grade).  This group shall meet up to two times in Washington, D.C. and shall have not
more than 12 members.

  B.1  Extend the assessment battery to fifth grade.  The contractor shall review the
ECLS cognitive assessment frameworks, pool of existing items, existing kindergarten and
first grade instruments, and documentation of item statistics from the 1996-97 field test.  In
addition, the contractor shall include the extensive work completed by the National
Assessment of Educational Progress (NAEP) in developing NAEP frameworks, and relevant
published curricular standards in its review.  The contractor shall also review the ECLS
framework for measuring socioemotional development, existing direct and indirect measures
and relevant item statistics.  Based on these reviews, the contractor shall provide in a
memorandum any additions or modifications needed to create the direct and indirect
assessment instruments for grades two through five.  The contractor shall include any new
domain elements or items which were not previously included but are related to the study
issues and have been shown to be predictive of children's school performance in these later
years in the memorandum.  Furthermore, in the memorandum, the contractor shall prioritize
the key elements according to their relationships to the empirical evidence establishing the
importance of the element in each domain and the research questions to be addressed by the
ECLS. 

  B.2  Operationalize new elements in each domain.  After reviewing the documents
pertaining to the development of the ECLS assessment battery in Phase I, the contractor shall
provide in a memorandum definitions of each new element and its relationship to each of the
identified dimensions of learning (i.e., cognitive and socioemotional, and physical) for grades
two to five.  

  B.3 Extend the item pool for each element in all assessment domains.  The pool of
items for measuring each of the existing and new elements shall be of sufficient number to
reliably measure growth across the grades. The contractor shall submit a memorandum
describing the approach to be taken to expand the item pool to the COTR for approval before
undertaking any work.  With the approval of the COTR, the contractor shall build upon Phase
I work to extend the item pool from second grade to fifth grade.  Attention shall be given to
maintaining the longitudinal nature of the ECLS assessment battery in extending the item
pool.

  Draft items shall be submitted to NCES/ED for review and comment.  Following the
contractor's receipt of comments and review from NCES/ED, the contractor shall respond to
the comments and suggestions made by NCES/ED by revising the set of items.  With the
approval of the COTR, the contractor shall submit these versions of the items to curriculum
and child assessment specialists for review.  Copies of comments and suggestions from
reviewers shall be submitted to NCES/ED.  Following contractor's receipt of the COTR's
approval, the contractor shall revise the items responding to reviewer comments and
suggestions, and then submit a revised set of items.

  B.4 Adapt items for group administration.  In developing the cognitive battery
assessment instruments for group administration, the contractor shall first identify the
appropriate items that will form the set of common "anchor" or linking items between the
preceding and current grade assessment instruments (e.g., first and second grade) and the sets
of items for the second stage forms.  The first stage form shall contain items that will reliably
identify which second stage form will be administered.  It shall also be relatively easy and
fast to score.  The second stage forms shall include a sufficient number of items shared in
common with the first grade pool and items that are sufficiently distributed at both the upper
and lower levels of difficulty.  Secondly, the contractor shall identify the items suitable for
adapting to group administration.  The contractor shall submit the collection of items along
with a memorandum delineating the approach used and justification for item selection to
NCES/ED, child assessment specialists, curriculum specialists, and teachers for review and
comment.  Following receipt of comments from these groups, the contractor shall respond to
the comments and suggestions and submit a second set of revised items.  With approval of
the COTR, the contractor shall submit a draft of the first and second stage forms of the
instruments. 

  The kindergarten through second grade cognitive item pool created during Phase I
includes items from several published assessment instruments and items created for the
ECLS.  Agreements from licensors have been obtained to copy, adapt, and use specific items
in the kindergarten and first grade assessment batteries.  Permission to use any of these items
in the later grade batteries or any new items is needed before use.  The contractor shall
consult with the COTR before pursuing any permissions. 

  B.5  Conduct small-scale pilot.  The validity and sensitivity of any items and
measures of children's cognitive and socioemotional status is critical to the success of the
ECLS.  In support of these objectives, the contractor shall design and implement research
activities to ascertain the extent to which items accurately reflect the elements measured, how
the items are understood by the respondents (i.e, children and teachers) and how they might
be interpreted.  These activities shall seek to identify any areas of ambiguity and confusion. 

  Such early testing may include cognitive laboratory work similar to that described
under subtask A.4.  Prior to conducting any small-scale pilot research, the contractor shall
develop and submit a memorandum outlining its plans for conducting the research.  The plan
shall describe the contractor's general approach to this work and identify the specific methods
and procedures that shall be used.  Justification shall be provided for the selection of certain
methods over others relative to the objects of the research.  The characteristics of individuals
and/or groups that will be recruited for participation in this research activity shall be
identified and justified in according to the research objectives.  The plans shall describe how
the outcomes of the research shall be analyzed and used to improve the overall quality of the
ECLS assessment battery.  Implementation of the plan shall be contingent upon the approval
of the COTR.

  All methods and procedures advanced for collecting information directly from
children shall be appropriate for their age.  Special care shall be taken to limit any negative
impact of children's participation.

  After implementing the small-scale pilot activities, the contractor shall analyze results
and submit a memorandum that summarizes the methodology and findings of the ECLS
research activity.  The memorandum shall include recommendations for revising the
instruments and procedures when indicated by the findings.

  B.6 ECLS assessment plan.  Completion of subtasks B.1 - B.5 shall result in a set of
clear plans for and justification of the design of the second through fifth grade assessment.  A
report summarizing the contractor's plans for the design and conduct of the assessment shall
be submitted to NCES/ED.  At a minimum, the plan shall: 1) identify the key assessment
domains and the critical elements within each domain to be collected; 2) describe the
methods and procedures that will be used to conduct the assessment, identifying the sources
of data for each domain/element; 3) include copies of any questionnaires and assessment
instruments; 4) describe in detail any procedure that involves the direct assessment of young
children and steps that will be taken to minimize the impact of children's direct participation;
and 5) describe procedures for analyzing the data.

Optional Task C. Design third through sixth followup samples

  A dual-frame (i.e., list and area frames) multi-stage stratified probability
proportionate to size (PPS) design shall be used to select the base-year sample for this study.
Geographic areas (e.g., county or group of counties) will be the primary sampling units
(PSUs); school/kindergarten programs will be the secondary  sampling units (SSUs); and
children will be the ultimate sampling units (USUs). In followup surveys, children who were
selected for previous waves will be selected into the current sample with given probabilities.
For the school sample, only those schools that enroll sampled students from previous waves
are eligible for the current wave and will be selected with given probabilities.

  In most  longitudinal surveys, mobility of sample units will influence their sample
design. Student mobility is a major cost-driver in studies such as ECLS. High rates of
mobility may require more drastic subsampling of students who transfer out of their schools,
and in turn decrease sample efficiency.  Mobility is an important contributor to sample
attrition. Without sufficient efforts, resources,  and adequate procedures to track students'
movement from one school to another school, many sampled students will be lost in
followup surveys due to "unable to locate". Hence, in just a few waves, the sample can
deteriorate rather quickly.  It is desirable to have some device in the ECLS sample design that
would limit the impact of mobility to sample attrition.

  C.1  Sample size requirements.  The study will collect several different types of data.
Estimates that will be produced by this study include: proportions (e.g., percent of children
with certain characteristics) and means(e.g., mean mathematics achievement scores). The
government is interested not only in detecting changes of the estimates over time but also in
the precision of the cross-sectional estimates. For alpha equal to .05 with correlation of .60,
the study should be able to detect a 20% change in proportions and a 5% change in means for
ALL WAVES of data collection. The study also should be able to produce level estimates
for which the standard error is no bigger than 10% of a proportion and 2.5% of a mean in
ALL WAVES.

  The above requirements shall be applied not only to the entire sample but to
important subgroups. These subgroups include children who ever attended center-based early
childhood programs, children who participate in public/private kindergarten programs,
children who 

participate in full-and part-day kindergarten programs, and children whose mothers are in the
labor force and whose mothers do not participate in labor force.

  Based on information on student mobility rates and sample attrition observed from
other studies, a sample of 23,500 children attending 1,000 schools with kindergartens (200
private and 800 public schools) in 100 PSUs was determined to be adequate for meeting the
objectives of the study for ALL WAVES. However, mobility rates may change over time,
and they differ by socio-economic, geographic, and demographic characteristics such as
family income, school type, locality, race-ethnicity.  Students with certain characteristics are
more likely than their peers to attend schools enrolling only one or two students from the
previous wave. Consequently, they are more likely to be subsampled out and excluded from
the current wave. It is important to have a sufficient number of students in the followup
samples to support statistical analyses for these subgroups with the desired precision. 

  Data collected from the ECLS sample will, in addition to be used for descriptive
purposes (e.g., estimate means, proportions), also be used for analytical purposes (e.g.,
estimate relationships). Many statistical procedures such as regression analysis and
Hierarchical Linear Model (HLM) analysis have been applied to other NCES longitudinal
studies data files (e.g., NELS:88). It is important that the ECLS sample data be able to
support analyses using these procedures. Therefore, the contractor shall consider these factors
when devising subsampling plans for followup surveys.

  C.2  Mobility and tracking sample units.  Sampled kindergarten children may change
schools for many reasons including: 1) their families moving to a new location; 2) children
completing the highest grade of the school they are currently attending; 3) the school closing;
or 4) the school changing its boundaries. Such student mobility may necessitate subsampling
in future waves in order to control cost.

  In order to limit sample attrition due to "unable to locate"  and to facilitate
subsampling  in followups, it is important to have some mechanism that gathers information
on movers and locates them. The information that can be used to locate movers may be
obtained from parents or legal guardians, and/or school staff. Information that has been
collected and used successfully in locating sample units in other NCES longitudinal studies
include, but are not limited to: 1) names, addresses, telephone numbers , and relationship to
the child for parent /legal guardians , and two other people who would always know the
whereabouts of the child. 2) names and location of the schools the child expect to attend after
completing  the highest grade of the school s/he is currently attending. In addition, school 
administration records may also provide information on school to which the student has
transferred. Contractor shall maintain or enhance the procedures used in the base year and
first follow up studies.

  C.3  Oversampling  for minorities.  The study shall be able to support analyses of 
children by race-ethnicity (i.e., Hispanics, blacks, Asian and Pacific Islanders). In order to
yield sample sizes sufficient for analytic purposes at the required level of precision,  Asian
and Pacific Islanders shall be oversampled in base-year. In subsequent sampling for
followup, it is important to ensure that sample sizes meet the desired precision for the race-ethnicity subgroups.  

  C.4  Alternate sample design specifications.  Even though the government envisions a
dual-frame, multi-stage, stratified sample design for the base-year survey, and acknowledges
the strong possibility that students and the schools in which they are currently attending will
need to be subsampled for the followup surveys for ECLS,  it recognizes that other designs
may be as or more efficient. The final design shall be the one that best meets the objectives of
the study.

  C.5  Possible biases resulting from the proposed design and methods for adjusting
these biases.  Sample attrition may not be random. It is likely that children with certain
characteristics have a higher attrition rate than others. Consequently, they are less likely to be
available for subsampling in the followup surveys. This may cause bias.  Other types of
biases include nonresponse bias. The contractor shall develop methods for adjusting these
biases and include them in its sample design plan (see subtask C.6).

  C.6  Sample design plan.  The contractor shall update the sample design plan found in
the Final ECLS Study Design Report, presenting a sampling plan that meets the objectives
and the requirements of the ECLS as stated in this RPF. The updated plan shall include, but
not be limited to, specific sample sizes for each stage, the expected level of precision of
survey estimates, sample allocation, stratification, sample units sort order, sample selection
procedures, methods for adjusting biases, freshening, weighting procedures (including
weights for followup schools), procedures for variance estimation, and a schedule for
implementing the plan. All aspects of the sample design plan shall be well documented and
submitted to NCES/ED as a Sample Design Report for review and approval.  The Sample
Design Report shall be submitted to NCES/ED within 12 months of the start of this option. 

  In order to assist NCES/ED staff monitoring the implementation of the sampling plan,
the contractor shall provide NCES/ED with copies of all documents and specifications (e.g.,
computer specifications, formula for selection probability, sample size derivation and
allocation, etc.) that describes and document how the sampling plan was developed.

D.1.  Optional special studies 

  Validation Study of Teacher Reports of Classroom Characteristics.  An important
goal of the ECLS is to provide high quality and reliable data that will allow researchers to
study the relationship between the environments in which learning takes place and children's
education outcomes.  The ECLS is especially interested in the classroom environment and its
relationship to student achievement in core subject areas.  It is important that the data
collected on the structure, activities, and interactions that take place within the classroom be
reasonably sound and representative of some of the key constructs of interest to the ECLS.

  While NCES/ED believes that teachers represent a valuable source of  information
about the structure, activities, and interactions of the classroom,  it also recognizes the value
of conducting studies designed to monitor, evaluate, and improve the quality of the survey
data.  NCES/ED is especially interested in evaluating the validity of key classroom measures
of instructional practices and content coverage for important curricular domains (e.g.,
reading, mathematics, science, and social studies) found in the teacher self-administered
questionnaire.

  The contractor shall design and implement a validation study of key classroom
characteristics reported on the teacher questionnaire. The validation study of teacher reports
of classroom characteristics shall focus on relevant measures of key constructs related to
instructional practices and content coverage.  In selecting and recommending an approach,
the contractor shall consider the level of classroom intrusion/disruption and burden that the
study may entail.  The study shall consider the validity of both kindergarten and first grade
teacher reports.  It shall be conducted in the spring of the base year and first follow up
studies.

  Special Education Teacher Questionnaire.  Extensive design work and field testing
has gone into developing the survey instruments and data collection procedures for surveying
the special education teachers of core ECLS sample children with disabilities. A survey
instrument for teachers of core sample children with disabilities will be field tested during the
spring of the 1996-1997 field test year.  This instrument will be finalized based on the
outcomes of the field test, suggestions from expert reviewers, OSEP, and NCES/ED.  

  The Office of Special Education Programs (OSEP) is funding this design and
developmental work during Phase I.  It may decide to fund this activity in the full-scale
ECLS.  The tasks that the contractor shall complete if this optional study is awarded are
outlined in the following sections. 

  Reproduction of instruments.  NCES/ED will provide the contractor with the final
version of the special education teacher questionnaire at the project initiation meeting (task
1). The contractor shall reproduce or arrange for the printing of the special education teacher
questionnaire in accordance with government regulations and requirements.  A sufficient
number of copies shall be produced to meet the needs of the study (see task 9).  

  Identify special education teachers of core sample children with disabilities.  Once
children with disabilities are identified by their classroom teachers and/or by school records,
the contractor shall identify the special education teachers of these children.   Children with
multiple disabilities may receive multiple special education services from more than one
special education teacher. For example, a child with multiple disabilities may receive services
from a speech therapist and a learning disabilities teacher.  In instances where a core sample
child with disabilities receives special services from more than one provider, the contractor
shall identify up to two special education service providers to include in the study.  The rules
that the contractor will follow in identifying and selecting these teachers will be specified in a
memorandum to the NCES/ED COTR within six weeks of start date of this option.

  Administer survey instrument to special education teachers of core sample children
with disabilities, process survey data, and report on survey methodology.  The contractor
shall follow the procedures for administering the survey instruments found in the Study
Design Report, documenting any deviations or problems.  The contractor shall have
responsibility for all tasks associated with securing the cooperation of special education
teachers, administering the survey instrument, reporting on the progress of this activity,
processing the survey data, and reporting on the results of these activities (see core tasks 5, 6,
10, 11, 12, and 14).

  Verification of Head Start Program Participation.  During the 1996-97 field test,
school and parent reports of kindergartner's participation in Head Start are being evaluated. 
Schools and parents are being asked about children's participation in Head Start and to
identify the Head Start program sampled children attended the year prior to kindergarten.  All
identified programs are being contacted to verify children's participation.  The field test is
designed to provide better information about 1) the level of misreporting (especially
overreporting) of Head Start program participation, 2) the best source of information
concerning children's Head Start participation, and 3) the difficulties of verifying children's
past participation.

  Depending on the outcomes of the field test, Head Start/ACYF might decide that it is
necessary to verify children's Head Start participation in the base year ECLS and fund this
activity.  If Head Start/ACYF funds this activity, the contractor shall verifying the
participation of all children who are identified as having attended Head Start the year prior to
kindergarten.  The contractor shall use information by the ECLS parent interview and/or the
school in determining the identities of children whose participation requires verification.  The
contractor shall contact the Head Start programs these children presumably attended and
verify the child's attendance.  The contractor shall also collect a limited amount of
information concerning the program and the child's participation (e.g., dates of attendance,
length of attendance).  The contractor shall utilize the outcomes of the 1996-97 field test
when deciding on 1) the source of information to use to identify the group of possible Head
Start participants, 2) the information that should be collected from programs about children's 

Head Start participation and the program attended, and 3) the approach to contacting and
collecting the information needed to verify children's participation.

  Fall first grade and summer learning.  NCES/ED is considering adding a fall first
grade data collection to the ECLS.  If a decision is made to add a fall 1999 first grade data
collection, it will be necessary to develop a plan 1) for selecting a sample of first grade
schools and first graders, 2) developing the instruments and procedures that are needed for
this additional round of data collection, and 3) for collecting and processing these additional
data.  The contractor shall specify its plans in a memorandum to the NCES/ED COTR.  This
memorandum shall contain the contractor's  recommendation regarding what portion of the
ECLS sample (all or a subsample) should be included in this new wave of data collection,
and how the subsample (if planned) will be selected.  It shall pay particular attention to how
the contractor intends to administer the ECLS assessment battery (both direct and indirect
components), and shall identify any changes to the assessment battery or administration
procedures.  It shall also include the contractor's plan for developing and administering a 15
minute parent interview to capture information about children's summer activities and
experiences, and a 15 minute self-administered teacher questionnaire to collect information
about sampled children's fall first grade school/classroom experience.  Approval of the plan
by the NCES/ED COTR is required before the contractor shall begin to implement its
approach.  The memorandum outlining its approach shall be submitted within 60 working
days of award of this optional data collection.

  In preparing for and conducting the fall first grade data collection, the contractor shall
follow the same core tasks that cut across the fall and spring kindergarten and spring first
grade data collections.  For example, the contractor shall include fall first grade instruments
in the forms clearance package for obtaining IMT/OMB approval (task 3), select appropriate
samples of schools, students, teachers, and parents, and secure their participation in the study
(tasks 4 and 5), be able to regularly report on the status of data collection (task 6), arrange for
the printing of the teacher questionnaires, the production of the assessment battery,  and
programming of CATI/CAPI instruments (task 9), prepare a detailed schedule for completing
all data collection and successfully complete data collection according to this schedule (task
11), and have responsibility for processing all data collected (task 12). However, before
implementing the applicable tasks specified in the core design, the contractor shall first
develop the necessary parent and teacher instruments following the same set of steps outlined
in optional tasks A.3 and A.4.

  Prior to conducting the full-scale collection in the fall of 1999, the contractor shall
conduct a small-scale pilot test of the fall first grade data collection instruments and
procedures.  This test shall take place in the fall of 1998.  The pilot test shall involve
approximately 250-300 first graders in 10 to 15 schools along with their parents and teachers. 
Prior to conducting the pilot test, the contractor shall develop and submit a memorandum to
the NCES/ED COTR outlining its plan for both conducting the test and selecting schools,
children, parents, and teachers, and for analyzing the pilot test data.  A short (10-15 page)
report on the results of the pilot shall be submitted to NCES/ED within 90 working days of
the close of pilot data collection.  At least some of the child assessments must take place in
schools within the Washington, DC metropolitan area.

D. 2 Optional Supplements

  Oversample of American Indian Children.  If the option to oversample American
Indian children is exercised, the contractor shall first review Appendix C: Issues in
Oversampling American Indians and Alaskan Natives in the ECLS Study Design Report. 
The contractor shall then in a memorandum delineate to NCES/ED how it plans to implement
either one of the design options mentioned in the appendix or its own oversampling
recommendations.  This memorandum shall be submitted within six weeks of start date of
this option.  

  Upon approval of the oversampling plans by NCES/ED, the contractor shall
implement its American Indian oversampling design within the steps outlined in core tasks 4
and 5 for the selection of and the securing of cooperation from schools, students, parents, and
teachers.  Those sampled shall then be treated the same as the core sample in all other aspects
of the study (e.g., in terms of questionnaires administered, tracking procedures, data
processing, etc.)

  Oversampling of Head Start children.  A decision as to whether or not to
oversample kindergartners who attended Head Start during the year preceding kindergarten
will be made during Phase I.  If this option is exercised, the contractor shall select additional
children with Head Start experience during the base year ECLS.  Those sampled shall then be
treated the same as the core sample in all other aspects of the study. That is, all instruments
and procedures used with the members of the ECLS core sample of kindergartners shall be
used with this sample of children.  

  The contractor shall proceed to implement this optional supplement following the
steps outlined in tasks 1 through 14.  All activities and deliverables associated with tasks 1
through 14 of the core ECLS shall apply equally to this supplement.

  Oversampling populations of children with disabilities. The core ECLS sample
will include sufficient numbers of children receiving special education services to support
independent analyses of this group of children as a whole.  Also, the core ECLS sample will
include sufficient numbers of children to support independent analyses of certain high
incidence disabilities (e.g., speech impairments and learning disorders).  However, sufficient
numbers of children to support independent analyses of certain low incidence disabilities is
not expected unless these children are oversampled.  OSEP is particularly interested in four
groups children and will support the oversampling of these groups in the ECLS: 1) blind, 2)
deaf and hearing impaired, 3) mental retardation, and 4) physical and health impairments.  

  The contractor shall select samples of children to support independent analyses of
each of the four groups.  To the extent possible, those sampled shall then be treated the same
as the core sample in all other aspects of the study. That is, all instruments and procedures
used with the members of the ECLS core sample of kindergartners shall be used with this
sample of children, except where it is not feasible to do so.  Children who cannot
meaningfully participate in the core ECLS direct assessment shall participate in an
appropriate alternative assessment.  Alternative assessments which are appropriate for use
with these samples will be identified during Phase I.    

  The contractor shall proceed to implement this optional supplement following the
steps outlined in tasks 1 through 14.  All activities and deliverables associated with tasks 1
through 14 of the core ECLS shall apply equally to this supplement.

  Child Nutrition Supplement.  To implement the Child Nutrition Supplement,
funded by the U.S. Department of Agriculture's Food and Consumer Service (FCS), the
contractor shall follow the core design as outlined in task 9.  The items on how often children
engage in activities requiring physical exertion and on families' abilities to get enough to eat
have already been placed in the teacher and parent instruments.  Thus, this component of the
supplement shall be executed through the reproduction of the teacher checklist (task 9.1),
programming of the parent instrument (task 9.3), and finally the administering, processing,
and reporting of information from these two instruments as outlined in the core tasks 10, 11,
12, and 14 of the work statement.  These data are collected in the spring of the school year.

  With regards to measuring children's height and weight, the contractor shall acquire
the following materials in addition to those already specified in task 9.2:

  strain gauge scale for measuring weight
  height measurement board with a head and foot board

All data on children's height and weight shall be collected, processed, and reported in the
same manner as the measures existing in the core design (see tasks 10, 11, 12, and 14). 
Height and weight data shall be obtained in the fall and spring of kindergarten and in the
spring of first grade.
                                                    
                     IV. STUDY REQUIREMENTS 


A.  Reporting

  The contractor shall submit monthly letters of progress to the Contracts Officer for
the duration of the contract.  Letters of progress shall describe major activities by task and
accomplishments, problems and suggested solutions, significant findings and events, any
decisions which may be needed from NCES/ED, and plans for the period until the next
report.  In addition to reporting on the progress of the contract, the letter of progress shall be
used as one of the ways of keeping all parties (e.g., subcontractors, cosponsors) up to date on
project activities.  Given these dual purposes, the contractor shall submit a draft letter to the
NCES/COTR for review prior to circulating it more widely.  Progress letters shall be
submitted to the Contracts Officer on the 10th business day of each month.

The contractor shall communicate regularly (several times each week) and work closely with
the NCES/ED COTR on all aspects of the study.  To facilitate this communication, the
contractor shall set up an electronic system for transferring information via electronic mail
and microcomputer.  The contractor shall make frequent phone calls and up to four visits per
year to NCES/ED to discuss project problems and progress.  The contractor's project director
shall have a regularly scheduled weekly telephone conference with the ECLS/ED COTR to
discuss project progress, upcoming activities, and problems. 

B.  Schedule of Milestones and Deliverables

  The products that the contractor is required to prepare and submit to NCES/ED are
indicated in the descriptions of the 14 core tasks, 3 optional tasks (tasks A - C), and the
optional studies and supplements.  While there may be other possible schedules for
completing the tasks associated with the ECLS, several activities must take place at specific
times during the conduct of the project.   The final IMT/OMB clearance package for the base
year and first follow up data collections (see task 3) shall be submitted to NCES/ED at least
150 working days prior to the start of the fall 1998 data collection.  Any slippage in the
delivery of the IMT/OMB clearance package can seriously jeopardize the study.  During
Phase I it was determined when the best time was for conducting in-school child assessments
and for asking teachers to report on the skills of the children they teach.  The schedules for
the base year and first follow up data collection components described in A.5 Schedule of
Data Collection must be adhered to.  Exhibit 3 summarizes the critical project milestones and
deliverable dates.




C.  NCES/ED Review and Approval

  The contractor shall consult with the NCES/ED COTR before making any major
decision concerning a deliverable product.  Reports prepared by the contractor shall follow
guidelines referenced by the most current version of NCES/ED' Standards (1992).  It is a
requirement that schedules for deliverables found in the contractor's proposal be kept and
that NCES/ED standards be followed during all phases of project work.  

     An outline shall be submitted and approved by the NCES/ED COTR before
preparation of specific reports and plans.  Outlines are required for the following reports and
plans: 

          User's manuals (task 12)
          Methodology report (task 14)
          Sampling plan (optional task 17)

     The contractor shall expect that major reports and deliverables (e.g., user's manuals,
IMT/OMB clearance package) shall require three revisions. The contractor shall expect that
other deliverables (e.g., monthly progress letters, meeting agenda) shall require one revision.

     The contractor shall allow sufficient time for NCES/ED and OERI review.  Unless
noted elsewhere, the contractor shall allow two weeks for NCES/ED review of each
draft deliverable.  Two weeks  shall be allowed for the contractor to respond to any
requests for revisions.  All revisions requested by NCES/ED shall be incorporated
unless justified in a cover memorandum.

D.  Quality Control

     The contractor is required to develop and submit a number of reports, memorandum,
and other materials and products during the conduct of the ECLS.  The expectation is that all
such materials (even when they are labeled draft) will be of high quality and of sufficient
quality to be released as is.  Quality control of these materials is a contractor responsibility. 
All reports, memorandum, and other deliverables (including any produced by subcontractors)
submitted to NCES/ED shall be accompanied by a letter of transmittal signed by the ECLS
project director and the corporate official with corporate oversight responsibility for the
ECLS.  



                    V. PROPOSAL REQUIREMENTS

A.  Use of Subcontractors

     Proposals may include plans to subcontract parts of the work, provided that evidence
is presented that any proposed subcontractor has agreed to perform and is fully capable of
performing the assigned tasks, and that the offeror has effective control of the subcontractor's
work in the project.  Offerors are encouraged to make use of subcontractors for specialized
tasks where subcontractor expertise would strengthen the offeror's proposal.

B.  Period of Performance

     Tasks 1 through 14 are to be completed within 48 months of contract award.  At the
option of the government, optional design tasks A through C, and the optional special studies
and supplements are to be completed within the same 48 month period of performance, or 18
months from the start date of the options.   The contract will be incrementally funded.

C.  Documents Available for Inspection

     Copies of all ECLS design documents and materials referenced in this RFP may be
obtained by calling the National Data Resource Center, C/O Pinkerton Computer
Consultants, Inc. at (703) 845-3151.