Data & Research EVALUATION OF PROGRAMS
National Educational Technology Trends Study: Local-level Data Summary
Introduction and Data Sources and Methods



Introduction

This data summary presents data from the U.S. Department of Education's National Educational Technology Trends Study (NETTS). NETTS is a multiyear evaluation that documents the implementation of the Enhancing Education Through Technology (EETT) program from fiscal year (FY) 2002 to FY 2007. This summary briefly reviews the methods used to collect and analyze the NETTS data collected from states in the winter of 2004–05, from districts in the spring of 2005, and from teachers in fall of 2005. It also provides descriptive analyses of district and school implementation of the EETT program, focusing on issues that are central to the program: distribution of funds; EETT district investment in educational technology; teacher and student access to technology; technology-related teacher professional development; and technology integration in teaching and learning.

State subgrant and district survey data are presented by whether districts received formula or competitive subgrants in FY 2003, and teacher survey data are presented by school poverty level. Appendices A and B present the descriptive results by question from the 2005 NETTS District Survey and the 2005 NETTS Teacher Survey, respectively.1

1 Public use files of all NETTS datasets will be released at the end of the project, anticipated in the winter of 2008–09.


Data Sources and Methods

This section of the summary describes the methods that were used to examine how districts invested their EETT and other technology funds in school year (SY) 2003–04 and the ways that teachers and students in high- and low-poverty schools used technology in teaching and learning in school year 2004–05. State educational technology directors provided information about their EETT formula and competitive subgrants to districts in FY 2003. Survey data collected at the district and teacher levels aimed to compile information about EETT funds that were spent and district-provided services that were provided in schools during school year 2004–05. To account for the time it takes for federal funds to be allocated to states, awarded to districts, and distributed to schools, the surveys collected information from different fiscal and school years (Exhibit 1).

Exhibit 1. Data Sources and Analysis Methods

Evaluation Question Data Source Analysis Method
To what extent have high-poverty districts been targeted by the EETT program? Reports of state educational technology directors (FY 2003) Descriptive statistics
How do EETT districts invest their educational technology dollars? District Survey (SY 2003–04) Descriptive statistics
To what extent do K–12 teachers and students have access to hardware, software, and the Internet?
Do classrooms in high- and low-poverty schools have similar hardware, software, and Internet access?
Teacher Survey (SY 2004–05) Descriptive statistics
How much technology-related professional development do teachers receive?
Do teachers in high- and low-poverty schools participate in similar amounts of technology-related professional development?
Teacher Survey (SY 2004–05) Descriptive statistics
How often do teachers integrate technology into curriculum and instruction?
Do teachers in high- and low-poverty schools integrate similar amounts of technology into instruction?
Teacher Survey (SY 2004–05) Descriptive statistics
How do students use technology for learning in school?
Do students in high- and low-poverty schools use similar amounts of technology in school?
Teacher Survey (SY 2004–05) Descriptive statistics

Data Collection

Districts' Grant Awards

State EETT directors provided data about the numbers and sizes of formula and competitive awards that districts received. States supplied lists of the districts to which they made formula and competitive awards, either directly or through consortia, and the amounts of those awards, in FY 2003. In a small number of cases, districts provided FY 2004 award data rather than FY 2003 data.

District Survey

In spring 2005, NETTS researchers surveyed district technology coordinators about their EETT programs and the use of EETT funds for districtwide technology activities. The district survey asked technology coordinators to report on technology spending and support in school year 2003–04. The eight-part survey collected information about the EETT application process, the use of EETT partnership or consortium funds, spending on educational technology, support provided to encourage technology integration in classroom instruction, activities associated with technology-related professional development, the use of student data management systems, and estimates of districts' technology inventories. The final section of the survey gathered information on survey respondents' roles and responsibilities in the district. The district survey also asked districts to report on spending and technology support in school year 2003–04.

NETTS researchers administered the district survey to 1,039 technology coordinators selected from the 50 states, the District of Columbia, and Puerto Rico. The survey respondents represented districts that received EETT funds, districts that did not receive EETT funds, and non-districts that were lead entities for competitive EETT awards.

The sampling frames for the survey were populated by using state-provided lists for each entity and were based on data collected from the Common Core of Data (CCD), Internet searches, and phone calls. The sampling strategy considered the type of educational entity (district or non-district), poverty status, student enrollments, and location (urban or rural status). The sampling frames included the 60 largest urban districts, 12,423 other districts that had received EETT funds, and 70 non-district entities that had received EETT competitive awards. From these sampling units, 1,050 entities were stratified in proportion to EETT funding if they received EETT funds, and in proportion to enrollment if they did not. Sample sizes by strata were designed to meet prespecified precision thresholds established by the U.S. Department of Education.

To obtain an adequate survey response rate, district technology coordinators could respond to surveys online, on paper, or by phone. The response rate for the district survey was 99 percent, with 1,029 entities responding. Respondents were weighted to reflect a nationally representative sample of districts for data analysis. Descriptive results of the district survey are reported by survey question in Appendix A.

Teacher Survey

The fall 2005 NETTS teacher survey asked teachers about their use of technology in school year 2004–05. Teachers were asked to describe their access to technology and technical support, their participation in technology-related professional development, their use of technology for instruction, their students' use of technology for learning, and supports for and barriers to technology use in their schools.

The teacher sample was created by drawing a probability sample of 975 schools from respondents to the district survey, stratified by school type (elementary or secondary), and poverty level (high or low).2 Schools were randomly sampled in proportion to the number of teachers and in inverse proportion to district size to produce a sample of schools whose selection probabilities were roughly independent of the size of their district's enrollment. From these schools, 233 high-poverty schools were randomly sampled for more intensive study.

NETTS researchers obtained teacher rosters for the 975 selected schools. Teachers who did not teach at the same school in school years 2004–05 and 2005–06 and who did not teach in a core subject area were excluded from the sample. Targets of four teachers from each of the schools in the original probability sample (742 schools) and of 25 teachers from each of the high-poverty schools (233 schools) were randomly selected for the teacher sample. The final teacher sample consisted of 6,017 teachers.

NETTS researchers administered the NETTS teacher survey in fall 2005. Teachers could complete their surveys online or on paper. Researchers collected completed surveys from 4,935 teachers for an overall response rate of 82 percent. In analyzing the data, survey respondents were weighted to reflect a nationally representative sample of teachers. Descriptive results by survey question are reported in Appendix B.3

Data Analyses

Targeting of EETT Grants

States' reports of their EETT grants to districts were examined for districts that received only formula funds and for districts that received competitive funds, either alone or in combination with formula funds. Grant amounts were summarized for these two types of districts and for districts with different percentages and numbers of students in low-income families.

Technology Integration in Districts That Received EETT Subgrants

Survey responses from the subsample of teachers in districts receiving EETT subgrants in school year 2003–04 were used to describe the extent to which teachers in these districts used technology at least weekly in instruction, their students used technology in learning at least weekly, and their students used technology for the critical-thinking or decision-making skills related to technology literacy at least once a week.

Teacher and Student Technology Use

Teachers' survey responses from the full nationally representative sample were used to describe the extent to which teachers used technology once a week or more in instruction, students used technology in learning at least once a week, and students used technology for the critical-thinking or decision-making skills related to technology literacy at least weekly (Exhibit 2).

Exhibit 2. Measures of Teacher and Student Technology Use

Teachers' Use of Technology in Instruction Students' Use of Technology in Learning Students' Use of Technology for the Critical Thinking and Decision-Making Skills Related to Technology Literacy

Teachers used technology at least once a week to:

  • Develop curricula or assignments in reading, math or other subjects
  • Present reading, math, or other subject concepts to students
  • Create tests or quizzes
  • Test students
  • Collaborate with experts or teachers in other locations
  • Adapt instructional activities to students' individual needs
  • Do research and lesson planning using the Internet

Students used technology at least once a week to:

  • Practice or review reading, math, or other subject areas
  • Extend learning in reading, math, or other subject areas with enrichment activities
  • Produce media, Web, or presentation products
  • Conduct online research
  • Take tests or quizzes using a computer
  • Prepare for standardized tests

Students used technology at least once a week to:

  • Communicate electronically about academic content with experts, peers, or others
  • Solve real-world problems
  • Visually represent or investigate concepts
  • Take tests or quizzes using a computer
  • Create products that had real-world audiences
  • Work cooperatively or collaboratively with other students
  • Work with content in multiple disciplines
  • Use inquiry-based strategies
  • Use authentic tools

Technology Access and Support

Technology access and support variables describe the extent to which teachers and students had access to technology resources (Exhibit 3). These variables described teachers' access to technology, students' access to technology in the classroom, students' access to technology in labs or media centers in the school, and teachers' completion of technology-related professional development.

Exhibit 3. Measures of Technology Access and Support in Schools

Teachers' Access to Technology

These technologies were available at school for teacher use:

  • Desktop or laptop computer
  • PDAs or content-specific technologies
  • High-speed Internet access
  • Printers
  • Device to project computer screen for whole-class viewing
  • Presentation software
  • Software for instruction in reading, math, or other subjects
  • E-mail software
Students' Access to Technology in the Classroom

Students had classroom access to:

  • One computer per student
  • One computer for every two students
  • One computer for every three students
  • One computer for every four students
  • PDAs or content-specific technologies
  • High-speed Internet access
  • Printers
  • Presentation software
  • Software for instruction in reading, math, or other subjects
Students' Access to Technology in Computer Labs or Media Centers

Students had lab or media center access to:

  • One computer per student
  • One computer for every two students
  • One computer for every three students
  • One computer for every four students
  • PDAs or content-specific technologies
  • High-speed Internet access
  • Printers
  • Presentation software
  • Software for instruction in reading, math, or other subjects
Teachers' Completion of Technology-Related Professional Development

Teachers completed professional development that addressed:

  • Use of technology for new methods of teaching
  • Use of technology to enhance student learning in reading, math, or other subjects
  • Improving students' technology literacy
  • Use of technology to design or administer student assessments
  • Use of technology for curriculum development and lesson planning
  • Use of technology to meet the needs of students with disabilities or limited English proficiency
  • Use of software to tailor tasks to individual student ability
  • Developing proficiency with online teaching
  • Effective or ethical use of the Internet

Conceptual Framework

District data on EETT grant amounts were organized to compare districts that reported receiving only formula grants with districts that reported receiving competitive grants, usually in combination with formula monies. District funding was also analyzed by examining median district awards that were above and below the state medians of numbers and percentages of poor students. Cross-tabulations of district funding data by type of award (only formula, and combined formula and competitive) and by student poverty (above or below the state medians of numbers or percentages of poor students) were produced to examine relationships between EETT funding levels and district need.

Data for the nationally representative samples of districts and teachers described teacher and student technology use and technology access and support. Individual item-level data detailed above were used to describe the spending patterns of districts that received only formula grants and districts that received competitive grants only or in combination with formula grants.4 Two-tailed t-tests were used to examine the statistical significance of differences between these two types of districts. These descriptive data and analyses are shown in Exhibits 4 through 7.

To address questions about the differences across poverty levels in technology access and use, individual item data from the teacher survey responses were examined for teachers at high-, middle-, and low-poverty schools. High-poverty schools were in the top poverty quartile based on free and reduced-price lunch program rates for each of the three levels of schooling (elementary, middle, and high). Middle-poverty schools were those in the second-highest poverty quartile for their level of schooling, and low-poverty schools were those whose poverty rates were in the bottom 50 percent (i.e., the two least poor quartiles) for their schooling level. Two-tailed t-tests were calculated to determine whether the differences by poverty level in technology access, support, and use were statistically significant. Descriptive statistics for the survey items describing activities involving teacher and student technology integration at least once a week were calculated for the subsample of teachers in districts that received EETT funds (Exhibits 8 through 10). Descriptive data for survey items that describe weekly technology integration activities and technology access and support are shown in Exhibits 11 through 19.


2 For elementary schools, the poverty threshold, measured in terms of the percentage of students who were eligible for free and reduced-price lunches (FRPL), was 29.7 percent. For middle schools and high schools, the poverty thresholds were 24.3 percent and 15.9 percent, respectively.

3 The teacher data presented in the exhibits appear lower than the results shown in Appendix B. The analysis required imputation of zeroes for some missing values to allow comparisons between the same respondent pool for the survey items used to investigate technology access, technology integration, and technology-related professional development. As a result, the exhibits reflect slightly conservative estimates of teacher access to and use of technology in their schools.

4 Districts that received only competitive awards were grouped with districts that received both formula and competitive awards because the “competitive only” group size was small (67 districts).


 
Print this page Printable view Send this page Share this page
Last Modified: 07/09/2008