U.S. Department of Education: Promoting Educational Excellence for all Americans
Name of Program or Resource Type of Program or Resource/Year Initiated Type of Evaluation Featured in Guidea Year Evaluation Started Evaluation Objective
Alabama Connecting Classrooms, Educators, & Students Statewide Distance Learning Online courses and interactive video-conference classes for students across state/Piloted in spring 2006; statewide implementation in fall 2006 External; formative & summative; includes comparisons with traditional instructional settings 2006 Monitoring program implementation; program improvement; sharing best practices
Algebra I Online Online algebra courses for students across the state/2002 External and internal; formative and summative; includes comparisons with traditional instructional settings 2003; comparative study in 2004-05 Determine if program is effective way to provide students with certified algebra teachers and to support the in-class teacher's certification efforts
Appleton eSchool Online courses for students enrolled in district's high schools (some students take all courses online)/2002 Internal; formative; evaluation process based on internally developed rubric Rubric piloted in 2006 Program improvement; sharing best practices
Arizona Virtual Academy Virtual charter school for students enrolled in public schools and homeschool students (no more than 20%)/2003 Formative and summative; external and internal 2003 State monitoring; quality assurance; program improvement
Chicago Public Schools - Virtual High School Online courses for students enrolled in district's high schools/2002 External; formative 2002 Assess need for mentor training and other student supports; identify ways to improve completion and pass rates
Digital Learning Commons Web site with online courses and a wide array of resources for teachers and students/2003 External and internal; formative 2003 Understand usage of site; assess impact on student achievement and college readiness
Thinkport - Maryland Public Television with Johns Hopkins Web site with a wide array of resources for teachers and students/2000 External and internal; formative and summative, including randomized controlled trial 2001; randomized controlled trial in 2005 Understand usage of site; assess impact of "electronic field trip" on student performance

Cost of Evaluation Funding Source for Evaluation Data Collected Data Collection Tools Improvements Resulting From Evaluation
$60,000 in 2007; $600,000 in 2008 Specific allocation in program budget (originating from state of Alabama) Student enrollment, completion, grades; APb course pass rates; student and teacher satisfaction; description of implementation and challenges Surveys, interviews, observations Teacher professional development; improvements to technology and administrative operations
$110,000 for the most labor-intensive phase, including the comparative analysis during 2004-05 General program funds, grants from NCREL,c BellSouth Foundation, and U.S. Department of Education Student grades and state test scores; pre- and posttests; student use and satisfaction data; focus groups; teacher characteristics and teachers' certification outcomes Pre- and posttests developed by evaluator, surveys Teacher professional development; increased role for in-class teachers, curriculum improvements, new technologies used; expansion to middle schools
No specific allocation; Approx. $15,000 to make the rubric and evaluation process available in Web format General program funds (originating from charter grant from state of Wisconsin) Internal descriptions and assessments of key program components (using rubric); mentor, student, and teacher satisfaction data; course completion and grades Internally developed rubric and surveys Mentor professional development; course content improvements; expanded interactivity in courses; improvements to program Web site and printed materials; sharing of best practices
No specific allocation General program funds (originating from state of Arizona) Student enrollment, grades, and state test scores; parent, teacher, and student satisfaction data; internal & external assessments on key program components Electronic surveys; externally developed rubric Wide range of operational and instructional improvements
Approx. $25,000 District's Office of Technology Services Student enrollment, course completion, grades, and test scores; student use and satisfaction data; mentor assessments of needs Surveys, interviews, focus groups Designated class periods for online learning; more onsite mentors; training for mentors
Approx. $80,000 for the college-readiness study Bill & Melinda Gates Foundation Student transcripts; student grades and completion rates; use and satisfaction data Surveys Improvements to student orientation; curriculum improvements; development of school use plans to encourage best practices
Estimated $40,000 for the randomized controlled trial (part of a comprehensive evaluation) Star Schools Grant Student test scores on custom-developed content assessment; information about delivery of curriculum; use and satisfaction data Test of content knowledge developed by evaluator, surveys, teacher implementation logs Changes to teaching materials; changes to online content and format

a See Glossary of Common Evaluation Terms on page 65.
b Run by the nonprofit College Board, the Advanced Placement (AP) program offers college-level course work to high school students. Many institutions of higher education offer college credits to students who take AP courses.
c North Central Regional Educational Laboratory.

Back to Evaluating Online Learning

Last Modified: 07/31/2008