|Name of Program or Resource||Type of Program or Resource/Year Initiated||Type of Evaluation Featured in Guidea||Year Evaluation Started||Evaluation Objective|
|Alabama Connecting Classrooms, Educators, & Students Statewide Distance Learning||Online courses and interactive video-conference classes for students across state/Piloted in spring 2006; statewide implementation in fall 2006||External; formative & summative; includes comparisons with traditional instructional settings||2006||Monitoring program implementation; program improvement; sharing best practices|
|Algebra I Online||Online algebra courses for students across the state/2002||External and internal; formative and summative; includes comparisons with traditional instructional settings||2003; comparative study in 2004-05||Determine if program is effective way to provide students with certified algebra teachers and to support the in-class teacher's certification efforts|
|Appleton eSchool||Online courses for students enrolled in district's high schools (some students take all courses online)/2002||Internal; formative; evaluation process based on internally developed rubric||Rubric piloted in 2006||Program improvement; sharing best practices|
|Arizona Virtual Academy||Virtual charter school for students enrolled in public schools and homeschool students (no more than 20%)/2003||Formative and summative; external and internal||2003||State monitoring; quality assurance; program improvement|
|Chicago Public Schools - Virtual High School||Online courses for students enrolled in district's high schools/2002||External; formative||2002||Assess need for mentor training and other student supports; identify ways to improve completion and pass rates|
|Digital Learning Commons||Web site with online courses and a wide array of resources for teachers and students/2003||External and internal; formative||2003||Understand usage of site; assess impact on student achievement and college readiness|
|Thinkport - Maryland Public Television with Johns Hopkins||Web site with a wide array of resources for teachers and students/2000||External and internal; formative and summative, including randomized controlled trial||2001; randomized controlled trial in 2005||Understand usage of site; assess impact of "electronic field trip" on student performance|
|Cost of Evaluation||Funding Source for Evaluation||Data Collected||Data Collection Tools||Improvements Resulting From Evaluation|
|$60,000 in 2007; $600,000 in 2008||Specific allocation in program budget (originating from state of Alabama)||Student enrollment, completion, grades; APb course pass rates; student and teacher satisfaction; description of implementation and challenges||Surveys, interviews, observations||Teacher professional development; improvements to technology and administrative operations|
|$110,000 for the most labor-intensive phase, including the comparative analysis during 2004-05||General program funds, grants from NCREL,c BellSouth Foundation, and U.S. Department of Education||Student grades and state test scores; pre- and posttests; student use and satisfaction data; focus groups; teacher characteristics and teachers' certification outcomes||Pre- and posttests developed by evaluator, surveys||Teacher professional development; increased role for in-class teachers, curriculum improvements, new technologies used; expansion to middle schools|
|No specific allocation; Approx. $15,000 to make the rubric and evaluation process available in Web format||General program funds (originating from charter grant from state of Wisconsin)||Internal descriptions and assessments of key program components (using rubric); mentor, student, and teacher satisfaction data; course completion and grades||Internally developed rubric and surveys||Mentor professional development; course content improvements; expanded interactivity in courses; improvements to program Web site and printed materials; sharing of best practices|
|No specific allocation||General program funds (originating from state of Arizona)||Student enrollment, grades, and state test scores; parent, teacher, and student satisfaction data; internal & external assessments on key program components||Electronic surveys; externally developed rubric||Wide range of operational and instructional improvements|
|Approx. $25,000||District's Office of Technology Services||Student enrollment, course completion, grades, and test scores; student use and satisfaction data; mentor assessments of needs||Surveys, interviews, focus groups||Designated class periods for online learning; more onsite mentors; training for mentors|
|Approx. $80,000 for the college-readiness study||Bill & Melinda Gates Foundation||Student transcripts; student grades and completion rates; use and satisfaction data||Surveys||Improvements to student orientation; curriculum improvements; development of school use plans to encourage best practices|
|Estimated $40,000 for the randomized controlled trial (part of a comprehensive evaluation)||Star Schools Grant||Student test scores on custom-developed content assessment; information about delivery of curriculum; use and satisfaction data||Test of content knowledge developed by evaluator, surveys, teacher implementation logs||Changes to teaching materials; changes to online content and format|
a See Glossary of Common Evaluation Terms on page 65.
b Run by the nonprofit College Board, the Advanced Placement (AP) program offers college-level course work to high school students. Many institutions of higher education offer college credits to students who take AP courses.
c North Central Regional Educational Laboratory.
Back to Evaluating Online Learning