U.S. Department of Education: Promoting Educational Excellence for all Americans

A r c h i v e d  I n f o r m a t i o n

ESRA: Research, Development and Dissemination - FY 2006


CFDA Number: 84.305 - Education Research

Program Goal: Transform education into an evidence-based field.

Objective 1 of 2: Raise the quality of research funded or conducted by the Department.
Indicator 1.1 of 2: The percentage of new research proposals funded by Institute of Education Sciences that receive an average score of excellent or higher from an independent review panel of qualified scientists.
 
Measure 1.1.1 of 1: The percentage of new research proposals funded by the Department's National Center for Education Research that receive an average score of excellent or higher from an independent review panel of qualified scientists.
Year Actual Performance Performance Targets
2003 88  
2004 97  
2005 100 100
2006   100

Source: Expert panel review

Frequency: Annually.

Next Data Available: September 2006

Limitations: Evaluations are only as good as the qualifications of the peer review panel. Inclusion of senior scientists who are leading researchers in their fields ensures the quality of the data.
 
Explanation: The measure is calculated as the average panel review score for newly funded IES research proposals.
 
Indicator 1.2 of 2: Of new research and evaluation projects funded by the IES that address causal questions, the percentage of projects that employ randomized experimental designs.
 
Measure 1.2.1 of 1: Of new research and evaluation projects funded by the Department's National Center of Education Research that address causal questions, the percentage of projects that employ randomized experimental designs.
Year Actual Performance Performance Targets
2001
32
32
2002
100
75
2003
97
75
2004
90
75
2005
 
75
2006
 
75

Source: IES researchers evaluate all research and evaluation proposals newly funded by IES.

Frequency: Annually.

Next Data Available: March 2006
Having qualified researchers conduct the reviews, as well as having two IES researchers independently evaluate a subset of proposals (with minimum inter-rater agreement of 90 percent), minimizes threats to the validity and reliability of data.

Limitations: Evaluations are only as good as the qualifications of the proposal reviewers.
 
Explanation: The 75 percent target for 2002-2006 recognizes that some high-quality research addressing causal questions will not be able to employ randomized experimental designs. Presence of a causal question is defined as instances in which the investigation is designed to examine the effects of one variable on a second variable. A causal relation might be expressed as one variable influencing, affecting, or changing another variable. A randomized experimental design is defined as instances in which there are (a) an experimental (treatment) group and one or more comparison groups, and (b) random assignment of participants to treatment and comparison groups, or random assignment of groups (e.g., classrooms or schools) to treatment and comparison conditions. If a proposal includes a design in which two or more groups of participants are compared, but the PI does not explicitly indicate that random assignment procedures will be used, the proposal is recorded as not using a randomized experimental design.
 
Objective 2 of 2: Increase the relevance of our research in order to meet the needs of our customers.
Indicator 2.1 of 3: The percentage of new research projects funded by IES that are deemed to be of high relevance to education practice as determined by an independent review panel of qualified practitioners.
 
Measure 2.1.1 of 1: The percentage of new research projects funded by the Department's National Center of Education Research that are deemed to be of high relevance as determined by an independent review panel of qualified practitioners.
Year Actual Performance Performance Targets
2001
21
 
2002
25
25
2003
60
37
2004
50
50
2005
 
65
2006
 
75

Source: External panel of qualified practitioners

Frequency: Annually.

Next Data Available: March 2006

Limitations: Evaluations are only as good as the qualifications of the external review panel. Inclusion of experienced practitioners and administrators in education and special education assures the quality of the data.
 
Explanation: The target of 75 percent for 2006 recognizes that some important research may not seem immediately relevant but will make important contributions over the long term.
 
Indicator 2.2 of 3: The number of annual hits on the What Works Clearinghouse Web site.
 
Measure 2.2.1 of 1: The number of annual hits on the What Works Clearinghouse Web site.
Year Actual Performance Performance Targets
2003
1,522,922
1,000,000
2004
4,249,668
2,000,000
2005
4,734,767
4,500,000
2006
 
5,000,000
2007
 
5,500,000

Source: What Works Clearinghouse.

Frequency: Annually.

Next Data Available: March 2006
A Web based program automatically counts the hits on this Web site.
 
 
Indicator 2.3 of 3: The percentage of WWC Web site users surveyed randomly who responded to the statement, ''Evidence provided on the WWC Web site is useful in making decisions about education programs and practices'' by checking ''agree'' or ''strongly agree.''
 
Measure 2.3.1 of 1: The percentage of WWC Web site users surveyed randomly who responded to the statement, ''Evidence provided on the WWC Web site is useful in making decisions about education programs and practices'' by checking ''agree'' or ''strongly agree.''
Year Actual Performance Performance Targets
2005
67
30
2006
 
31

Source: What Works Clearinghouse (WWC) Web site survey.

Frequency: Annually.

Next Data Available: March 2006
 
 


Return to table of contents

Last Modified: 12/28/2005