At the Department of Education, we are working to make sure American taxpayers receive better value for their money by improving federal program performance and increasing efficiency, while reducing waste. These goals go hand-in-hand with promoting increased program flexibility and innovation. To measure performance in the Graduate Assistance in Areas of National Need (GAANN) program, which provides fellowships to graduate students of superior ability who demonstrate financial necessity and plan to pursue the highest degree available in designated areas of national need, the Department has established both results (impact) and efficiency measures. It is the efficiency measure that is discussed in detail here.
What is the efficiency measure for the GAANN program?
The efficiency measure is the federal cost per successful outcome. By successful outcome, we mean students who advance to candidacy, as well as students who achieve a doctorate. To be considered qualified for preparing and defending a doctoral thesis, candidates are required to pass a preliminary examination, thus "advancing to candidacy." Because the three- to four-year grant award period is shorter than seven years*, which is the average time required for students to achieve a doctorate in the areas of national need, this definition is a more realistic assessment of fellows' progress.
The cost used in the efficiency measure is the amount of federal funds expended for a given cohort of grants. This is the total of program funds obligated to all grantees (institutional program or academic unit) divided by the total number of fellows who advanced to candidacy or achieved a doctorate during the grant period. In addition to this overall program level efficiency measure calculation, the Department calculates efficiency measures at the individual grantee level.
How are the grantee-level efficiency measures calculated?
The approach is the same as for the program overall. The data that grantees provide in their final performance report are combined with Department program records to calculate the resulting efficiency measures. The percentage of fellows who advance to candidacy or achieve a doctorate is reported by each grantee. The financial data are based on the Department's records.
How are the grantee-level efficiency measures analyzed?
The efficiency measure values are combined with some of the data that grantee institutions submit to the Department's National Center for Education Statistics on the Integrated Postsecondary Education Data System (IPEDS). These results are presented in the Grantee Efficiency Analysis [ MS Excel (46K) | PDF (25K)], with grantees listed alphabetically by the institution's name. A detailed explanation for each column in the analysis report can be obtained in the Glossary. Note: if none of the fellows at a grant had achieved a doctorate or advanced to candidacy by the time the final report was filed, the success rate is 0 percent, and the cost per successful outcome cannot be calculated.
How can the efficiency measure analysis be used to make improvements?
The Summary Efficiency Analysis [ MS Excel (33K) | PDF (16K)] groups grantees in ways that facilitate comparison and identifies potential areas for further analysis that could result in program improvements. A detailed explanation of each title on the summary spreadsheet is in the Glossary. In reviewing these data, it is important to note that lower costs per successful outcome cannot be considered in isolation. Rather, a variety of factors must be considered. For example, efficiency should be considered in terms of overall success rate, and comparisons between institutions should take into account differences in the way the program is administered. Differences between grantees are related to many factors:
- Graduate tuition and fees, which range from $1,300 to $30,000/year;**
- Student living expenses, which range from $10,000 to $21,000/year depending on the school location;***
- Individual fellows' financial need and overall financial aid package, which affects the amount of the fellowship and may include student loans;
- Institutional matching contributions, which range between 23 percent and 100 percent; although schools are required to provide at least 25 percent of the federal award;
- The point in their graduate school tenure that fellows receive GAANN fellowships, which may be at anytime from the first to final year;
- The length of time for which fellows receive the fellowship, which may be as little as one semester or as many as four years; and
- Grant extensions, which extend the three-year grant into a fourth or fifth year giving fellows more time to progress in a doctoral program and likely resulting in a higher success rate.
Keeping this in mind, grantees are encouraged to compare their performance results to those of other grantees with similar characteristics.
Generally, we note that the data for new grants awarded in 2000 and 2001 and closed by December 2004 appear to show a relationship between the successful outcome rate for the grantee, the average amount of the federal fellowship, and the average amount of matching funds per fellow. Grantees that provide larger federal fellowship amounts have a higher success rate, on average, than grantees that provide smaller fellowships. In addition, grantees that supplement the largest average federal fellowship amounts with the highest average institutional match amount per fellow have the highest success rates, on average. See charts [ MS Word (4.6MB) | PDF (39K)].
If you have questions or comments about the data and analyses presented here, please contact firstname.lastname@example.org.
*Hoffer, T.B., S. Sederstrom, L. Selfa, V. Welch, M. Hess, S. Brown, S. Reyes, K. Webber, and I. Guzman-Barron. 2003. Doctorate Recipients from United States Universities: Summary Report 2002. Chicago: National Opinion Research Center. This report gives the results of data collected by the Survey of Earned Doctorates, conducted for six federal agencies, National Science Foundation, National Institutes of Health, U.S. Department of Education, National Endowment for the Humanities, U.S. Department of Agriculture and NASA by National Opinion Research Center.
**(IPEDS average of 2000-04 in-state average full-time graduate tuition and required full-time graduate fees (IC2000_AY, IC2001_AY, IC2002_AY, IC2003_AY, IC2004_AY).***www.collegeboard.com/highered/res/leb/leb.html accessed 6/19/2006.