- Table 1 - McNair Grantee-level Cumulative Graduate School Enrollment Rate: 2004-05 and 2005-06; Cohort year 2003-04 (All grantees)
- Table 2 - McNair Grantee-level Graduate School Enrollment Rate: 2005-06; Cohort year 2004-05 (All grantees)
The 2005-06 performance measures for the Ronald E. McNair Postbaccalaureate Achievement (McNair) program are measurable educational outcomes for the projects funded by the program. The following provides an introduction, description of the methodology and terms used to calculate and analyze the one-year and two-year graduate school enrollment rates of McNair projects, and a summary of the preliminary findings. The tables provide the actual data and results of the analyses for each grantee and for the program. The analyses are based on the data provided by the grantees in the annual performance reports for 2003-04, 2004-05 and 2005-06 and are not the result of a rigorous, independent evaluation of the McNair program. Data, therefore, should be interpreted with caution given various reporting issues as described in the section below on data constraints.
WHY IS THE U.S. DEPARTMENT OF EDUCATION CALCULATING GRANTEE-LEVEL PERFORMANCE MEASURES?
The Department is committed to continually improving its management of programs and improving the educational outcomes of students. Improvements are guided by monitoring and assessing performance, improving the data used for these assessments, collaborating with stakeholders, implementing recommendations, and re-assessing performance. Providing data to the public is a key element in promoting improvement and collaborating with stakeholders.
WHAT IS THE PERFORMANCE MEASURE FOR TRIO MCNAIR PROJECTS?
Since the TRIO McNair program prepares upper-class undergraduate students for doctoral studies, the primary performance measure is the graduate school enrollment rate of program participants. Our goal is to show, for each McNair project, the number of McNair participants receiving a bachelor's degree (BA) who subsequently enroll in graduate school anytime during the three academic years following attainment of their BA. Although this methodology will not capture all program successes, research data indicate that a substantial number of individuals who pursue graduate degrees begin their graduate programs within three years of receiving their BA1. Thus, the three-year timeframe is a reasonable measurement for comparing program outcomes among projects.
To report performance information on all currently funded McNair grantees, this analysis reflects the current four-year grant cycle that began in 2003-04. For this reason, we are able to show two cohorts of McNair participants, based on BA attainment. These cohorts are:
- McNair participants who received their BA in 2003-04 (Table 1); and
- McNair participants who received their BA in 2004-05 (Table 2).
Table 1 shows both the one-year and the two-year graduate school enrollment rates for those McNair participants who received their BA in 2003-04 and enrolled in graduate school in 2004-05 (one-year rate) and/or 2005-06 (two-year rate). The data are not yet available to calculate the three-year rate. The overall two-year graduate school enrollment rate for those McNair participants who received their BA in 2003-04 is 65.7 percent.
Table 2 shows the one-year graduate school enrollment rate for McNair participants who received their BA in 2004-05 and enrolled in graduate school in 2005-06. At this time we can only display the one year rate for these McNair participants, as the data are not yet available to calculate the two and three-year rates.
The data, in Tables 1 and 2, are organized alphabetically by grantee name and show all McNair grantees funded in 2003-04, 2004-05 and 2005-06 (179 grantees). Please note the last column in Table 1, "Newly Funded Grantee in 2003-04**," identifies grantees that were not previously funded. For most of these grantees, we could not calculate either a one- or two-year graduate school enrollment rate because these grantees had no 2003-04 BA recipients in the first year of the project. This is not an unusual situation for new grantees, as many McNair grantees begin working with undergraduate students when they are sophomores and juniors, and do not serve seniors during the first year of the grant.
HOW DID THE U.S. DEPARTMENT OF EDUCATION CALCULATE THE GRADUATE SCHOOL ENROLLMENT RATES?
The data sources used for calculating the graduate school enrollment rates are the 2003-04, 2004-05, and 2005-06 McNair Annual Performance Reports (APRs) submitted by grantees.
- For the one-year rate of the 2003-04 cohort, we divide the number of students enrolling in graduate school in 2004-05 by the number of students receiving a BA in 2003-04 and multiply by 100.
- For the one-year rate of the 2004-05 cohort, we divide the number of students enrolling in graduate school in 2005-06 by the number of students receiving a BA in 2004-05 and multiply by 100.
- For the two-year cumulative rate of the 2003-04 cohort, we divide the number of students enrolling in graduate school in 2004-05 and 2005-06 (i.e., within two years after receiving a BA) by the number of students receiving a BA in 2003-04 and multiply by 100. Please note, this is a cumulative rate that includes in the numerator students who enrolled in graduate school one and two years after receiving a BA (i.e., enrolled in graduate school in 2004-05 and/or 2005-06).
To calculate the numbers of students in each cohort and numbers of BA recipients for use in the rate formulas above, the following APR fields are used:
- For the number of students receiving BAs in 2003-04 and 2004-05 (i.e., cohorts), data are derived from several fields on the APR. The first criterion in determining the cohort of BA recipients is to determine if a student earned a BA degree prior to the cohort year. If a student earned a BA degree prior to the cohort year, the student is not included in the cohort. The second criterion is to examine the responses to fields #21 and #22. If field #21 (highest degree earned) equals option 1, BA, and field #22 (date of highest degree earned) is between September 1, 2003, and August 31, 2004, for the 2003-04 cohort and September 1, 2004, and August 31, 2005, for the 2004-05 cohort, then the student is included in the cohort of BA recipients. If field #22 (date of highest degree earned) is out-of-range, missing, or invalid, then the responses to field #18 (College grade level at the end of the spring/summer term), field #17 (college grade level at entry into the project), and field #19 (enrollment status for academic year being reported) are examined to determine whether or not a student should be included in the cohort.
- For the 2003-04 cohort, the number of students enrolling in graduate school in 2004-05 (one-year rate) and/or 2005-06 (two-year rate) is captured from the data reported in field #18 (college grade level at the end of the 2004-05 and 2005-06 academic years), options 7, 8, 9, 10, and 11 (graduate/professional program).
- For the 2004-05 cohort, the number of students enrolling in graduate school in 2005-06 (one-year rate) is captured from the data reported in field #18 (college grade level at the end of the 2005-06 academic year), options 7, 8, 9, 10, and 11 (graduate/professional program).
WHAT WERE SOME OF THE DATA CONSTRAINTS IN CALCULATING THE GRADUATE SCHOOL ENROLLMENT RATES?
An examination of the data indicated various reporting issues such as some grantees not updating their student records, inconsistent responses among data fields, and changes to some data fields in the 2003-04 APR from previous APRs. Some grantees may not have data on the graduate school enrollment status of all prior participants at the time the APR is submitted. The extent to which these factors impacted the number of students in each cohort and the number enrolling in graduate school is not known. Also, a grantee may not have data on the graduate school enrollment status of all prior participants at the time the annual performance report is submitted which likely results in some under-reporting of the project's successes.
WHAT ARE SOME OF THE PRELIMINARY FINDINGS?
The data show that graduate school enrollment rates vary significantly between projects. One reason for this variance is the number of BA recipients reported for each institution. The average cohort has only 12 students, so each student accounts for a significant change in graduate school enrollment rates. For example, in Table 1 one grantee has a 100 percent graduate school enrollment rate because only one student received a BA in 2003-04 and this same student enrolled in graduate school in 2004-05. The grantee two-year rate remained the same because the rate is a cumulative number and the student had already enrolled in the first year. This contrasts with another grantee that has a 67 percent graduate school enrollment rate based on 22 students enrolling in graduate school out of 33 students who received a BA in 2003-04.
1 Nevill, S.C., and Chen, X. (2007). The Path Through Graduate School: A Longitudinal Examination 10 Years After Bachelor's Degree (NCES 2007-162). U.S. Department of Education. Washington, DC: National Center for Education Statistics, p. 18.