Answers in the Tool Box: Academic Intensity, Attendance Patterns, and Bachelor's Degree Attainment June 1999
Answers in the Tool Box is a study about what contributes most to long-term
bachelor's degree completion of students who attend 4-year colleges (even if they also attend other types of institutions).
Degree completion is the true bottom line for college administrators, state legislators, parents, and most importantly, studentsnot retention to the second year, not persistence without a degree, but completion.
This study tells a story built from the high school and college transcript records, test scores, and surveys of a national cohort from the time they were in the 10th grade in 1980 until
roughly age 30 in 1993. The story gives them 11 years to enter higher education, attend a 4-year college, and complete a bachelor's degree. In these respectsbased in transcripts and using a long-term bachelor's degree attainment markerthis story is, surprisingly, new.
This study was motivated by four developments in higher education during the 1990s:
- The growing public use of institutional graduation rates as a measure of accountability, and the tendency in public policy and opinion to blame colleges for students'
failure to complete degrees and/or for failure to complete degrees in a timely manner.
- An ever expanding proportion of high school graduating classes entering postsecondary education, and new federal policies encouraging even more students to enter or return to higher education. Our system is being challenged simply to maintain, let alone improve, college graduation rates.
- The increasing tendency, overlooked in both policy and research, for students to
attend two, three, or more colleges (sometimes in alternating patterns, sometimes
simultaneously) in the course of their undergraduate careers.
- The rising heat of disputes involving admissions formulas at selective colleges where affirmative action policies have been challenged. These disputes, carried into the media and hence dominating public understanding, involve two indicators of precollege attainmentgrades/class rank versus test scoreswithout any reference to high school curriculum and its role in the degree completion rates of the mass of minority students.
The story of what contributes most to bachelor's degree attainment works toward six ordinary
least squares regression equations that progressively add blocks of key variables following the
progress of students from high school into higher education and through the first true year of
attendance. The penultimate model (the fifth in the series) accounts for about 43 percent of the
variance in bachelor's degree completion. The sixth equation simply indicates that one
hits a plateau of explanation at this point. For a story-line such as this, 43 percent is a very
high number. A five-step logistic regression then provides both a dramatic underscoring of
the principal findings and some enlightening variations.
There are 11 variables in the penultimate linear regression model. The two most important
variables, accounting for the bulk of the model's explanatory power are:
- "Academic Resources," a composite measure of the academic content and performance the student brings forward from secondary school into higher education. This measure is dominated by the intensity and quality of secondary school curriculum [Part I and Appendix C].
- Continuous enrollment once a true start has been made in higher education.
In the logistic version of the penultimate model, the same 11 variables (out of 24) are
statistically significant, but those displaying the strongest relationships to degree completion
(the highest "odds ratios") are all post-matriculation phenomena: continuous enrollment,
community college to 4-year college transfer, and the trend in one's college grades.
Among the 11 variables, the following are not usually found in similar analyses:
- Proportion of undergraduate grades indicating courses the student dropped,
withdrew, left incomplete, or repeated.
- A final undergraduate grade point average that is higher than that of the first
"true" year of attendance.
- Parenthood prior to age 22.
- Whether the student attended more than one institution and did not return to the
first institution of attendance, a situation that includes, but transcends, the classical
community college to 4-year college transfer pattern.
The only demographic variable that remains in the equation at its penultimate iteration is
socioeconomic status, and by the time students have passed through their first year of college,
SES provides but a very modest contribution to eventual degree completion. No matter now
many times (and in different formulations) we try to introduce race as a variable, it does not
meet the most generous of threshold criteria for statistical significance.
Conclusions That Follow from These Findings
- When nearly 60 percent of undergraduates attend more than one institution and 40 percent of this group do not complete degrees, institutional graduation rates are not very meaningful. It is not wise to blame a college with superficially low graduation rates for the behavior of students who swirl through the system.
- Analysis of institutional effects on degree completion is compromised when students attend two or more institutions. One wastes precious research time trying to figure out which type of experience in institution X had an impact if the student also attends institutions Y and Z. There are some exceptions to these principles, e.g. when the second institution involves a study abroad semester.
- When the academic intensity and quality of one's high school curriculum is such a dominant determinant of degree completion, and both test scores and (especially) high school grade point average or class rank are so much weaker contributors to attainment, college admissions formulas that emphasize test scores and (especially) high school grade point average or class rank are likely to result in lower degree completion rates.
- The type and amount of remediation matters in relation to degree completion. Increasingly, state and local policy seeks to constrictif not eliminatethe amount of remedial work that takes place in 4-year colleges. But there is a class of students whose deficiencies in preparation are minor and can be remediated quickly without excessive damage to degree completion rates.
What We Learned: Variables to Discard
Examples of stock building-block variables that are discarded because of weak architecture:
- Highest level of parents' education. As reported by students, these data are uneven and unreliable. In the most recent of the national longitudinal studies, the highest degree of agreement between students and parents on this score was 72 percent in the case of fathers with "some college." One out of six students would not even venture a guess as to their parents' education.
- "Persistence" defined in temporal terms, e.g. from the 1st to 2nd year of college. Transcripts reveal an enormous range in the quality of arrival at the putative 2nd year: some 30 percent of those who were "retained" or "persisted" arrived with either less than 20 credits or 3 or more remedial courses.
- "Academic track" (sometimes called "college preparatory") in secondary school curriculum, whether reported by students or by schools. When the transcripts for a third of the students on the "academic track" show 8 or fewer Carnegie units in core academic subjects, it is obvious that the transcriptsnot the labelmust be the source of judgment.
- "Part-time" enrollment in postsecondary education. Students change status from term to term. Part-year enrollment may be more important than light credit loads. Most importantly, students change status within a given term, by dropping, withdrawing from, or leaving incomplete large portions of their credit loads. The "DWI Index" (ratio of drops/withdrawals/incompletes to total courses attempted) derived from transcript records is far more important than what the student says in an interview about full-time/part-time status.
. . . and Variables Reconstructed
- Academic intensity and quality of high school curriculum. This is the most elaborate construction in the study. It includes Carnegie units in 6 academic areas, accounts for highest mathematics studied, remedial work in English and math, and advanced placement. The construction results in a criterion-referenced scale with 40 gradations.
- Educational aspirations. Traditionally defined on the basis of a single question asked in the senior year of high school. Reconstructed on the bases of 6 pairs of questions asked in both 10th and 12th grades, and on the principles of consistency and level. The result is a statement of "anticipations," not "aspirations."
- First institution/date of attendance in postsecondary education. Redefined from college transcript data to exclude false starts and incidental attendance in the summer following high school graduation.
- Transfer. The classic form of community college to 4-year college transfer is now a sub-set of a larger multi-institutional attendance pattern universe defined here in terms of 9 sets of institutional-type combinations. Transfer as we knew it has been replaced by what one might call "portfolio building." But the classic form of transfer is an extremely effective route to bachelor's degree completion.
What We Learned: Principles to Guide Research and Evaluation
- Institutions may "retain" students, but it's students who complete degrees, no matter how many institutions they attend. So follow the student, not the institution.
- Common sense can tell us what's likely to be important at every step toward the degree. A fierce empiricism will validate common sense.
- Before one accepts a variable simply because it has been used for decades or because a federal agency paid for it, one must examine the bricks and mortar of that variable very carefully. Where the architecture is faulty, the data must be fixed or the variable discardedor one will never tell a true story.
- We should not compute bachelor's degree attainment rates for people who never set foot in a bachelor's degree-granting institution.
- The most useful data lie in the details, not the generalities.
The monograph concludes with "tool box" recommendations to those who execute policy
regarding both pre-college opportunity-to-learn and post-matriculation advisement. The tool
box metaphor is a logical consequence of the analysis. It says that if we are disappointed with
uneven or inequitable outcomes of postsecondary education, we must focus our efforts on
aspects of student experience that are realistically subject to intervention and change. We do not have tools to change intentions or perceptions, or to orchestrate affective influences on students' decisions. The events of students' life course histories through their 20s lie largely beyond the micromanagement of collegiate institutions. But we do have the tools to provide increased academic intensity and quality of pre-college curricula, to assure continuous enrollment, to advise for productive first-year college performance, and to keep community college transfer students from jumping ship to the 4-year institution too early.
The recommendations thus address dual enrollment, direct provision of secondary school
curriculum by college instructors, an 11-month rolling admissions cycle for all 4-year
colleges, using Internet situated courses to keep college students continuously enrolled (even
for one course), implementation of institutional policies restricting the extent of course
withdrawals/ incompletes/repeats, realistic credit loads, and advisement that is both sensitive
The story and its analyses are derived from and apply to a cohort whose history covers the
period 1980-1993. There is another and more contemporary cohort whose history, beginning
in 1988, is still in progress. Will the story-line change? Will the analyses be validated? Will
we have attained greater equity in degree-completion rates for minority students? Have
attendance patterns become even more complex, and more or oriented toward competences and
certifications as opposed to degrees? Only a full data-gathering for this cohort in the year
2000 and the collection of its college transcripts in 2001 will tell.
On Reading Tables In This Study