A r c h i v e d  I n f o r m a t i o n

5. Evaluating Correctional Education

How does correctional education work? Does it improve the skills of inmates and result in employment once released? And most importantly to policymakers and the public, does correctional education reduce the likelihood of its participants committing crimes once released and returning to prison?

Research addressing these questions is limited and inconclusive. As long as resources remain tight and other programs competing with correctional education can demonstrate results, correctional education will suffer. Evaluating correctional education, or any other type of program offered to inmates, is not an easy task nor is it inexpensive. Recidivism, the rate at which released prisoners return to jail or prison, can be difficult to measure. Measuring recidivism requires some type of follow-up effort and needs to allow for a sufficient period of time to elapse between release and possible return. It also requires devoting resources to individuals who are no longer being served by the facility. However, correctional education programs cannot be discouraged by the difficulties of measuring recidivism.

This section of the handbook describes the components of good evaluations of correctional education programs. While it focuses heavily on recidivism studies, it also considers other potential outcomes that program administrators and evaluators might find useful. While these other outcomes are not as powerful measures as recidivism, they nonetheless provide some information regarding behavioral changes that may be related to recidivism.

Recidivism: The Ultimate
Measure of Success

Measuring recidivism requires considerable planning and commitment. A good recidivism study rules out the possibility that other factors (e.g., participant characteristics, participation in another program, post-release support activities) explain the observed outcomes. A good recidivism study includes some type of comparison or control group so that comparisons can be made between those who participate in correctional education and those who do not.

Below, several issues that should be considered when designing and implementing recidivism studies are discussed.

Issue #1: How Do You Measure Recidivism?

The precipitating event refers to the activity that ultimately determines whether or not a former inmate is considered a recidivist. Some research uses the revocation of parole as the precipitating event, other studies use rearrest, and still others use reconviction. Some studies have also examined whether the seriousness of the crime for which the individual was reconvicted was greater or lesser than the previous conviction.

The second measurement dimension is the length of time between release from prison and the precipitating event. Obviously, the longer the period of time between the release and return dates, the greater the likelihood that former inmates will have recidivated. Some evaluations have used six months as an initial cutoff after release while others go indefinitely. Most evaluations limit the time period being examined; one to two years after release is a reasonable length of time to use.

Three Dimensions of a Recidivism Measure Precipitation event Time between release and return Readmitting facility

The third measurement dimension that needs to be considered in measuring recidivism, if the precipitating event determining recidivism is violation of parole or reconviction, is the facility to which the individual returns. If programs are only interested in determining whether former inmates return to the facility from which they were released, data collection becomes relatively easy. This means that the program does not need to link their records to those of other prisons or systems within the state or elsewhere in the nation. However, limiting definitions of recidivism to a single facility can produce misleading results. State-level data systems provide ready access to information across facilities and thus eliminate the problems of tracking rearrests or reconvictons that occur in different locations across the state. In addition, there are federal databases that contain arrest information.

Issue #2: How Can You Be Sure that Your Results Can Be Attributed to Correctional Education?

Self-selection bias can produce outcomes that result from characteristics of the participants and not participation in correctional education.

Whenever participants of correctional education programs show lower recidivism rates than nonparticipants, the question of self-selection bias can be raised. Cynics, as well as sophisticated statisticians and methodologists, might argue that it is not participation in correctional education that is lowering recidivism rates. Instead, correctional education participants are less likely to recidivate because of certain characteristics they possess prior to program entry. As an extreme example, one might realize that those who participated in correctional education are less likely to have substance abuse problems than those who did not participate in these programs. Can the lower recidivism rate of correctional education participants be attributed to education gains or is it the case that their lower rates of substance abuse is what really reduces the recidivism rates of this group?

When confronting the possibility of self-selection bias, evaluation designs have several options. The ideal solution for eliminating self-selection bias is a procedure commonly referred to as "random assignment." Under this system, some inmates are randomly assigned to correctional education (the treatment group) while others are not assigned to correctional education (the control group). By randomly assigning inmates to these two groups, the problem of self-selection bias is theoretically eliminated. However, correctional education, as well as many other social service programs, has been reluctant to employ this methodology for several good reasons. In some states, correctional education is mandated for inmates who do not meet certain stated requirements; thus inmates, by law, must be assigned to these programs. Even when there are no mandates or requirements, program officials often believe that those who want to participate in a program should be given every possible opportunity. Finally, there is always the possibility of litigation because some inmates were provided certain opportunities or privileges that others were not.

There are several alternatives to random assignment that are frequently used in many different evaluations. One alternative is to use an inmate control group. Participants (those who are enrolled in correctional education programs) are matched with nonparticipants (those who are not enrolled in correctional education programs) on a number of characteristics that could potentially affect the outcome (recidivism). Some of these factors might be age, highest level of education attained, number of prior convictions, and prison behavior indicators. The two groups would "look alike" with one major exception: inmates in one of these groups participated in a correctional education program and inmates in the other group did not.

Another technique that provides a ready-made comparison between a group of program participants and a group of nonparticipants is available to facilities that have extensive program waiting lists. Unless inmates are assigned to programs on the basis of criteria that result in major differences between program participants and those who are placed on waiting lists, the outcomes of these two groups can be compared. This technique assumes that not all of those on a waiting list are actually placed in programs before they are released from the facility.

Recidivism studies of correctional education participants must rule out the possibility that other factors may explain the observed outcomes.

There are also other possibilities for comparing participants in correctional education to others who did not participate. The recidivism of correctional education participants can be compared to the recidivism of participants in other types of programs, an institution-wide recidivism rate, or a state-wide recidivism rate. The prior recidivism histories of inmates can be used to determine whether program participants stay out in the community longer after they have been in correctional education programs. And as pointed out earlier, some studies have attempted to determine whether the severity of the crimes before and after participating in correctional education change.

All of these efforts attempt to eliminate the possibility that something other than participating in correctional education programs might be producing the observed outcome, whether it is positive or negative. Other prison experiences with potentially positive "treatment effects" need to be ruled out as well as personal inmate characteristics that might predispose an individual toward crime or rehabilitation. And there is always the possibility that a combination of rehabilitative-oriented programs, including correctional education, is the strongest predictor of success.

A note on statistics. There are almost as many statistical procedures and approaches that can be employed to analyze recidivism data as there are measures of recidivism. The choice of appropriate statistical measures is certainly beyond the scope of this handbook. At a minimum, if correctional education program staff do not have access to in-house statisticians (and many states do have research and evaluation units with staff trained in research methodology and statistics), they should consider looking to their local university or to other outside consultants for statistical assistance. Graduate students in search of thesis data, for example, may be willing to volunteer their time in exchange for data.

Issue #3: How Do You Collect Information about Inmates Once They Are Released into the Community?

There are a variety of resources for collecting post-release data. Because of the difficulty of collecting post-release data and problems of reliability with some data sources, program officials should consider a multi-faceted approach for collecting information on inmates after they are released. First, there are a number of state and federal databases that contain arrest information. Program officials should plan to access these data when designing their data systems and evaluating their programs. Second, parole officers should be contacted on a regular basis to obtain information about those inmates released to their supervision. However, because parole officers are often overburdened, have poor record keeping systems, and are viewed, at times, with hostility by former inmates, program officials should attempt to use their own staff to contact inmates regarding their post-release experiences. Program staff often have good relationships with their former student inmates.

When collecting post-release data, program officials should collect information beyond rearrest and reconviction. Specifically, data regarding additional education and training, as well as other services received, are needed to determine whether it is correctional education or other factors that produces any observed outcomes.

Issue #4: What Do You Do If Your Results Indicate that Correctional Education Does Not Reduce Recidivism?

Program administrators who have invested considerable energy in developing correctional education programs and those who have funded these programs hope to learn that their efforts benefit both the participants and the larger society. But what happens when evaluation results indicate that correctional education has no impact, or possibly a negative impact, on recidivism? First and foremost, don't panic! Clearly, evaluating a program like correctional education is a complex undertaking. If results do not indicate what is expected, several steps should be taken to ensure that the results are accurate:

• Reconsider the evaluation design. Given the variety of factors that can contribute to an outcome such as recidivism, it is entirely possible that some other data elements need to be considered.
• Examine the measures of correctional education used in the model. How correctional education is defined and operationalized might affect results. Simply considering participation in correctional education, for example, may not be enough. It may be necessary to include progress in the program (e.g., changes in test scores, completion of program) or other information about program content, presentation, etc.
• Examine the measures of other data elements included in the model as well. How other components of the model are defined and measured may also need fine tuning. Sit back and think about what you believe is being measured in relation to the definition of the data element that is measuring it.
• Consider other statistical approaches. As noted, the range of statistical techniques that can be used in evaluation efforts is wide and beyond the scope of this handbook. Using an inappropriate statistic, however, can mask results. Review findings with a colleague or someone who is trained in statistics.
• Most importantly, try again. Do not assume that there is only one way to design an evaluation and to analyze the results. If you have reason to believe that the findings either are inaccurate or do not capture what is occurring, conduct another evaluation.

Of course, it is possible that the evaluation results which indicate that correctional education has little or no impact on recidivism or other outcomes are accurate. At this point, program administrators need to consider ways to improve the program itself and then re-evaluate its impact.

Other Measures
of Program Success

Academic Gains. Why might correctional education reduce recidivism? Possibly because skills obtained in classes provide opportunities for inmates that they may not have had without these classes. Measuring changes in test scores at different points in time tracks progress and provides an indicator of what skills inmates are obtaining. This measure is relatively easy to obtain yet it provides important data both in and of itself and as a potential factor for explaining why correctional education might reduce recidivism.

Behavioral Changes. Several different types of behavioral changes might result from participation in correctional education. Prison infraction incidences, for example, can be monitored to determine whether correctional education participants receive fewer infractions than other inmates or whether they receive fewer infractions than they themselves did prior to taking courses. In some programs, this measure is not as meaningful as it might be in others since inmates who violate prison rules are automatically dropped from programs.

One facility has collected data on inmates' use of medical services and determined whether this use was appropriate or inappropriate. Inmates in correctional education programs were somewhat less likely to use medical services inappropriately than those who were not in these programs. The use of other types of prison services can also be measured and examined to see if their usage is related to participation in correctional education.

Early Release Savings. Some correctional education programs offer as an incentive to participants early release from jail or prison. The amount of money saved by releasing inmates earlier than originally scheduled can be calculated by simply multiplying the estimated cost per day of incarceration by the number of days released early.

Employment and Increased Earnings. Post-release employment and the earnings that go along with employment are yet another possible outcome of correctional education participation. Employment and earnings are also another factor that could explain the link between correctional education and recidivism. Obtaining information on employment and earnings is not a simple task, however. It requires followup once the individual is released and possibly linking to other data bases.

A Final Thought:
Don't Be Intimidated!

Evaluating correctional education programs is a complex undertaking. It requires planning, resources, commitment, and patience. Yet it needs to be done.

In this section, several issues related to conducting evaluations are described. It may seem like an overwhelming endeavor, one that simply cannot be undertaken with available staff and resources. Very few, if any, programs will be able to design and conduct an evaluation that accounts for all of the factors discussed above. Evaluation efforts can be developed in a step-by-step fashion, beginning with simple outcome measures and adding others over time. Evaluation efforts can also begin with a small group of inmates so that procedures and measures can be tested. Evaluation efforts can build upon existing data already gathered and can tap a variety of resources that can assist you (See Exhibit 13). Programs have to begin somewhere and it is better not to wait until the day when maximum resources and staff are available to undertake the ideal evaluation.

Exhibit 13

Possible Resources for Evaluation

Need help with Evaluations? Consider . . . Contacting your local college or university Using a graduate student looking for an interesting project Inquiring among your staff (untapped skills abound!) Getting in touch with other facilities that have conducted evaluations Training inmates to do data entry (if permitted) or programming Locating businesses willing to donate time and possibly equipment Obtaining volunteers from the community

-###-

---

[4. Reporting Correctional Education Data] [Selected Bibliography]