Data & Research EVALUATION OF PROGRAMS
Evaluation Plan
National Assessment of Vocational Education (NAVE)

Overview of the Evaluation Plan

I. Introduction

Federal support for vocational education, and for understanding its consequences, has a long history. As was true with previous vocational legislation, the recently amended (1998) Carl Perkins Vocational and Technology Education Act (Perkins III), directs the Secretary of Education to complete an "independent evaluation and assessment of vocational and technical education programs under this Act," in this case, by 2002. As currently designed, this national assessment plan will include multiple components. This paper discusses the main issues that NAVE will address, and the research strategies that will be undertaken to address those issues.

Perkins III (section 114) directs the NAVE to address a wide range of topics, including: (1) implementation of state and local programs; (2) impact of changes in federal funding formulas; (3) teacher quality and teacher supply and demand; (4) student participation in vocational education, (5) academic and employment outcomes; (6) employer involvement and satisfaction with vocation education programs; (7) education technology and distance learning; and (8) the impact of accountability requirements on program performance. Perkins III also directs the Secretary to appoint an Independent Advisory Panel to provide advice on conducting the NAVE and to submit to Congress its own independent analysis of NAVE findings and recommendations (see Appendix A for list of panel members).

Congress has mandated a final NAVE report in 2002 to provide it with information that can guide reauthorization of the Perkins Act in 2003. Of primary interest is how some of the new provisions in the law have been implemented. Unfortunately, many of the important changes in accountability, integration with workforce development, and increased flexibility will have barely begun. Most states have opted for "transitional plans" which in effect, defer implementation of Perkins III provisions until October 2000 (Figure 1). Given the July 2002 date for reporting NAVE results to Congress, data collected during the 2000/2001 school year will reflect the very early efforts made in response to specific new provisions in Perkins III. Conclusions regarding the longer-term prospects for Perkins III will require further research.

At the same time, however, Perkins III continues to emphasize several major themes reflected in the 1990 Perkins amendments. These include the integration of academic and vocational education, broadening the focus of vocational education to emphasize industries and careers, and strengthening the links between secondary and postsecondary education through tech-prep and other strategies. The current NAVE will be able to track the extent to which these longer run themes are reflected in school practice, and where possible, what effects they have on outcomes for students.

All of these objectives will be addressed in a series of studies soon to get underway. Two factors are guiding the design of the NAVE agenda: (1) the current policy environment, and (2) the specific research questions that reflect policymaker and practitioner interests.

A. Policy Context for the NAVE

Over the past two decades, three prior national assessments of vocational education have been conducted.[1] Each study sought to focus upon what were thought to be the major issues facing vocational education at the time—funding, special populations, and economic competitiveness of the workforce. The current NAVE must also be sensitive to the dominant educational issues of its time.

Vocational education is a field in transition, prompted by sweeping changes in state and local education priorities. New goals, program offerings, and terminology increasingly characterize vocational education. Federal legislation has encouraged several major changes—from an historic emphasis on entry-level job preparation in semi-skilled occupations to a broader focus on preparation for careers that offer high wages and requires higher level skills; from preparing students to enter the workforce directly after high school to providing students with the choice of pursuing employment or attending college, or as is increasingly the case, doing both simultaneously; and from expecting vocational students to do less well in school than other students, to holding such students to the same academic standards as others. Many of the overarching issues that NAVE will address will consider whether this transition in the field of vocational education is "on track."

Several key policy concerns are likely to shape this assessment:

  • Federal Funding For Vocational Education. Federal budgets are widely regarded as a basic indicator of policy priorities. Although overall funding on Department of Education (ED) programs has increased by 177 percent from FY 1980-FY 1999, vocational education funding increased by only 47 percent during the same period (Figure 2).[2] In 1980, funding for vocational education was about 6 percent of total ED expenditures; it has now shrunk to about 3 percent.

    [Graphic data not available]

    These trends strongly suggest that for nearly two decades vocational education has been increasingly viewed as less worthy of investment than other areas. One reason for this pattern may be that vocational education has historically prepared students for work at a time when priorities have shifted toward academic preparation for college. To the extent that vocational education has changed to address these as well as more traditional objectives, it will be important for NAVE to describe and assess this development.

    [Graphic data not avialable] Although overall federal funding for vocational education has not changed substantially, the Perkins Act remains the largest single source of ED funds directed to high schools. Comparing ED's three major sources of funding for high schools (only a share of the funds from each program is directed at high schools), Perkins Act funds spent to help high school students are more than Title I and TRIO funds combined (Figure 3).[3]

    Given the major role that Perkins funding plays at the high school level relative to other ED programs, it makes sense to think about how this investment contributes to high school improvement generally, and the contribution that vocational education makes, or could make, to such efforts.

  • Academic Reform. There is little question but that the principal focus of recent education reforms has been on improving students' academic achievement and increasing their opportunities to attend college. Federal vocational education policy now places top priority upon ensuring that vocational education students are academically well prepared for both careers and/or success in postsecondary education. But what is, or ought to be, the responsibility of vocational educators for ensuring that students leave high school with both a solid academic foundation as well as technical skills? One major goal of NAVE should be to help Congress better understand what the contribution of vocational education is to achieving these objectives and how this contribution can be increased.

  • Changes in Perkins III. Although Congress did not alter the basic structure of the Perkins Act in the 1998 reauthorization, it did make several important substantive changes. Among these are: (1) increased emphasis on academics; (2) greater flexibility in the use of funds through elimination of major set-asides for gender equity and other rules governing the use of funds; (3) a higher proportion of funds directed to local programs and the establishment of a 10 percent reserve fund; and (4) creation of a "higher stakes" accountability system. At the same time, the Workforce Investment Act of 1998 (WIA) also encouraged greater integration of vocational education and the workforce development system. Implementing these changes will be a major challenge for school districts and postsecondary institutions.

B. Main Policy Issues

Given the educational and political context in which Perkins III operates, as well as specific legislative changes recently enacted, the NAVE will be guided by several main questions:

  1. What strategies improve the performance of "vocational students" and how does, or can, vocational education contribute to improving academic and occupational skills, access to postsecondary education, and earnings?

    Standards-based reform is fundamentally altering policies and practices from kindergarten through high school. NAVE will examine the role of standards—both academic and occupational—in vocational education, the effects on students' academic and technical preparation of promising reform strategies, and the role of federal policy, as implemented by states and communities, in facilitating implementation of effective programs of study.

  2. What are the pathways by which sub-baccalaureate students prepare for careers, and what is the contribution of workforce reform efforts to improving their training?

    In passing Perkins III and the WIA, Congress was concerned that federally supported job training programs operated independently of vocational education. Congress expects that linking Perkins III and WIA together will encourage better integration of vocational education and workforce development policies. Issues to be examined include: (1) the role of postsecondary vocational in developing state workforce training plans; (2) the importance of unified planning; (3) the early impact on postsecondary institutions (e.g., programs of study, one-stop career centers, accountability) of efforts to align Perkins III and workforce development, and (4) patterns of enrollment and participation in postsecondary occupational education, and their relationship to outcomes and impact.

  3. Is the policy shift from set-asides and legislative prescription to flexibility and accountability likely to improve program quality and student outcomes? How do special populations fare?

    For the past two decades, federal policy has focused on serving those most at-risk, commonly termed the "special populations." Perkins III represents a major shift in direction—eliminating set-asides for "single parents...," requirements that local funds be prioritized to serve the highest concentrations of special populations, and requirements to coordinate with the Individuals for Disabilities Education Act. In its place is an increased emphasis on accountability, including the requirement that states track the progress of special population groups. Has increased flexibility resulted in changes in educational priorities or practices? Have "at-risk" populations been helped or hurt as a result? Are accountability requirements improving the quality of vocational education for all students?

II. NAVE Research Agenda


Figure 4

Research Strategies to Address
Main Policy Issues

Secondary Vocational Education

  1. Participation, Outcomes and Impact
  2. Quality
  3. Effectiveness of Promising Strategies

Postsecondary Occupational Education

  1. Participation, Pathways, Outcomes, and Impact
  2. Alignment with WIA

Program Management

  1. Accountability
  2. Funding

Other

Addressing the primary vocational policy issues requires a set of interrelated but distinct studies. No one study could encompass the broad array of interests and questions. Moreover, the overall research agenda calls for diverse data collection and analysis methods, including: qualitative case studies, national surveys, examination of existing databases, and sophisticated econometric estimation. Finally, while some broad themes are relevant to both secondary and postsecondary vocational education, each also has its own key issues.

For these reasons, the NAVE plans to undertake studies in seven main areas (Figure 4). These are described below, grouped together by major topic: secondary vocational education, postsecondary vocational education, and program management.



1 1982, 1989, and 1994.

2 Figure 2 shows the trends in federal funding for vocational education over the past 20 years, and compares funding for vocational education with overall funding for Department of Education programs.

3 Figure 3 compares ED funding of high schools from three major sources—the Perkins Act, Title I of Improving America's Schools Act (IASA), and the TRIO program.

Table of Contents NextSecondary Vocational Education


 
Print this page Printable view Send this page Share this page
Last Modified: 09/16/2003