On March 31, 1994, President Clinton signed Public Law 103-227, which includes Title IX, the "Educational Research, Development, Dissemination, and Improvement Act of 1994" (Act). The Act restructured the Office of Educational Research and Improvement (OERI) and provided it with a broad mandate to conduct an array of research, development, dissemination, and improvement activities aimed at strengthening the education of all students. The legislation directed the Assistant Secretary, in consultation with the National Educational Research Policy and Priorities Board (Board), to develop standards to govern the conduct and evaluation of all research, development, and dissemination activities carried out by OERI to ensure that these activities meet the highest standards of professional excellence. Known as the Phase II standards, they were published in the Federal Register on November 17, 1997. (See http://www.ed.gov/legislation/FedRegister/finrule/1997-4/111797b.html).
The OERI legislation also requires that expert panels be established to review educational programs and to recommend to the Secretary those programs that should be designated as exemplary or promising. The legislation defines "educational program" to include "educational policies, research findings, practices and products." These programs should be disseminated through the Department's National Education Dissemination System (NEDS), the Educational Resources Information Center (ERIC) Clearinghouses, the regional educational laboratories, and other like entities. The legislation further requires that the Assistant Secretary develop standards that describe the procedures the panels will use in reviewing educational programs. For an educational program to be considered for designation as exemplary or promising, the eligible entity must submit to the Secretary a description of the program, program materials, and a discussion of the program that is responsive to the established criteria.
After a review of the public comments on the Phase II standards, it was agreed that for the next year or two it would be best to keep the standards as general as possible. Since the expert panels and program reviewers are composed of experts in the field, the general thinking was that the panels themselves would be competent to establish criteria according to their own disciplines.
OERI established two pilot expert panels, one on mathematics and science, the other on gender equity. These two panels have established procedures for receiving, evaluating, and--to some extent--disseminating exemplary and promising practices in their respective domains.
Mathematics and Science Education Expert Panel
The math/science panel is composed of 15 experts in mathematics or science from around the country. It includes academics, association representatives, regional lab representatives, and practitioners. The panel has established procedures for submissions for mathematics and science, and developed criteria and rubrics that the reviewers can use to evaluate the submissions. The criteria flesh out the standards categories outlined in the Phase II standards, namely, quality of the program, educational significance, replicability, and evidence of effectiveness or success.
Programs that are recommended as promising do not have to meet the rigorous standards for exemplary programs. Part of the purpose of the legislation was to increase the number of good educational practices available to the field. The math/science panel recognizes the need to make a wide variety of good programs available to states, districts, and schools. Programs designated as promising have a role to play in meeting this need. They will, however, be designated as promising for only a five-year period and then either meet the criteria for exemplary programs or be removed from the list of promising programs. (Under extraordinary circumstances, a program may continue to be labeled as promising for up to 3 additional years.)
The math/science panel reviewed 61 mathematics submissions in the 1997-1998 review cycle. Science education programs will be reviewed in the 1998-1999 cycle. The Federal Coordinator of this Panel is Patricia O'Connell Ross who manages the Eisenhower Professional Development Program in OERI's Office of Reform Assistance and Dissemination (ORAD). She may be reached at 202-219-2169 or patricia.ross@ed.gov.
The Gender Equity Expert Panel
The Gender Equity Expert Panel consists of 30 members and started its planning work in the fall of 1995. The panel formed six subpanels: Prevention of Violence and Sexual Harassment; Gender Equity and Disability; Mathematics, Science and Technology; Teacher Education and Professional Development; Vocational Technical Education/School to Work; and Core Gender Equity.
OERI staff, the support contractor, and the subpanel chairs, with help from other panel members and the 100 plus member group of interested colleagues, have developed submission and review guidelines based on the draft OERI Standards for Exemplary and Promising Programs. As part of its pilot phase, the 30-member panel actively solicited nominations from the field and obtained 20 full submissions by the due date of July 1997. The Panel is in the final phases of completing the review of those programs. The Subpanel on the Prevention of Sexual and Racial Harassment and Violence Against Students in Higher Education issued a call for programs in Spring 1998. They received 50 submissions and are also in the final phases of review.
The panel has presented at annual meetings of AERA, the National Coalition For Sex Equity In Education, and written extensively about the Gender Equity Expert Panel to guide its development and to help interested colleagues participate in Panel related activities. Susan Klein, in the Knowledge Applications Division of ORAD is the Federal Coordinator of this Panel. She may be reached at 202-219-2038 or sue.klein@ed.gov.
Safe, Disciplined, and Drug-Free Schools Expert Panel
The U.S. Department of Education's Safe and Drug-Free Schools Program (SDFS), in cooperation with the Office of Educational Research and Improvement, has established the Safe, Disciplined, and Drug-Free Schools (SDDS) Expert Panel. The purpose of the Expert Panel is to oversee a process for identifying and designating as promising and exemplary school-based programs that promote safe, disciplined, and drug-free schools. Once programs are designated as exemplary or promising, the Department will disseminate information about the programs and will encourage their use in new sites. The Expert Panel initiative is a way of enhancing prevention programming by making schools aware of alternative programs that have proven their effectiveness when judged against rigorous criteria.
The 18-member SDDS Expert Panel is composed of education practitioners; researchers; school reformers; evaluators; and representatives from local and state education agencies, businesses, and institutions of higher education.
The Panel held its second meeting in October 1998 to consider finalization of its review criteria and guidelines and expects to issue its first "call" for program submissions in November 1998. Programs interested in being designated by the U.S. Secretary of Education as promising or exemplary can access the Panel's application package, including review criteria, at this website as soon as they become available. Printed copies can be requested from RMC Research, the project's contractor, at 800-258-0802 or from lvandegriff@rmcres.com. Submissions will be due in Winter 1999.
The Panel will make use of a cadre of field-based reviewers, including many practitioners, to assist with program review. Individuals wishing to be considered to serve as reviewers should complete a Reviewer Profile Form available from RMC Research at the number or e-mail address noted above. The Panel will select approximately 50 reviewers who will be trained in early Spring 1999 to assist with program review in April and May.
It is expected that the first group of promising and exemplary prevention programs will be announced by the Secretary of Education in Summer 1999.
The Federal Coordinator for this Panel is Ann Weinheimer, Senior Program Analyst in the Safe and Drug-Free Schools Program. She can be reached at 202-708-5939 or ann.weinheimer@ed.gov.
Educational Technology Expert Panel
The Educational Technology Expert Panel was established in Summer 1998. The 17-member panel is composed of education practitioners; researchers; school reformers; evaluators; and representatives from local education agencies, institutions of higher education, businesses, foundations, and state and federal agencies.
The Panel held its first meeting in mid-September to discuss its plans to identify and recommend to the Secretary promising and exemplary "programs" in the area of educational technology. The types of "programs" that will be considered this year will be shared in the Spring of 1999. The Panel proposes to make a call for "programs" in mid-April 1999 with applications due in mid-May 1999.
The federal coordinators for this Panel are Cheryl P. Garnette, who serves as the Acting Director for the Learning Technologies Division in OERI's Office of Reform Assistance and Dissemination, and Diane Aleem, who serves as the Staff Coordinator. Ms. Garnette may be reached at 202-219- 2267 or HREF="mailto:cheryl.garnette@ed.gov">Cheryl.Garnette@ed.gov SIZE="4" COLOR="#000080"> and Ms. Aleem at 202-219-2148 or Diane_Aleem@ed.gov.
The math/science panel recommended that a panel of evaluation experts review their submissions to see if the results provided are valid before a program receives an exemplary designation. This idea has been adapted and applied to all Panels to ensure consistency and reliability in the decisions on the programs that the Panels recommend that the Secretary of Education designate as exemplary. As a result, all the topic focused expert panels will use this Impact Review Panel (IRP) of evaluation experts to obtain advice on whether each individual program being considered as exemplary has evaluation design and evidence to support meaningful claims of success. The IRP will examine whether the claims are supported by evidence that is logically and, as appropriate, statistically valid.
Contractor
The technical assistance contractor for the System of Expert Panels is RMC Research Corporation of Portsmouth, NH. Susan Klaiber serves as project director and Peggy Simon as assistant director. They can be reached at (800) 258-0802 or sklaiber@rmcres.com or psimon@rmcres.com.
Personnel
Peirce Hammond, the Director of ORAD, and Sharon Bobbitt, the Director of ORAD's Knowledge Applications Division (KAD) are providing overall support and leadership for this new System. Within KAD, The Acting Team Leader and COTR for the System of Expert Panels is Stephen O'Brien. As noted previously, The Department of Education Staff serving as coordinators for the specific panels are: Patricia O'Connell Ross for the Mathematics and Science Education Expert Panel, Susan Klein for the Gender Equity Expert Panel, Ann Weinheimer for the Safe, Disciplined and Drug-Free Schools Expert Panel and Cheryl Garnette and Diane Aleem for the Educational Technology Expert Panel.
