Skip Program Navigation
National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR)

Current Section
 Office of Special Education and Rehabilitative Services Home
National Institute on Disability, Independent Living, and Rehabilitation Research External Evaluation Page

In 2009, NIDILRR contracted with the National Academy of Sciences (NAS) disclaimer to conduct an external evaluation of NIDILRR and its grantees. The resulting report was released in 2012 and titled, Review of Disability and Rehabilitation Research: NIDILRR Grant making Processes and Products. The study interviewed NIDILRR staff and management, surveyed customers, stakeholders and sampled past peer reviewers.

A total of 148 outputs, produced by the 30 grantees in the study, were rated using the quality criteria developed by NAS. The committee found that 75% of these outputs were rated “good to excellent.”

The committee concluded that NIDILRR grants have produced valuable research, tools, and other outputs for advancing the field of disability and rehabilitation research in line with the agency’s mandate. Improvements to NIDILRR’s priority setting, peer review, and grant management processes, as well as considerations of alternative evaluation goals, and strategies, would further enhance the quality of these processes, their results, and the agency’s efforts to improve the lives of individuals with disabilities. Click here to see quote within the online NAS report

The evaluation report resulted in a number of recommendations for process improvement which NIDILRR is working to implement. One of the recommendations made by NAS was that NIDILRR should develop a more systematic and ongoing evaluation process.

With this in mind, NIDILRR recently contracted with Abt. Associates, Inc. disclaimer in 2012 to develop a ten year evaluation plan and corresponding tools for monitoring and assessment of the NIDILRR research portfolio. Abt. Associates, Inc. has recently begun working on this evaluation plan and related tools.

In 2013, Abt. Associates, Inc. completed its ten year evaluation plan. The plan includes a set of research questions aimed at assessing the effectiveness and efficiency of NIDILRR’s operations as well as the quality and impacts of NIDILRR-funded activities and products. These questions are grouped around four domains which are presented below:

Domain 1 questions:
explores the quality and impacts of NIDILRR funded activities and products.

Domain 2 questions:
explores the effectiveness of NIDILRR’s process for determining grant priorities.

Domain 3 questions:
explores the effectiveness of NIDILRR’s funding decision process.

Domain 4 questions:
explores the effectiveness of NIDILRR’s process of grants monitoring.

Each of these domains has a subset of questions to be addressed as part of the overall evaluation. Abt. Associates then identified tools that needed to be developed to answer these questions.

A broad listing of potential tools is listed below:

  1. Analysis of the APR data

  2. Grantee and trainee survey

  3. Bibliometric analysis

  4. NIDILRR staff survey

  5. Expert panel review of grantee products

  6. NARIC data abstract

  7. NIDILRR partner survey

  8. KT feasibility study

  9. Advisory board survey

  10. Peer review survey

As part of the plan, Abt. Associates, Inc. also developed a schedule of evaluation activities which identifies the programmatic focus for each year’s analysis, and identifies specific questions to be addressed and the data source. During 2013-2014, Abt. Associates, Inc. begins the second phase of this contract by developing the tools for collecting the data to address key questions.

Top Top

Print this page Printable view Bookmark  and Share
Last Modified: 04/08/2015