National Institute on Disability and Rehabilitation Research (NIDRR)
In 2009, NIDRR contracted with the National Academy of Sciences (NAS) to conduct an external evaluation of NIDRR and its grantees. The resulting report was released in 2012 and titled, Review of Disability and Rehabilitation Research: NIDRR Grant making Processes and Products. The study interviewed NIDRR staff and management, surveyed customers, stakeholders and sampled past peer reviewers.
The evaluation report resulted in a number of recommendations for process improvement which NIDRR is working to implement. One of the recommendations made by NAS was that NIDRR should develop a more systematic and ongoing evaluation process.
With this in mind, NIDRR recently contracted with Abt. Associates, Inc. in 2012 to develop a ten year evaluation plan and corresponding tools for monitoring and assessment of the NIDRR research portfolio. Abt. Associates, Inc. has recently begun working on this evaluation plan and related tools.
In 2013, Abt. Associates, Inc. completed its ten year evaluation plan. The plan includes a set of research questions aimed at assessing the effectiveness and efficiency of NIDRR’s operations as well as the quality and impacts of NIDRR-funded activities and products. These questions are grouped around four domains which are presented below:
Domain 1 questions:
explores the quality and impacts of NIDRR funded activities and products.
Domain 2 questions:
explores the effectiveness of NIDRR’s process for determining grant priorities.
Domain 3 questions:
explores the effectiveness of NIDRR’s funding decision process.
Domain 4 questions:
explores the effectiveness of NIDRR’s process of grants monitoring.
Each of these domains has a subset of questions to be addressed as part of the overall evaluation. Abt. Associates then identified tools that needed to be developed to answer these questions.
A broad listing of potential tools is listed below:
- Analysis of the APR data
- Grantee and trainee survey
- Bibliometric analysis
- NIDRR staff survey
- Expert panel review of grantee products
- NARIC data abstract
- NIDRR partner survey
- KT feasibility study
- Advisory board survey
- Peer review survey
As part of the plan, Abt. Associates, Inc. also developed a schedule of evaluation activities which identifies the programmatic focus for each year’s analysis, and identifies specific questions to be addressed and the data source. During 2013-2014, Abt. Associates, Inc. begins the second phase of this contract by developing the tools for collecting the data to address key questions.