RFP ED-00-R-0040 Amendment No. 1 Effective Date: June 14, 2000 The purpose of this modification is to answer the questions that have been submitted in response to this solicitation. 1. Question: The RFP requires a response rate of 95 percent for state directors and directors of training programs and 90 percent for recent graduates. While 95 percent response is attainable for the directors, this goal is unrealistic for graduates contacted by telephone or mail. Will the government reduce this requirement substantially or consider other designs and modes of data collection? Response: ED wants some effort in obtaining a high response. However, ED does not have unlimited funds. If the respondents are resistant then it could prove to be expensive to try to reach an arbitrary response rate. There may be more cost-effective ways to assure that the sample is not biased by the lack of response than repetitively asking for information until a certain high response rate is attained. The goal is to limit the bias and not to get a particular response rate. The offeror could propose lower response rates, if the offeror provided adequate justification concerning why those response rates would be sufficient. It will be much harder to get a high response rate from recent graduates than from training program directors and directors of Human Resources. ED set the goal lower in recognition of that fact. But, until ED conducts the survey ED won't know whether 95 or 90 percent or whatever is reasonable for any of these three surveys. ED believes that it will be fairly easy to survey the recent graduates. ED believes that the grantees have excellent records of their graduates as required under the payback provision. Since these are graduates of Master's level programs and are likely to be working in the profession, it is assumed that the response rate will be high. As a matter of estimating the cost of this study, this is the assumption. As a result, the RFP requires the contractor to test this assumption. In the section for "Instructions for the Technical Proposal" it states: Task 4.2 Sampling Plan The Statement of Work suggests three surveys: 1) approximately 83 Human Resource Directors; 2) 86 RSA-funded training programs; and 3) 1,600 graduates who have graduated in the last four years. The Offeror must provide a sampling design for the three surveys and how they intend to assess the feasibility of the approach suggested in the Statement of Work. The Offeror must also suggest other cost- effective approaches if the suggested approach proves not to be feasible. The offeror is required to have a plan to assess the feasibility and an alternative approach as part of the offer. The government will reduce the rate requirement if a cost-effective alternative way to limit response bias is suggested. And not only will the government consider other designs, it requires those designs as part of the offer. 2. Question: The RFP requires a sample of 1,600 recent recipients of Master's degrees in rehabilitation counseling. The RFP suggests that the sample will be drawn from among the supported graduates, since the government expects that the training programs will maintain accurate records of the whereabouts of their graduates who received scholarships (p. 7). We expect that training institutions will have less success tracking scholarship recipients who are not meeting their payback requirement. Furthermore, those not meeting their payback requirements will be far less likely to respond to a survey on the subject. Will the government consider another design that meets the six study objectives but yields a smaller sample of of recent recipients of Master's degrees in rehabilitation counseling? Response: The training program supports approximately 400 graduates a year. After four years approximately 1,600 graduates would be supported. So the 1,600 number is the total expected number of scholars and not a sample. It takes four years to payback the support so only a few will have satisfied the payback requirement. For the purpose of cost, assume that it will be relatively easy to obtain a census of 1,600 scholars. It is possible that scholars who do not intend to payback will be unwilling to respond to a survey. But, ED has heard that often graduates find high paying jobs in industry and are quite willing to payback the limited support rather than work at a lower paying job for the State. ED won't know until the feasibility is tested. ED suspects that it may be difficult to find and obtain information from many of the 1,600 scholars. As a result, other designs are not only welcome but also required. As this offeror suggests the most likely approach is to take a sample in a way that still allows ED to address the six study questions. 3. Question: For which of the three anticipated surveys does ED expect an institutional review board to be required? Response: It's not clear which institutional review board is being referred to. Offerors should have their own institutional review boards to review quality. There are review boards at grantee institutions but we don't anticipate these to be a cost factor in this study. 4. Question: In budgeting for the Panel of Experts (POE) should we assume the contractor will be responsible for honoraria for members? Will ED provide facilities for the two-day meeting? Response: Yes the contractor will be responsible for the honoraria for the POE. The contractor will be responsible for providing facilities for the two-day meeting. In fact, the POE is a panel for the contractor. It is advising the contractor and not the government. 5. Question: Other than for the POE, are persons currently employed or receiving funds through RSA training programs precluded from serving as staff or consultants on this evaluation project? Response: Federal employees who work on the RSA training program cannot work for a contractor. It is also a conflict of interest where one is both the evaluated and the evaluator. The person would have to quit one or the other employer. What may appear to be impartial advice may be strongly influenced by a person's employer. The contractor's work needs to be impartial. It is best when those who work for the contractor are impartial. There is a problem when someone works for or has been a recent advocate for one of the interested parties. Especially, if that person is responsible for conducting the study or contributing to the conclusions and findings of the study. If that person is a consultant and not directly responsible for the conduct of the study, then you still have to be careful to balance the affiliations across your consultants, not an easy thing to do. Offeror's will need letters of commitment from all people not currently employed by your organization. 6. Question: The RFP provides sign-in instructions for hand carried proposals. Are you prepared to receive proposals delivered by Federal Express or other carriers on the morning of the due date? Response: ED is prepared to receive proposals delivered by Federal Express or any other carrier up until June 21, 2000 at 2:00 p.m. Any proposal received after 2:00 p.m. on June 21, 2000 will not be accepted. Proposals should be delivered to the address in Block 8 on Standard Form 33.