Choose the Right Policy Evaluation Method

Q1. A ministry wants to know which of two housing support models is most cost-effective for improving the same outcome. What is the best evaluation choice?
- Run a randomized trial only and ignore costs
- Use an economic evaluation that combines a robust impact estimate with programme costs
- Choose a process evaluation because it avoids outcome measurement
- Use no evaluation because cost-effectiveness cannot be studied
Answer: Use an economic evaluation that combines a robust impact estimate with programme costs[[cite:1]][[cite:2]]
Q2. A pilot cannot be randomized, but the agency has strong administrative data and can build a credible counterfactual. Which approach is the best fit?
- Randomized trial, because it is always the best design
- Quasi-experimental or other counterfactual analysis
- Process evaluation only, because it is always enough
- Do nothing until new survey data are collected
Answer: Quasi-experimental or other counterfactual analysis[[cite:3]][[cite:4]]
Q3. Officials are deciding whether to withhold a new service from some people. There is genuine uncertainty about effectiveness, cost-effectiveness, and scalability. What does the OECD source suggest?
- A randomized trial can be especially justified, with concerns reduced through consent, waitlists, staggered roll-out, or current standards of care for the control group
- A randomized trial is never ethical in this situation
- A process evaluation is the only ethical option
- The best answer is always to avoid evaluation
Answer: A randomized trial can be especially justified, with concerns reduced through consent, waitlists, staggered roll-out, or current standards of care for the control group[[cite:5]][[cite:6]]
Q4. A department is asking whether a programme is being delivered as intended, not whether it caused better outcomes. Which evaluation is most appropriate?
- Randomized trial because all questions need causal attribution
- Process evaluation because the question is about implementation and fidelity
- Economic evaluation because it focuses on costs only
- Quasi-experimental analysis because it always answers implementation questions
Answer: Process evaluation because the question is about implementation and fidelity[[cite:7]]
Q5. A pilot produced disappointing results, and the team needs to know whether to discontinue it or redesign it. What approach best matches the OECD source?
- Mixed-method evaluation using process and qualitative data to interpret the result
- Randomized trial only, because mixed methods are unnecessary
- Economic evaluation only, because it answers all questions
- No evaluation, because disappointing results cannot be explained
Answer: Mixed-method evaluation using process and qualitative data to interpret the result[[cite:8]]
Q6. A ministry has fragmented administrative systems and privacy approvals that make access slow, and the team is also short on staff who can assess trial quality. What should it conclude?
- Impact evaluations may be resource-intensive and harder to deliver, and the ministry needs stronger analytical skills for assessing quality and research integrity
- A randomized trial is always easy once a policy is announced
- Data access and analytical skills do not matter for evaluation
- Only economic evaluation is possible in every case
Answer: Impact evaluations may be resource-intensive and harder to deliver, and the ministry needs stronger analytical skills for assessing quality and research integrity[[cite:9]][[cite:10]][[cite:11]][[cite:12]]

cost-effective way of increasing school attendance

rich administrative data when they are available

ensure they are following the same people

Choose the Right Policy Evaluation Method

Related Content From The Pandipedia