Workshop #3: Evaluation Criteria

Author

EPA Teaching Team

View all slides in new window Download PDF of all slides

Step 4: Select the Criteria

Understanding Criteria

Criteria are the standards used to evaluate the potential outcomes of each alternative. They bridge the gap between the analytic and evaluative aspects of policy analysis.

“The most important step for introducing values and philosophy into the policy analysis, because some possible ‘criteria’ are evaluative standards used to judge the goodness of the projected policy outcomes that are associated with each of the alternatives.”
— Bardach & Patashnik, 2020, p. 53

Types of Criteria

  1. Evaluative Criteria: Standards used to assess the desirability of outcomes (e.g., efficiency, equity).
  2. Practical Criteria: Practical considerations related to policy implementation (e.g., legality, political acceptability).

Common Evaluative Criteria

  • Efficiency: Maximizing aggregate welfare or public interest.
  • Effectiveness: The degree to which the policy achieves its objectives.
  • Equity: Fair distribution of benefits and burdens.
  • Political Acceptability: The extent to which the policy is supported by stakeholders.

Example

For a policy aimed at reducing homelessness: - Principal Criterion: Minimize the number of homeless families. - Secondary Criteria: Cost-effectiveness, political feasibility, and social equity.

Selecting and Defining Criteria

  1. Primary Criterion: Directly related to solving the core problem.
  2. Secondary Criteria: Additional factors that influence the desirability of outcomes.

Steps to Define Criteria

  • Maximize/Minimize: Clearly state whether you aim to increase or decrease a particular value.

  • Operationalize: Define metrics for each criterion to ensure clarity and measurability.

    Criterion Metric
    Efficiency Cost per homeless family reduced
    Effectiveness Percentage decrease in homelessness
    Equity Distribution of benefits across demographics

Avoiding Common Mistakes

  • Do Not Confuse Alternatives with Criteria: Alternatives are actions; criteria are standards for evaluation.

    “Alternatives are courses of action, whereas criteria are mental standards for evaluating the results of action.”
    — Bardach & Patashnik, 2020, p. 69

Weighting Criteria

  • Relative Importance: Determine the weight of each criterion based on their importance to the policy goals.
  • Balancing Values: Address conflicts between criteria by prioritizing based on overarching values or philosophical considerations.

Approaches to Weighting

  1. Political Process: Let existing political frameworks determine weights.
  2. Analyst Imposition: Adjust weights based on fairness and underrepresented interests.

Practical Application

When selecting criteria for your analysis: - Group Positive and Negative Criteria Separately: For clarity in evaluation. - Specify Metrics: Ensure each criterion has a clear, measurable indicator. - Focus on Clarity and Relevance: Criteria should be directly related to the policy objectives and practical considerations.

Conclusion

By meticulously assembling evidence, constructing a comprehensive set of alternatives, and thoughtfully selecting evaluation criteria, you lay a strong foundation for your policy analysis. Use this handout as a guide to navigate these critical steps effectively in your project analysis paper.