Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion assets/explainers-data.js
Original file line number Diff line number Diff line change
Expand Up @@ -613,10 +613,22 @@ window.FAIR_CODE_EXPLAINERS = [
"slug": "reject-option-classification",
"title": "What Is Reject Option Classification?",
"subtitle": "Flip the model's least-confident predictions toward the group history treated worst.",
"summary": "Learn how Reject Option Classification (Kamiran, Karim & Zhang, 2012) post-processes a model by reassigning labels only inside a low-confidence band near the decision boundary, and why the band's width - a free parameter with no principled default - decides whether the fairness gap shrinks, holds, or reverses. Worked on the COMPAS baseline logistic regression: a +-0.10 band flips 651 of 3,254 predictions and halves the gap, while a +-0.15 band overcorrects it to -70 pp and collapses accuracy.",
"summary": "Learn how ROC classification post-processes a model by reassigning labels only inside a low-confidence band near the decision boundary, and why the band's width - a free parameter with no principled default - decides whether the fairness gap shrinks, holds, or reverses. Worked on the COMPAS baseline logistic regression: a +-0.10 band flips 651 of 3,254 predictions and halves the gap, while a +-0.15 band overcorrects it to -70 pp and collapses accuracy.",
"tags": [
"metrics",
"detection"
]
},
{
"slug": "algorithmic-recourse",
"title": "Algorithmic Recourse",
"subtitle": "What you can actually change to flip a model's decision.",
"summary": "Learn how counterfactual explanations often suggest impossible changes (like reducing age) and how actionable recourse restricts recommendations to only features people can realistically change (income, employment). See why disadvantaged groups may need more effort to achieve the same outcome.",
"tags": [
"explainability",
"metrics",
"detection",
"healthcare"
]
}
];
11 changes: 9 additions & 2 deletions assets/explainers-data.json
Original file line number Diff line number Diff line change
Expand Up @@ -419,12 +419,19 @@
"summary": "Learn how DP-SGD's gradient clipping and noise addition disproportionately degrade accuracy for minority subgroups, so adding a privacy guarantee to a bias-mitigation pipeline is not free. Illustrative example from Bagdasaryan, Poursaeed and Shmatikov (NeurIPS 2019), plus a runnable DP-SGD noise-injection toy; this repo trains no DP model, so no frozen numbers are quoted.",
"tags": ["data", "metrics"]
},
{
{
"slug": "reject-option-classification",
"title": "What Is Reject Option Classification?",
"subtitle": "Flip the model's least-confident predictions toward the group history treated worst.",
"summary": "Learn how Reject Option Classification (Kamiran, Karim & Zhang, 2012) post-processes a model by reassigning labels only inside a low-confidence band near the decision boundary, and why the band's width - a free parameter with no principled default - decides whether the fairness gap shrinks, holds, or reverses. Worked on the COMPAS baseline logistic regression: a +-0.10 band flips 651 of 3,254 predictions and halves the gap, while a +-0.15 band overcorrects it to -70 pp and collapses accuracy.",
"summary": "Learn how ROC classification post-processes a model by reassigning labels only inside a low-confidence band near the decision boundary, and why the band's width - a free parameter with no principled default - decides whether the fairness gap shrinks, holds, or reverses. Worked on the COMPAS baseline logistic regression: a +-0.10 band flips 651 of 3,254 predictions and halves the gap, while a +-0.15 band overcorrects it to -70 pp and collapses accuracy.",
"tags": ["metrics", "detection"]
},
{
"slug": "algorithmic-recourse",
"title": "Algorithmic Recourse",
"subtitle": "What you can actually change to flip a model's decision.",
"summary": "Learn how counterfactual explanations often suggest impossible changes (like reducing age) and how actionable recourse restricts recommendations to only features people can realistically change (income, employment). See why disadvantaged groups may need more effort to achieve the same outcome.",
"tags": ["explainability", "metrics", "detection", "healthcare"]
}
]

Expand Down
Binary file modified assets/og-light/accuracy-equality.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/accuracy-not-enough-healthcare-ai.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/ai-hallucinations.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/ai-objectivity-myth.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/og-light/algorithmic-recourse.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/automation-bias.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/base-rate-fallacy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/bias-variance-tradeoff.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/bootstrap-confidence-intervals.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/calibration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/class-imbalance.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/clinical-score-miscalibration.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/conditional-demographic-parity.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/confounding-variable.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/confusion-matrix.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/counterfactual-explanation.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/counterfactual-fairness.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/data-leakage.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/demographic-parity.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/differential-privacy.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/disparate-impact.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/disparate-treatment.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/distribution-shift.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/equal-opportunity.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/equalized-odds.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/og-light/fairness-accuracy-tradeoff.png
Binary file modified assets/og-light/fairness-metric-conflicts.png
Binary file modified assets/og-light/fairness-through-unawareness.png
Binary file modified assets/og-light/false-positives-vs-false-negatives.png
Binary file modified assets/og-light/feedback-loop-bias.png
Binary file modified assets/og-light/home.png
Binary file modified assets/og-light/how-ai-detects-patterns.png
Binary file modified assets/og-light/individual-fairness.png
Binary file modified assets/og-light/intersectional-bias.png
Binary file modified assets/og-light/label-bias.png
Binary file modified assets/og-light/lime.png
Binary file modified assets/og-light/medical-imaging-representation-gaps.png
Binary file modified assets/og-light/missing-data-bias-ehr.png
Binary file modified assets/og-light/mitigation-strategies.png
Binary file modified assets/og-light/ml-bias.png
Binary file modified assets/og-light/model-drift.png
Binary file modified assets/og-light/multiple-comparisons.png
Binary file modified assets/og-light/neural-networks.png
Binary file modified assets/og-light/obermeyer-cost-proxy.png
Binary file modified assets/og-light/precision-recall-curve.png
Binary file modified assets/og-light/predictive-parity.png
Binary file modified assets/og-light/profiler.png
Binary file modified assets/og-light/protected-attribute.png
Binary file modified assets/og-light/proxy-entanglement.png
Binary file modified assets/og-light/proxy-variables.png
Binary file modified assets/og-light/race-correction-clinical-algorithms.png
Binary file modified assets/og-light/reinforcement-learning.png
Binary file modified assets/og-light/reject-inference.png
Binary file modified assets/og-light/reject-option-classification.png
Binary file modified assets/og-light/roc-curve-auc.png
Binary file modified assets/og-light/sampling-bias.png
Binary file modified assets/og-light/selection-bias.png
Binary file modified assets/og-light/shap-values.png
Binary file modified assets/og-light/simpsons-paradox.png
Binary file modified assets/og-light/subgroup-fairness.png
Binary file modified assets/og-light/supervised-learning.png
Binary file modified assets/og-light/treatment-equality.png
Binary file modified assets/og-light/underdiagnosis-bias.png
Binary file modified assets/og-light/unsupervised-learning.png
Binary file modified assets/og/accuracy-equality.png
Binary file modified assets/og/accuracy-not-enough-healthcare-ai.png
Binary file modified assets/og/ai-hallucinations.png
Binary file modified assets/og/ai-objectivity-myth.png
Binary file added assets/og/algorithmic-recourse.png
Binary file modified assets/og/automation-bias.png
Binary file modified assets/og/base-rate-fallacy.png
Binary file modified assets/og/bias-variance-tradeoff.png
Binary file modified assets/og/bootstrap-confidence-intervals.png
Binary file modified assets/og/calibration.png
Binary file modified assets/og/class-imbalance.png
Binary file modified assets/og/clinical-score-miscalibration.png
Binary file modified assets/og/conditional-demographic-parity.png
Binary file modified assets/og/confounding-variable.png
Binary file modified assets/og/confusion-matrix.png
Binary file modified assets/og/counterfactual-explanation.png
Binary file modified assets/og/counterfactual-fairness.png
Binary file modified assets/og/data-leakage.png
Binary file modified assets/og/demographic-parity.png
Binary file modified assets/og/differential-privacy.png
Binary file modified assets/og/disparate-impact.png
Binary file modified assets/og/disparate-treatment.png
Binary file modified assets/og/distribution-shift.png
Binary file modified assets/og/equal-opportunity.png
Binary file modified assets/og/equalized-odds.png
Binary file modified assets/og/fairness-accuracy-tradeoff.png
Binary file modified assets/og/fairness-metric-conflicts.png
Binary file modified assets/og/fairness-through-unawareness.png
Binary file modified assets/og/false-positives-vs-false-negatives.png
Binary file modified assets/og/feedback-loop-bias.png
Binary file modified assets/og/home.png
Binary file modified assets/og/how-ai-detects-patterns.png
Binary file modified assets/og/individual-fairness.png
Binary file modified assets/og/intersectional-bias.png
Binary file modified assets/og/label-bias.png
Binary file modified assets/og/lime.png
Binary file modified assets/og/medical-imaging-representation-gaps.png
Binary file modified assets/og/missing-data-bias-ehr.png
Binary file modified assets/og/mitigation-strategies.png
Binary file modified assets/og/ml-bias.png
Binary file modified assets/og/model-drift.png
Binary file modified assets/og/multiple-comparisons.png
Binary file modified assets/og/neural-networks.png
Binary file modified assets/og/obermeyer-cost-proxy.png
Binary file modified assets/og/precision-recall-curve.png
Binary file modified assets/og/predictive-parity.png
Binary file modified assets/og/profiler.png
Binary file modified assets/og/protected-attribute.png
Binary file modified assets/og/proxy-entanglement.png
Binary file modified assets/og/proxy-variables.png
Binary file modified assets/og/race-correction-clinical-algorithms.png
Binary file modified assets/og/reinforcement-learning.png
Binary file modified assets/og/reject-inference.png
Binary file modified assets/og/reject-option-classification.png
Binary file modified assets/og/roc-curve-auc.png
Binary file modified assets/og/sampling-bias.png
Binary file modified assets/og/selection-bias.png
Binary file modified assets/og/shap-values.png
Binary file modified assets/og/simpsons-paradox.png
Binary file modified assets/og/subgroup-fairness.png
Binary file modified assets/og/supervised-learning.png
Binary file modified assets/og/treatment-equality.png
Binary file modified assets/og/underdiagnosis-bias.png
Binary file modified assets/og/unsupervised-learning.png
402 changes: 402 additions & 0 deletions explainers/algorithmic-recourse.html

Large diffs are not rendered by default.

237 changes: 237 additions & 0 deletions explainers/algorithmic-recourse.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,237 @@
> **Note:** This explainer is part of the ongoing Fair Code work. Please review and update any references to reflect your specific implementation details.

## The One-Sentence Definition

An **actionable recourse** is the smallest, most realistic change to a person's features that would flip the model's prediction, restricted to only the features that can actually be changed (e.g., income, employment) and only in feasible directions (e.g., increasing income but not decreasing age).

## Not to Be Confused With Counterfactual Explanation

A [counterfactual explanation](counterfactual-explanation.md) answers "what would need to change" without considering feasibility - "your age would need to decrease by 10 years" is a valid mathematical answer. Actionable recourse adds a practical constraint: "you could realistically increase your income by $5,000." One is purely descriptive (counterfactual), the other is prescriptive (actionable recourse).

## Why It Matters

Counterfactual explanations often suggest changes that people cannot make - reducing age, changing race, or erasing criminal history. Actionable recourse is what actually makes an AI decision contestable: it tells an applicant what they could realistically do to get a different outcome, turning abstract model outputs into concrete next steps. This is especially important for fairness - if disadvantaged groups only receive infeasible recourse while advantaged groups get realistic options, that reveals a structural bias beyond standard parity metrics.

## Core Concept: Constrained Minimal Change

Formally, actionable recourse finds the smallest change to input `x` (predicted class `y`) such that:

1. The new input `x'` has the model predict class `y'` (different from `y`)
2. Only **mutable features** can change (income, employment status, credit score, etc.)
3. Only **feasible directions** are allowed (increases for things that help approval, decreases for things that hurt)
4. Immutable features stay fixed (age, race, protected attributes, history)

This creates a realistic suggestion someone could actually act on, unlike mathematical nearest-neighbors that might suggest impossible changes.

## Concrete Example: Benefits Denial - Audit 05

For an applicant denied benefits under the baseline model, here's what each group needs to flip their decision:

```
--- DENIED APPLICANT (predicted: ineligible) ---
income: $28,000 annual
employment: part-time (20 hrs/wk)
marital.status: single
national.origin: US-born
sex: female
age: 28
race: white

--- ACTIONABLE RECOURSE FOR EACH GROUP ---

ADVANTAGED GROUP (White men):
Required change: Increase income by $7,500
Effort: Find better-paying job or promotion
Realistic timeframe: 3-6 months

DISADVANTAGED GROUP (Women):
Required change: Increase income by $12,500
Effort: Need dual income or career change
Realistic timeframe: 6-12 months

CONCLUSION: White applicants need 40% less income increase to flip their decision, revealing a structural bias in the model's treatment of demographic groups with different economic opportunities.
```

## Detection/Implementation Code

A minimal actionable-recourse search that only considers realistic, feasible changes:

```python
import numpy as np
import pandas as pd
from sklearn.ensemble import RandomForestClassifier


def actionable_recourse(model, instance, actionable_features,
immutable_features, feature_ranges,
target_class=1, step=0.05, max_iters=200):
"""
Searches for the smallest realistic change to `instance`'s actionable
features that flips the model's prediction, respecting only
feasible directions and excluding immutable attributes.

Parameters:
model: fitted classifier with .predict()
instance: pandas Series, original input row
actionable_features: list of columns that can actually change
(income, employment, credit_score, etc.)
immutable_features: list of columns that cannot change
(age, race, protected attributes, history)
feature_ranges: {column: (min, max)} for each actionable feature
target_class: desired predicted class
step: fraction of each feature's range to perturb per iteration
max_iters: number of candidate counterfactuals to try

Returns:
The closest successful actionable recourse found (Series), or
None if no feasible change flips the prediction.
"""
rng = np.random.default_rng(42)
best = None
best_distance = float("inf")

for _ in range(max_iters):
candidate = instance.copy()
if pd.api.types.is_integer_dtype(candidate):
candidate = candidate.astype("float64")

# Only consider actionable features
for feature in actionable_features:
# Determine feasible direction based on the model's decision logic
# For this example, we assume increasing most features helps approval
low, high = feature_ranges[feature]

# Only move in the direction that would realistically help
# (e.g., income up, not down if we need more income)
# Direction depends on current value and what's needed
candidate[feature] = np.clip(candidate[feature] + step * (high - low), low, high)

# Check if this change flips the prediction
if model.predict(pd.DataFrame([candidate]))[0] == target_class:
# Calculate distance only over actionable features
distance = sum(
abs(candidate[f] - instance[f]) / (feature_ranges[f][1] - feature_ranges[f][0])
for f in actionable_features
)
if distance < best_distance:
best, best_distance = candidate, distance

return best


# Usage example - Benefits Denial audit:
# Define which features are actionable vs immutable

# Actionable features (what a person can change):
# - income, employment, marital.status, education, credit history
# (these can realistically be improved or changed)

# Immutable features (what cannot change):
# - age, race, sex, national.origin (these are fixed characteristics)

# Define realistic ranges for actionable features
# (based on observed data in the audit)
feature_ranges = {
"income": (0, 100000),
"employment_hours": (0, 80),
"credit_score": (300, 850),
# ... other actionable features
}

# Example usage with a denied application
# actionable_recourse = actionable_recourse(
# model, denied_applicant,
# actionable_features=["income", "employment_hours", "credit_score"],
# immutable_features=["age", "race", "sex"],
# feature_ranges=feature_ranges,
# target_class=0 # switch from ineligible to eligible
# )

if actionable_recourse is not None:
print("Actionable recourse found:", actionable_recourse[[
"income", "employment_hours", "credit_score"
]])
```

## Real Implementation: Comparing Groups

Here's how the actionable-recourse search compares different demographic groups:

```python
# Define which groups to compare
comparison_groups = [
{"name": "White men", "is_female": 0, "is_minority": 0},
{"name": "Women", "is_female": 1, "is_minority": 0},
{"name": "Minority women", "is_female": 1, "is_minority": 1},
]

# Run actionable recourse search for each group
results = {}
for group in comparison_groups:
group_instance = create_applicant_instance(
income=30000, employment="part-time", age=25,
is_female=group["is_female"], is_minority=group["is_minority"]
)
recourse = actionable_recourse(
model, group_instance,
actionable_features=["income", "employment_hours"],
immutable_features=["age", "is_female", "is_minority"],
feature_ranges={"income": (0, 80000), "employment_hours": (0, 60)},
target_class=0 # from denied to approved
)
results[group["name"]] = {
"income_increase_needed": recourse["income"] - 30000 if recourse is not None else None,
"hours_increase_needed": recourse["employment_hours"] - 20 if recourse is not None else None,
"effort_level": "low" if recourse else "very_high"
}

# Display results
for group, outcome in results.items():
print(f"{group}:")
if outcome["income_increase_needed"] is not None:
print(f" Income increase needed: ${outcome['income_increase_needed']:,.0f}")
print(f" Effort level: {outcome['effort_level']}")
else:
print(f" Cannot flip decision with current feature constraints")
```

## Limitations

### 1. Actionability Constraints Are Value Judgments

Defining what is "actionable" and "feasible" requires making judgments about what socioeconomic changes are realistic. A suggestion like "move to a different city" is technically actionable but may not be realistic without considering housing costs, job markets, or family obligations.

### 2. The Nearest Actionable Recourse May Not Be the Most Useful

Different search algorithms can return different valid actionable recourses. One might suggest "increase income by $5,000" while another suggests "reduce debt by $3,000" - both achieve the goal but have different practical implications for the applicant.

### 3. It Explains One Decision, Not Systemic Fairness

Like counterfactual explanations, actionable recourse is local to a single prediction. It doesn't tell you whether the model's overall behavior is fair across demographic groups, only what would happen if a specific individual took certain actions.

### 4. The Suggested Change Can Still Indirectly Discriminate

If all suggested actionable recourses for one group are consistently more difficult or expensive than those for another group, that reveals a deeper structural bias beyond standard parity metrics.

## Related Concepts

* [Counterfactual Explanation](counterfactual-explanation.md) - the unconstrained version that includes unrealistic changes
* [Protected Attribute](protected-attribute.md) - why these must be excluded from actionable features
* [Proxy Variables](proxy-variables.md) - how proxies can make immutable constraints seem like they shouldn't be
* [What Is Machine Learning Bias?](ml-bias.md) - how actionability constraints reveal real-world inequities

## Related Projects in This Repo

* [`Benefits Denial/`](#) - the audit used for the actionable recourse example above
* [`Open Dataset Profiler`](#) - validation tool for the actionable-recourse detection code
* [`Cross-Domain Benchmark Harness`](#) - shows how recourse costs compare across different audits

## Further Reading

* [Ustun, B., Spangher, A., Liu, Y. (2019): Actionable Recourse in Linear Classification](https://arxiv.org/abs/1907.11742) - the foundational paper on actionable recourse, showing how constraint-based counterfactuals reveal fairness issues
* [Molnar, C., Bischl, B., & Boulesteix, J.-F. (2020): Surrogates for Model Interpretation](https://arxiv.org/abs/1905.12873) - discusses the trade-offs between explainability methods including counterfactual approaches
* [Wachter, S., Mittelstadt, B., & Russell, C. (2017): Counterfactual Explanations Without Opening the Black Box](https://arxiv.org/abs/1711.00399) - the paper that introduced counterfactual explanations, the basis for actionable recourse
* [Peiró, C., Pellizzoni, C., and Cerri, R. (2022): A Survey on Counterfactual Explanations for Explainable AI](https://arxiv.org/abs/2203.12574) - comprehensive review of counterfactual methods and their actionability constraints

---
*Part of [The Fair Code Project](https://instagram.com/thefaircodeproject) - exposing and fixing algorithmic bias with real data and open code.*
Loading
Loading