Add explainers/fairness-in-ranking.md.
None of this repo's 61 explainers cover ranking fairness - every existing metric here (Demographic Parity, Equalized Odds, Predictive Parity, etc.) is defined for a binary or categorical classification decision - hired or not, flagged or not, approved or not. A search engine, a job-recommendation feed, or a lending marketplace's results page doesn't make one decision per applicant; it produces an ordered list, and where in that list someone lands determines how much attention (clicks, views, callbacks) they actually get. Two candidates can have numerically similar model scores while one is buried on page 3 - a harm no classification-style parity metric can even see, because it only exists at the group level once you account for position-based exposure decay (position bias: users overwhelmingly click near the top).
Suggested structure (match demographic-parity.md's structure as the closest sibling metric, contrasted directly against it): one-sentence definition -> why it matters (equal average scores between groups says nothing about equal exposure once a ranking function and a click/attention model are applied on top of those scores) -> core concept (exposure: a position-weighted measure of attention a ranked item receives, typically weighted by something like 1/log2(position+1) to mirror how attention drops off the list; fairness of exposure asks whether groups with equal average relevance/qualification receive equal average exposure, not equal average raw score) -> a concrete example - take the ranked output of one of this repo's real classifiers (e.g. sort AI Fair Recruitment candidates by the biased model's predicted probability instead of thresholding at 0.5) and compute each group's average exposure under a standard position-discount weighting, comparing the biased vs. mitigated model's ranking - with real code and real (not invented) results -> detection/implementation code -> limitations (choice of discount function is itself a modeling assumption that changes the answer; re-ranking to equalize exposure can trade off against ranking utility/relevance, a distinct fairness-accuracy-tradeoff-shaped problem) -> related concepts (demographic-parity, fairness-accuracy-tradeoff, subgroup-fairness) and projects.
Related: demographic-parity, fairness-accuracy-tradeoff, subgroup-fairness.
Key citation to verify and use: Singh, Joachims (2018), "Fairness of Exposure in Rankings" (KDD) - confirm the exact venue/year/finding yourself before citing, don't trust this issue's paraphrase.
See CONTRIBUTING.md: add explainers/<slug>.md plus a one-line entry in assets/explainers-data.json; the build script generates the page. Per CLAUDE.md, the paper freeze is lifted - if you quote a Fair Code benchmark result, use the current numbers in results/ (or paper/results-frozen/ for the earlier reference snapshot) and say which one.
Add
explainers/fairness-in-ranking.md.None of this repo's 61 explainers cover ranking fairness - every existing metric here (Demographic Parity, Equalized Odds, Predictive Parity, etc.) is defined for a binary or categorical classification decision - hired or not, flagged or not, approved or not. A search engine, a job-recommendation feed, or a lending marketplace's results page doesn't make one decision per applicant; it produces an ordered list, and where in that list someone lands determines how much attention (clicks, views, callbacks) they actually get. Two candidates can have numerically similar model scores while one is buried on page 3 - a harm no classification-style parity metric can even see, because it only exists at the group level once you account for position-based exposure decay (position bias: users overwhelmingly click near the top).
Suggested structure (match demographic-parity.md's structure as the closest sibling metric, contrasted directly against it): one-sentence definition -> why it matters (equal average scores between groups says nothing about equal exposure once a ranking function and a click/attention model are applied on top of those scores) -> core concept (exposure: a position-weighted measure of attention a ranked item receives, typically weighted by something like 1/log2(position+1) to mirror how attention drops off the list; fairness of exposure asks whether groups with equal average relevance/qualification receive equal average exposure, not equal average raw score) -> a concrete example - take the ranked output of one of this repo's real classifiers (e.g. sort AI Fair Recruitment candidates by the biased model's predicted probability instead of thresholding at 0.5) and compute each group's average exposure under a standard position-discount weighting, comparing the biased vs. mitigated model's ranking - with real code and real (not invented) results -> detection/implementation code -> limitations (choice of discount function is itself a modeling assumption that changes the answer; re-ranking to equalize exposure can trade off against ranking utility/relevance, a distinct fairness-accuracy-tradeoff-shaped problem) -> related concepts (
demographic-parity,fairness-accuracy-tradeoff,subgroup-fairness) and projects.Related: demographic-parity, fairness-accuracy-tradeoff, subgroup-fairness.
Key citation to verify and use: Singh, Joachims (2018), "Fairness of Exposure in Rankings" (KDD) - confirm the exact venue/year/finding yourself before citing, don't trust this issue's paraphrase.
See CONTRIBUTING.md: add
explainers/<slug>.mdplus a one-line entry inassets/explainers-data.json; the build script generates the page. Per CLAUDE.md, the paper freeze is lifted - if you quote a Fair Code benchmark result, use the current numbers inresults/(orpaper/results-frozen/for the earlier reference snapshot) and say which one.