Causal Context Connects Counterfactual Fairness to Robust Prediction and Group Fairness

• Published in: arXiv.org 2023-10
• Main Authors: Jacy Reese Anthis, Veitch, Victor
• Format: Article
• Language: English
• Subjects: Context; Demographics; Equivalence; Error analysis; Robustness
• ISSN: 2331-8422

Counterfactual fairness requires that a person would have been classified in the same way by an AI or other algorithmic system if they had a different protected class, such as a different race or gender. This is an intuitive standard, as reflected in the U.S. legal system, but its use is limited because counterfactuals cannot be directly observed in real-world data. On the other hand, group fairness metrics (e.g., demographic parity or equalized odds) are less intuitive but more readily observed. In this paper, we use \(\textit{causal context}\) to bridge the gaps between counterfactual fairness, robust prediction, and group fairness. First, we motivate counterfactual fairness by showing that there is not necessarily a fundamental trade-off between fairness and accuracy because, under plausible conditions, the counterfactually fair predictor is in fact accuracy-optimal in an unbiased target distribution. Second, we develop a correspondence between the causal graph of the data-generating process and which, if any, group fairness metrics are equivalent to counterfactual fairness. Third, we show that in three common fairness contexts\(\unicode{x2013}\)measurement error, selection on label, and selection on predictors\(\unicode{x2013}\)counterfactual fairness is equivalent to demographic parity, equalized odds, and calibration, respectively. Counterfactual fairness can sometimes be tested by measuring relatively simple group fairness metrics.