Bias And Fairness

Bias and Fairness is the area of responsible AI concerned with detecting, measuring, and mitigating systematic disparities in how machine learning systems treat individuals and groups. It addresses biases that enter through training data, model design, and deployment context, and it formalises competing notions of fairness such as demographic parity, equalised odds, and individual fairness. The goal is to ensure that automated decisions do not unjustifiably disadvantage protected populations.

Semantic Classification

Content

Compositional Relationships (Components)

SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:AlgorithmicBias))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:BiasDetectionMethods))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:FairnessMetrics))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:BiasMitigationTechniques))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:DisparateImpact))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:IntersectionalFairness))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:FairnessConstraints))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:hasPart ai:FairnessAuditingTools))

Dependency Relationships

SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:requires ai:ModelEvaluation))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:requires ai:DataQuality))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:requires ai:ProtectedAttributes))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:requires ai:TrainingData))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:dependsOn ai:MachineLearning))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:dependsOn ai:DataGovernance))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:dependsOn ai:AutomatedDecisionMaking))

Capability Relationships

SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:enables ai:Explainability))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:enables ai:AlgorithmicAccountability))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:enables ai:AlgorithmicAuditing))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:enables ai:AIFairness))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:enables ai:RegulatoryCompliance))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:supports ai:Accountability))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:supports ai:AIGovernance))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:supports ai:HumanOversight))

Implementation Relationships

SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:uses ai:BiasDetectionMethods))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:uses ai:FairnessMetrics))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:uses ai:CausalInference))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:uses ai:CounterfactualAnalysis))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:implements ai:AlgorithmicImpactAssessment))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:standardizedBy ai:EuAiAct))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:standardizedBy ai:NistAiRmf))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:standardizedBy ai:IeeeP7003))

Reduction Relationships

SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:reducesTo ai:StatisticalFairnessMeasure))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:reducesTo ai:EthicalConstraint))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:contrastsWith ai:AutomationBias))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:contrastsWith ai:HarmfulBias))
SubClassOf(ai:BiasAndFairness
  ObjectSomeValuesFrom(ai:partOf ai:ResponsibleAI))

About

Bias and Fairness as a formal field of inquiry emerged from the collision of two developments in the 2010s: the large-scale deployment of machine learning systems in high-stakes decisions affecting individuals, and the accumulation of empirical evidence that those systems systematically reproduced or amplified pre-existing societal inequalities. The field’s intellectual genealogy spans multiple disciplines. From social science it inherited frameworks for understanding systematic measurement error, construct validity (Campbell and Fiske, 1959), and the sociology of discrimination. From anti-discrimination law — particularly the 1971 US Supreme Court ruling in Griggs v. Duke Power Co., which established the doctrine of disparate impact, i.e., that facially neutral criteria producing discriminatory outcomes constitute unlawful indirect discrimination — it inherited the legal concept that intent is not required for discrimination to be wrongful. From statistics it inherited precision tools for detecting distributional differences and from moral philosophy it inherited normative frameworks for evaluating competing distributional justice claims.

The modern computational era of Bias and Fairness was catalysed by landmark empirical studies. The 2016 ProPublica investigation of the COMPAS recidivism scoring tool (Angwin et al.) demonstrated that Black defendants were labelled high-risk at rates inconsistent with equalised odds relative to white defendants. The 2018 Gender Shades study by Buolamwini and Gebru found error rates of up to 34.7% for darker-skinned women versus less than 1% for lighter-skinned men in commercial facial recognition systems. The 2019 Science paper by Obermeyer et al. revealed that a healthcare risk-stratification algorithm systematically underestimated the care needs of Black patients, affecting approximately 200 million individuals. These studies established that deployed AI systems cause measurable, large-scale discriminatory harm and catalysed rapid expansion of the academic literature and regulatory attention across the EU, UK, and United States.

The field’s central theoretical tension — that competing fairness criteria are mathematically incompatible — was formalised by Chouldechova (2017) and Kleinberg et al. (2016), who independently proved that no classifier can simultaneously satisfy demographic parity, equalised odds, and calibration when base rates differ across groups. Barocas, Hardt, and Narayanan’s definitive textbook (MIT Press, 2023) synthesises these impossibility results with legal and philosophical frameworks, establishing that the choice of fairness criterion is irreducibly normative: it depends on contextual judgements about which errors cause which harms to whom, and these judgements cannot be made by optimising an objective function. This recognition has driven a shift from purely technical intervention toward participatory, governance-embedded approaches in which affected communities, legal experts, and domain specialists collaborate in defining fairness requirements for specific deployment contexts.

Taxonomy of Bias Sources

Bias enters AI systems at multiple stages of the machine learning pipeline:

• Historical bias: Societal inequalities encoded in historical training data; a wage-prediction model trained on historical records learns the existing gender pay gap rather than an equitable alternative.

• Representation bias: Systematic under-sampling of minority subpopulations causing models to perform markedly worse for underrepresented groups, as documented in facial recognition (Buolamwini and Gebru, 2018), medical imaging, and speech recognition (Tatman, 2017).

• Measurement bias: Proxy variables that imperfectly capture the intended construct in ways correlated with protected attributes — postcode as a credit proxy in racially segregated housing markets, criminal history as a recidivism proxy where policing intensity varies by race.

• Aggregation bias: A single model trained across heterogeneous subpopulations with different statistical structures; the HbA1c threshold example (Obermeyer et al., 2019) demonstrates different diagnostic meaning across ethnic groups.

• Feedback-loop bias: Biased predictions influencing future data — over-policing a neighbourhood generates more recorded incidents, confirming the algorithmic prediction and intensifying the cycle.

• Label bias: Systematic error in ground-truth labelling correlated with protected attributes; annotator agreement on toxicity and sentiment labels varies with demographic encoding of stimuli.

• Selection bias: Divergence between the population on which a model is trained or evaluated and the population to which it is deployed; common in clinical AI trained on specialist-clinic cohorts rather than full care pathways.

  Fairness Criteria: Mathematical Formalisation

  Let Y denote the true outcome, D the prediction, and A the protected attribute. The principal statistical fairness criteria are:

• Demographic Parity: P(D = 1 | A = 0) = P(D = 1 | A = 1). Equal selection rates across groups, regardless of underlying qualification rates. Contested in recruitment because it does not account for pre-existing differences in the candidate pool.

• Equalised Odds (Hardt et al., 2016): P(D = 1 | A = a, Y = y) equal for y ∈ {0,1}. Both true positive rates and false positive rates are equalised; appropriate when both under-detection and over-prediction cause harm.

• Equal Opportunity: Special case of Equalised Odds requiring only equal true positive rates; appropriate when denial of benefit (false negative) is the primary harm.

• Calibration / Predictive Parity: P(Y = 1 | D = p, A = a) = p for all groups. Predicted probabilities are equally well-calibrated across groups; satisfies COMPAS’s design criterion but mathematically incompatible with equalised odds when base rates differ (Chouldechova, 2017).

• Individual Fairness (Dwork et al., 2012): Similar individuals — by a task-specific similarity metric — receive similar predictions. Requires a principled definition of “similar” that is itself contestable.

• Counterfactual Fairness (Kusner et al., 2017): A prediction is counterfactually fair if it is unchanged when the individual’s protected attributes are altered in the causal graph sense — i.e., accounting for legitimate mediating pathways. Requires a causal model of the data-generating process.

  Chouldechova’s impossibility result establishes that when group base rates differ, no classifier can simultaneously achieve calibration, equal false positive rates, and equal false negative rates. This means every deployed system implicitly embodies a normative judgement about which fairness criterion takes priority — a judgement that should be made explicitly, transparently, and with input from those most affected.

  Three-Stage Mitigation Pipeline

  The Bias Mitigation Techniques pipeline aligns with the machine learning workflow:

  Pre-Processing (acting on training data):

• Reweighting (Kamiran and Calders, 2012): higher loss weights for underrepresented groups.

• Resampling: SMOTE or ADASYN oversampling of minority positives, or undersampling of majorities.

• Disparate Impact Remover (Feldman et al., 2015): transform feature distributions to reduce protected-attribute correlation while preserving within-group rank ordering.

• Learning Fair Representations (Zemel et al., 2013): learn a latent representation that decouples task-relevant information from the protected attribute.

  In-Processing (modifying the training objective):

• Adversarial Debiasing (Zhang et al., 2018): train an adversarial network to predict the protected attribute from model representations; minimise adversary’s accuracy while maximising task accuracy.

• Fairness Constraints via Lagrangian Relaxation: Microsoft Fairlearn’s ExponentiatedGradient implements constraint-based training with formal approximation guarantees.

• Prejudice Remover (Kamishima et al., 2012): mutual-information penalty between prediction and protected attribute added to the training objective.

  Post-Processing (adjusting model outputs):

• Threshold Optimisation: group-specific decision thresholds to satisfy a target fairness criterion; Hardt et al. (2016) show equalised odds is achievable by post-processing any calibrated classifier.

• Reject Option Classification (Kamiran et al., 2012): apply fairness-correcting overrides for low-confidence predictions near the decision boundary.

• Calibrated Equalised Odds Post-Processing (Pleiss et al., 2017): linear programme finding group-specific thresholds minimising error subject to calibration constraints.

  Use Cases / Major Impact Domains

• Criminal justice: COMPAS recidivism scoring (Angwin et al., 2016); predictive policing systems. ProPublica analysis showed Black defendants misclassified as high-risk at 45% higher rates than white defendants, precipitating the central fairness-criteria debate between predictive parity (which COMPAS satisfies) and equalised odds (which it does not).

• Healthcare: Obermeyer et al. (Science, 2019) demonstrated that a healthcare risk-stratification algorithm affecting approximately 200 million patients systematically underestimated Black patients’ care needs by using healthcare cost as a proxy for health need, reflecting differential access rather than differential illness severity.

• Recruitment and hiring: Amazon’s now-discontinued CV screening system (2018) systematically down-rated female candidates due to historical male-dominated training data. Contemporary applicant tracking systems remain under scrutiny under EU AI Act employment provisions.

• Credit and financial services: Model fairness audits under the US Equal Credit Opportunity Act and Fair Housing Act; the EU AI Act designates credit scoring as a high-risk application requiring mandatory bias testing and transparency documentation.

• Facial recognition: NIST FRVT (2019) benchmarks found false match rates for Black female faces 10–100 times higher than for white male faces across major commercial systems. UK: R (Bridges) v Chief Constable of South Wales Police [2020] found that police use of automated facial recognition violated the Public Sector Equality Duty.

• Social welfare and benefits: Dutch Syri system declared unlawful (2020); UK Universal Credit algorithmic scoring; multiple EU member states’ benefits fraud detection systems found to disproportionately flag minority-ethnic claimants. These cases establish that Bias and Fairness governance applies directly to public sector algorithmic deployment.

• Content moderation and recommendation systems: Filter bubble dynamics, differential content suppression, and differential toxicity rates across demographic groups are documented in recommendation bias literature.

  Intersectionality and Multi-Dimensional Fairness

  Standard fairness metrics assess bias along a single protected dimension independently. This misses compound disadvantage faced by individuals belonging to multiple minority groups — theorised by Kimberlé Crenshaw (1989) as intersectionality. The BBQ Benchmark (Parrish et al., 2021) demonstrated that large language models exhibit stronger biases at intersectional combinations (e.g., African American women, elderly gay men) than along either constituent dimension separately.

  Technically, intersectional fairness requires fairness criteria to hold for every cell in the cross-product of protected attributes. For K attributes each with k values this produces k^K cells, many with insufficient sample sizes for reliable statistical testing — the curse of dimensionality in fairness evaluation. Approaches include constrained optimisation over intersectional groups (Kearns et al., 2018), resampling targeting specific intersections, and subgroup discovery algorithms identifying the worst-performing sub-populations. The intersectional counterfactual fairness framework (Biostatistics, Oxford Academic, 2024) extends causal fairness approaches to handle intersecting protected characteristics.

  Bias and Fairness in Large Language Models

  The emergence of large language models (LLMs) as general-purpose AI systems introduces new dimensions of Bias and Fairness that differ structurally from narrow supervised classification. LLM bias arises from complex interactions between pre-training corpora, instruction tuning datasets, reinforcement learning from human feedback (RLHF), and emergent capabilities of scale. Key phenomena include:

• Stereotype amplification: Bolukbasi et al. (2016) showed word embeddings encode gender stereotypes in geometric structure; subsequent work demonstrated persistence through fine-tuning.

• Toxicity bias: RealToxicityPrompts (Gehman et al., 2020) showed that prompts mentioning certain identity groups systematically induced higher toxicity completion rates.

• Differential task performance: Clinical NLP systems based on LLMs exhibit differential performance across patient demographic groups.

• LLM interventions: Targeted bias reduction can exacerbate unmitigated biases elsewhere in the model (arXiv 2511.18635, 2025), confirming that LLM fairness mitigation faces analogous impossibility challenges to classical statistical fairness.

• EU AI Act GPAI obligations: Providers of general-purpose AI models with systemic risk (over 10^25 FLOPs training compute) face mandatory adversarial testing for bias under Article 51.

  Sociotechnical Dimensions

  Selbst et al. (2019) identified five failure traps in purely technical approaches to algorithmic fairness:

  1. The Framing Trap: defining fairness without engaging with the social context.
  2. The Portability Trap: assuming a fairness solution from one context transfers to another.
  3. The Formalism Trap: assuming a formal criterion captures the moral concept it represents.
  4. The Solutionism Trap: assuming a technical intervention resolves the underlying social problem.
  5. The Ripple Effect Trap: ignoring downstream consequences of fairness interventions.

  These traps motivate participatory design, community impact assessments, and integration of legal expertise into AI development — positioning Bias and Fairness as a sociotechnical discipline requiring interdisciplinary collaboration rather than purely algorithmic optimisation.

  Academic Context

  The Bias and Fairness field’s theoretical foundations include:

• Chouldechova, A. (2017): Proved calibration/equalised odds incompatibility in recidivism prediction.

• Hardt, Price, Srebro (NeurIPS, 2016): Formalised equalised odds and proved it achievable via post-processing.

• Dwork et al. (ITCS, 2012): Individual fairness formalised.

• Kusner et al. (NeurIPS, 2017): Counterfactual fairness via causal graphs.

• Buolamwini and Gebru (PMLR, 2018): Gender Shades intersectional accuracy disparities.

• Obermeyer et al. (Science, 2019): Healthcare risk stratification racial bias at population scale.

• Barocas, Hardt, Narayanan (MIT Press, 2023): Definitive interdisciplinary textbook.

  Key research venues: ACM FAccT (premier interdisciplinary venue), AIES, EAAMO. Key institutions: MIT Media Lab (Algorithmic Justice League), Alan Turing Institute (Fairness, Transparency and Privacy interest group), Stanford HAI, Princeton CITP, Oxford Internet Institute, CMU Societal Computing.

  Current Landscape (2026)

  The global Bias and Fairness landscape in 2026 is shaped by accelerating regulatory enforcement, commercial maturation of AI auditing, and growing jurisprudence establishing that algorithmic discrimination triggers existing legal liability.

  The EU AI Act (Regulation 2024/1689) entered into force on 1 August 2024. The Digital Omnibus provisional agreement of 7 May 2026 deferred certain Annex III high-risk obligations (standalone hiring, credit, biometrics) from August 2026 to December 2027, though GPAI obligations are applicable from August 2025 and most other high-risk obligations remain on the original timeline. Article 10 requires high-risk AI training datasets to be “relevant, sufficiently representative, and free of errors” with explicit bias management obligations throughout the AI lifecycle. AI systems in employment, credit, healthcare, criminal justice, and education face mandatory conformity assessments, fundamental rights impact assessments, and continuous bias monitoring.

  The UK has not replicated the EU AI Act, maintaining a sector-led approach under the AI Regulation Policy Paper (2023) and AI Opportunities Action Plan (2025). The Data (Use and Access) Act 2025 introduced new Articles 22A–22D as operative UK provisions for automated decision-making rights, effective February 2026. The ICO is required to prepare a statutory code of practice on AI and automated decision-making under the Data Protection Act 2018. The EHRC and ICO collaborated on the Fairness Innovation Challenge to develop understanding of algorithmic bias and discrimination in AI systems, informing their regulatory approach. The AI Standards Hub — a DSIT-funded partnership of the Alan Turing Institute, BSI, and NPL — advances trustworthy AI standards as governance tools.

  Commercial AI fairness auditing has become a specialised sector: Holistic AI (London-founded, 2018), ORCAA, Parity AI, and the Algorithmic Justice League provide third-party audits for enterprise clients. Open-source toolkits — IBM AIF360, Microsoft Fairlearn, Google’s What-If Tool, Aequitas — provide the practitioner infrastructure. Hugging Face model cards and dataset cards have become de facto provenance documentation standards across NLP and computer vision communities.

  UK Context

  The Alan Turing Institute’s Fairness, Transparency and Privacy interest group has been central to UK academic contributions, including the development of the Algorithmic Transparency Recording Standard (ATRS) adopted by the UK government for public sector AI in 2021 and updated in 2023. The Centre for Data Ethics and Innovation (CDEI, now integrated into DSIT) conducted sector inquiries into bias in policing (2022), recruitment (2021), and financial services (2023). The DSIT-funded AI Standards Hub (Turing Institute, BSI, NPL) provides governance tool development and international standards engagement.

  UCL’s AI Centre and Imperial College London’s Data Science Institute have investigated representation fairness in foundation models and algorithmic bias in clinical AI respectively. Edinburgh’s Bayes Centre contributes probabilistic fairness frameworks. Cambridge’s Leverhulme Centre for the Future of Intelligence produced early philosophical frameworks on algorithmic discrimination.

  In Northern England, Manchester’s Centre for AI Fundamentals works on epistemic fairness — ensuring AI systems reflect appropriate uncertainty about protected attributes rather than making hard demographic inferences. Sheffield’s Humanising Autonomy and Leeds Institute for Data Analytics have examined bias in social care and welfare algorithms deployed in local government, areas of particular relevance given geographic concentration of economic deprivation and algorithmic welfare systems in Northern English councils.

  The landmark case R (Bridges) v Chief Constable of South Wales Police [2020] EWCA Civ 1058 established that algorithmic bias must be assessed under the Public Sector Equality Duty and ECHR Article 8 before deployment. DSIT’s Algorithmic Transparency Recording Standard requires public sector organisations to document and publish information about AI bias testing, with HMRC, DWP, and NHS England among early disclosers. The UK’s Data (Use and Access) Act 2025 extended automated decision rights effective February 2026.

  Future Directions (2026–2030)

• Causal fairness at scale: Moving from statistical correlation measures to causal graph representations (Pearl-Spirtes-Glymour tradition) that distinguish legitimate from illegitimate pathways of protected-attribute influence. NeurIPS 2024’s causal sensitivity analysis paper demonstrates practical progress toward scalable causal fairness auditing.

• Intersectional fairness methods: Rich subgroup fairness (Kearns et al., 2018) remains computationally intractable at scale; approximate satisfaction via online learning and the 2024 intersectional counterfactual fairness framework represent current leading approaches.

• Foundation model bias auditing: EU AI Act GPAI obligations require adversarial bias testing prior to deployment for high-compute models; new red-teaming methodologies and benchmarks (BBQ, WinoBias, StereoSet) are being systematically integrated into pre-deployment evaluation pipelines.

• Federated and privacy-preserving fairness auditing: Differentially private subgroup statistics compatible with formal privacy guarantees are a current research priority enabling bias auditing without centralised data access.

• Dynamic fairness monitoring: Online fairness algorithms that maintain fairness properties through distribution shift and feedback loops replace static pre-deployment audits as the primary assurance mechanism.

• Regulatory technical standards: CEN-CENELEC JTC 21 is developing harmonised EU standards for EU AI Act bias testing protocols; ISO/IEC JTC 1/SC 42 develops complementary international standards for AI bias measurement methodology, expected to become the definitive technical reference for conformity assessment from 2027.

• Participatory and community-centred approaches: Integration of community advisory boards, co-design workshops, and affected-community impact assessments into the AI development workflow reflects recognition that the choice of fairness criterion is irreducibly political.

  Research and Literature

  1. Angwin, J., Larson, J., Mattu, S., and Kirchner, L. (2016). Machine Bias. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
  2. Barocas, S., Hardt, M., and Narayanan, A. (2023). Fairness and Machine Learning: Limitations and Opportunities. MIT Press. ISBN 9780262048613. https://fairmlbook.org/
  3. Bellamy, R.K.E. et al. (2019). AI Fairness 360: An Extensible Toolkit for Detecting and Mitigating Algorithmic Bias. IBM Journal of Research and Development, 63(4/5).
  4. Bolukbasi, T., Chang, K-W., Zou, J., Saligrama, V., and Kalai, A. (2016). Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. NeurIPS, 29.
  5. Buolamwini, J. and Gebru, T. (2018). Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. PMLR, 81, 1–15.
  6. Chouldechova, A. (2017). Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data, 5(2), 153–163.
  7. Crenshaw, K. (1989). Demarginalizing the Intersection of Race and Sex. University of Chicago Legal Forum, 1(8).
  8. Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. (2012). Fairness Through Awareness. Proceedings of ITCS 2012, 214–226.
  9. European Parliament and Council (2024). Regulation (EU) 2024/1689 (EU AI Act). Official Journal of the European Union. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
  10. Feldman, M., Friedler, S., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. (2015). Certifying and Removing Disparate Impact. Proceedings of ACM SIGKDD, 259–268.
  11. Gehman, S. et al. (2020). RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models. EMNLP Findings.
  12. Hardt, M., Price, E., and Srebro, N. (2016). Equality of Opportunity in Supervised Learning. NeurIPS, 29.
  13. ICO (2022, updated 2024). Guidance on AI and Data Protection: Fairness, Bias and Discrimination. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/artificial-intelligence/guidance-on-ai-and-data-protection/how-do-we-ensure-fairness-in-ai/what-about-fairness-bias-and-discrimination/
  14. IEEE (2021). IEEE P7003-2021: Standard for Algorithmic Bias Considerations. IEEE Standards Association.
  15. ISO/IEC TR 24027:2021. Information Technology — Artificial Intelligence — Bias in AI Systems and AI-Aided Decision Making. ISO.
  16. Kamiran, F. and Calders, T. (2012). Data Preprocessing Techniques for Classification without Discrimination. Knowledge and Information Systems, 33(1), 1–33.
  17. Kearns, M., Neel, S., Roth, A., and Wu, Z. (2018). Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness. ICML.
  18. Kleinberg, J., Mullainathan, S., and Raghavan, M. (2016). Inherent Trade-Offs in the Fair Determination of Risk Scores. arXiv:1609.05807.
  19. Kusner, M., Loftus, J., Russell, C., and Silva, R. (2017). Counterfactual Fairness. NeurIPS, 30.
  20. Mehrabi, N. et al. (2021). A Survey on Bias and Fairness in Machine Learning. ACM Computing Surveys, 54(6).
  21. Microsoft Research (2018 onwards). Fairlearn: A Toolkit for Assessing and Improving Fairness in AI. https://fairlearn.org/
  22. NIST (2022). SP 1270: Towards a Standard for Identifying and Managing Bias in Artificial Intelligence. https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.1270.pdf
  23. NIST (2023). AI Risk Management Framework (AI RMF 1.0). https://airc.nist.gov/
  24. Obermeyer, Z., Powers, B., Vogeli, C., and Mullainathan, S. (2019). Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations. Science, 366(6464), 447–453.
  25. Parrish, A. et al. (2021). BBQ: A Hand-Built Bias Benchmark for Question Answering. arXiv:2110.08193.
  26. R (Bridges) v Chief Constable of South Wales Police [2020] EWCA Civ 1058. UK Court of Appeal.
  27. Selbst, A., Boyd, D., Friedler, S., Venkatasubramanian, S., and Vertesi, J. (2019). Fairness and Abstraction in Sociotechnical Systems. ACM FAccT, 59–68.
  28. Zemel, R., Wu, Y., Swersky, K., Pitassi, T., and Dwork, C. (2013). Learning Fair Representations. ICML, 28(3), 325–333.

Provenance