Understanding AI Bias & Fairness
AI bias discriminates — without intent, without awareness, on a large scale. Those who deploy AI without checking for bias risk harm to individuals and legal consequences (EU AI Act Art. 10).
You understand how bias in AI arises, recognize the 3 main types, and know what to do if you suspect bias.
Are We Automating Racism? (Vox)
Shows real examples of AI bias — COMPAS, Amazon Recruiting, Facial Recognition. Makes the topic immediately tangible.
Coded Bias — Documentary Film Trailer
3-minute trailer for the documentary on AI bias in facial recognition. Motivates why the topic must be taken seriously.
How Bias Arises
~15 MinHow Bias Enters AI Systems
The Basic Formula
Bias in → Bias out. AI learns from data. If data is biased, results will be biased. Without malicious intent. Without awareness. On a large scale.
The Three Main Types
Type 1 — Data Bias
Training data reflects the world as it was, not as it should be.
Example Amazon: Ten years of historical hiring data from a male-dominated industry. The model learns: male candidates are preferred. Resumes with the word "women" are downgraded. Result: Systemic bias without programming intent.
Identifying Sign: A group appears less frequently or more negatively in the training data.
Type 2 — Proxy Bias
The model uses indirect variables as a proxy for protected characteristics.
Example Credit Scoring: Postal code correlates with income level → correlates with origin → correlates with ethnicity. The model discriminates based on ethnicity without directly using the variable.
Identifying Sign: A variable "should" be neutral but systematically produces unequal outcomes.
Type 3 — Feedback Loop
The output of the AI becomes the new input — and reinforces existing patterns.
Example Predictive Policing: AI predicts more crime in district A → more police there → more arrests there → new data confirms the prediction → AI becomes more confident in its assessment. The reinforcement loop spins.
Why Bias is So Hard to Detect
| Reason | Explanation |
|---|---|
| No Intent | Bias comes from data, not from code |
| Technical Correctness | The model is "correct" in a statistical sense |
| No Visible Signal | Differences are not outputted, only reflected |
| Complexity | With 100+ variables, relationships are not intuitive |
Quick Check
1. Does bias have to be intentional to cause harm?
2. What is data bias?
3 Types of Bias
- Datenbias: Trainingsdaten spiegeln historische Ungleichheit wider
- Algorithmusbias: Modell-Design verstärkt bestehende Muster
- Feedback-Bias: Bias im Output wird zum neuen Input — Verstärkungs-Schleife
Recognizing Bias and What to Do
~20 MinDetecting Bias — and Acting Correctly
The Most Important Method: Analyze Outcome Distribution
No machine learning knowledge required. Just one question:
"Are there systematic differences in the outcomes between comparable groups?"
Practically:
- Export results of the AI system
- Divide by groups (gender, age, origin, postal code)
- Compare rates: rejection rates, approval rates, scores
If differences are visible that cannot be explained by legitimate professional factors: Suspected Bias.
Warning Signs — These Patterns Should Alert You
| Warning Sign | Possible Cause |
|---|---|
| Systematically poorer scores for a group | Data bias or proxy bias |
| Results never improve for a group | Feedback loop |
| System "knows" things it should not have directly learned | Proxy variable |
| Provider cannot explain patterns | Lack of transparency |
| Rates deviate significantly from population average | Structural bias |
The Correct Procedure in Case of Suspected Bias
1. IMMEDIATELY: Remove system from the critical process
(not: observe, not: wait for patch)
2. DOCUMENT: What was observed? Since when? Which groups affected?
3. ANALYZE: Root cause — training data? Proxy variable? Feedback loop?
4. DECIDE: Retraining, model adjustment, or system replacement?
5. TEST: Bias test with structured test cases before reactivation
6. DOCUMENT: Measures, results, responsible parties
What the Law Requires
| Basis | Requirement |
|---|---|
| EU AI Act Art. 10 | Training data must be representative, relevant, and error-free — high-risk obligation |
| EU AI Act Art. 9 | Risk management system must include bias as a risk category |
| AGG | Non-discrimination applies also to algorithmic decisions |
| ISO 42001 A.5.4 | Fairness as an explicit control measure |
Conclusion: Bias testing is not optional. It is legally mandatory — for high-risk systems from August 2026.
Case Study: The Recruiting Tool
Situation: After 6 months, it becomes apparent that an AI-supported recruiting system systematically rates applicants from certain postal codes lower. Postal code is not an official selection criterion.
What happened here? Proxy bias: Postal code → schools → socioeconomic status → origin. The model discriminates indirectly, without "knowing" it.
Correct Approach:
- Immediately pause the tool — not only after analysis
- Review all decisions made since deployment
- Root cause analysis: Which variable acts as a proxy?
- Retrain without postal code and correlating features
- Before reactivation: structured bias test with test dataset
Incorrect Approach: "We remove postal code from the input data." This solves the symptom. Correlating features (school name, street name, club membership) continue to carry the same proxy.
Back: How Bias Happens | Start Assessment →
Case Study: The Unequal Quotas
You analyze the results of an AI recruiting tool. Applicants from certain postal codes are systematically rated lower — even though postal code is not a selection criterion.
Lösung anzeigen
This is classic proxy bias: postal code correlates with socioeconomic status, access to education, etc.
What to do:
- Immediately remove the tool from the application process
- Review all decisions made since its implementation
- Confront the provider (high-risk obligation: bias testing)
- For own tool: root cause analysis, retraining
- Document — EU AI Act Art. 10 requires data management practices
Your Perspective
Are there AI systems in your environment where you see bias risks?
- Empfehlungsalgorithmen die immer ähnliche Profile bevorzugen
- Scoring-Systeme ohne nachvollziehbare Kriterien
- Tools die historische Entscheidungen als 'optimal' behandeln
What you take away
- Bias aktiv suchen — nicht hoffen dass keiner vorhanden ist
- Ergebnisverteilung regelmäßig prüfen: systematische Unterschiede?
- Bei Bias-Verdacht: sofort stoppen, nicht abwarten
- EU AI Act Art. 10: Datenverwaltung und Bias-Prüfung sind Pflicht