🛠️ Lab 12 — Audit, Mitigate, and Document a Fairness Gap (50 pts)

Goal: Train a simple classifier on a real, imperfect dataset; measure whether it treats demographic groups fairly using two independent open-source toolkits; apply a mitigation technique; and write up everything you learned as a proper Model Card — turning a hands-on experiment into a responsible-AI governance artifact. Everything in this lab is free: Google Colab (CPU runtime), scikit-learn, Fairlearn, AIF360, and the Hugging Face documentation template. Steps:

1. Set up your environment. Open a new Google Colab notebook (free CPU tier is sufficient). In the first cell, install your libraries: !pip install fairlearn aif360 scikit-learn pandas.
2. Load the dataset. Load the free UCI Adult / Census Income dataset (predict whether income exceeds \$50K/year) — you can pull it directly via fairlearn.datasets.fetch_adult() or from the UCI repository. Note which columns are present, including the sex and race fields you'll treat as protected attributes.
3. Train a baseline classifier. Split into train/test sets and train a simple model (logistic regression or a shallow decision tree) using scikit-learn to predict income. Record overall accuracy on the test set.
4. Audit with Fairlearn. Using Fairlearn's MetricFrame, compute demographic parity difference and equalized odds difference, sliced by sex (and optionally race). Also compute the disparate impact ratio by hand or via Fairlearn's selection-rate metrics. Write one paragraph in your notebook interpreting the gaps you find — do NOT just report the numbers, explain what they mean for real applicants.
5. Mitigate with Fairlearn. Apply one Fairlearn mitigation algorithm — either ExponentiatedGradient (in-processing) or ThresholdOptimizer (post-processing) — constrained to demographic parity or equalized odds. Recompute accuracy and your fairness metrics on the mitigated model. Record the before/after comparison in a table.
6. Cross-check with AIF360. On the same dataset and same protected attribute, apply one AIF360 mitigation algorithm from a different stage than the one you used in Step 5 — e.g., Reweighing (pre-processing), Adversarial Debiasing (in-processing), or Reject Option Classification (post-processing). Compare which stage traded accuracy for fairness most favorably in your case, and say why you think that happened.
7. Try to "fix" fairness through unawareness (and watch it fail). As a quick demonstration, remove the sex column entirely and retrain the baseline model. Recompute the fairness metrics using a correlated proxy analysis (e.g., check relationship status or occupation as potential proxies) and show that the disparity does not fully disappear. Write 2-3 sentences connecting this to the lecture's "fairness through unawareness" misconception.
8. Write the Model Card. Using the free Hugging Face Model Card template (from the Hugging Face Hub documentation), author a complete model card for your final mitigated classifier. Include: intended use and explicit out-of-scope uses; a description of the training data (cite the UCI Adult dataset and its known limitations); disaggregated evaluation results (accuracy per subgroup, plus your fairness metrics before and after mitigation); and an ethical considerations / limitations section that names the fairness-metric tradeoff you made and why.

Deliverables: Submit your Colab notebook (.ipynb, shared link or file upload) containing all code, outputs, and interpretation paragraphs, AND your completed Model Card (markdown or PDF) as a separate file or in the same submission.