🛠️ Lab 14 — Build an ML CI Pipeline: Quality Gates and a Docker Image Push (50 pts)

Goal: Build a complete, free, GitHub-hosted ML CI pipeline that trains a small classifier, tests both the data and the model as quality gates, and packages a FastAPI serving app into a Docker image pushed to GitHub Container Registry (ghcr.io) — all within free public-repo Actions minutes and free GHCR storage. Steps:

  1. Warm up with a minimal workflow. In a new public repo (e.g., aiinfra101-lab14), create .github/workflows/main.yml (note: workflows, plural — misspelling it fails silently) that triggers on: push, checks out your code (actions/checkout@v4), and runs echo "Hello, CI!". Push and confirm a green run in the Actions tab, then rename the folder to .github/workflow/ (singular), push again, and confirm no run appears — the silent-failure trap from the lecture. Rename it back and confirm it runs again. Delete this scratch workflow once confirmed (or keep it and proceed to build the real one in Step 8).
  2. Set up the repo structure. Add app/ (for a small FastAPI serving app), tests/, requirements.txt, and a Dockerfile.
  3. Get a free dataset and train a small model locally. Download the Titanic or Iris dataset from Kaggle. Write a short training script that loads the data, trains a simple classifier (e.g., logistic regression or decision tree with scikit-learn), and serializes it to model.pkl with joblib or pickle.
  4. Write data-validation tests. In tests/test_data.py, write a pytest (or Great Expectations) suite checking the raw dataset: no unexpected nulls in key columns, expected value ranges (e.g., Age between 0 and 100), and expected categorical values (e.g., Sex is only "male"/"female").
  5. Write a model-quality test. In tests/test_model.py, load model.pkl, run it against a held-out test split, and assert accuracy is above a minimum threshold (e.g., assert accuracy >= 0.70). Add a quick schema check that the model loads without error and exposes .predict() with the expected input shape.
  6. Build the FastAPI serving app. In app/main.py, load model.pkl at startup and expose a /predict endpoint (and a /health endpoint).
  7. Write the Dockerfile using lecture best practices. python:slim base, copy and install requirements.txt before copying the rest of the app (cache-friendly layering), add a .dockerignore, and add a HEALTHCHECK that hits /health.
  8. Write the full GitHub Actions workflow (.github/workflows/ci.yml), triggered on: push, that: checks out the repo and sets up Python; uses actions/cache keyed on the hash of requirements.txt to cache pip dependencies; runs pytest (data tests + model-quality gate — the build must fail if any test fails); logs in to GHCR with docker/login-action using the built-in GITHUB_TOKEN; and uses docker/build-push-action to build and push the image to ghcr.io/<your-username>/<your-repo>, tagged with both the git SHA and latest.
  9. Verify end-to-end. Push a commit and watch the Actions tab run through caching, tests, and the image push. Confirm the image appears under your Packages tab. Push a second, code-only change and confirm the pip cache step shows a cache hit.
  10. Break it on purpose, then fix it. Force the model test to fail (e.g., shuffle labels in a copy of the training script), confirm the workflow fails and blocks the image push, screenshot the failed run, then revert and confirm it passes again.

Deliverables: Your repo URL (with .github/workflows/, Dockerfile, tests/, and app/ visible); a link to a successful run showing the cache, tests, and image push all succeeding; your Step 1 observation (2–3 sentences) on the silent folder-name failure; a screenshot of the intentionally-failed run from Step 10; and a short paragraph (5–8 sentences) explaining what your model-quality gate checks and why you chose that threshold.