Title: Lab 14 — Reproducibility, Runbook, and Documentation Site
Course: AIINFRA 303: Capstone Project > Section: Week 14 — Documentation, Runbooks, and Reproducibility

🛠️ Lab 14 — Reproducibility, Runbook, and Documentation Site (50 pts)

Goal: Produce three artifacts that let a total stranger reproduce your capstone model, understand whether they should trust it, and operate it during an incident — using only free, local tools. Steps:

1. Install and launch MLflow locally. In your capstone project's virtual environment, pip install mlflow. Start a local tracking server backed by SQLite with a local artifact store: mlflow server --backend-store-uri sqlite:///mlflow.db --default-artifact-root ./mlruns --host 127.0.0.1 --port 5000. Confirm the UI loads in your browser at http://127.0.0.1:5000.
2. Log a real run from your capstone. Follow the MLflow Tracking Quickstart pattern: wrap your training or evaluation step with mlflow.start_run(), and use mlflow.log_param(), mlflow.log_metric(), and mlflow.log_artifact() (or the relevant flavor's autolog()) to capture at least: your key hyperparameters, at least two evaluation metrics, and one artifact (a saved model file, a confusion matrix image, or a sample output).
3. Register your best run in the Model Registry. From the MLflow UI (or mlflow.register_model()), promote the run that represents your current best capstone model to a named registered model, e.g. capstone-model, and confirm the registered version links back to the source run.
4. Write a Model Card. Using the Hugging Face Model Card template structure, create MODEL_CARD.md documenting: intended use and out-of-scope uses, training data (source, size, known gaps), evaluation metrics (with the numbers from step 2), and limitations/bias or fairness considerations specific to your project.
5. Write one incident runbook. Pick a realistic failure mode for your own system (e.g., "latency SLO breached," "service returning 500s," or "model returning degraded/garbage outputs"). Using the table structure from the lecture, write RUNBOOK-<incident-name>.md with trigger/symptom, ordered diagnosis steps with expected output, remediation, rollback procedure, and escalation. Keep every step short and executable — assume the reader is stressed and has never seen your project before.
6. Write or update your project README. Ensure it states the problem your capstone solves, exact setup steps (dependencies, environment variables), the exact command(s) to run it, and the expected result, so someone can reproduce it from source alone.
7. Build a documentation site with Material for MkDocs. pip install mkdocs-material, initialize a mkdocs.yml and docs/ folder, and add pages for your README content, your Model Card, and your runbook (as separate navigable pages). Run mkdocs serve locally and confirm all three pages render and navigate correctly.
8. (Optional, free) Publish your docs site. If you have a GitHub repo for your capstone, connect it to Read the Docs and publish the MkDocs site for free so it's reachable by URL, not just localhost.

Deliverables: A short write-up or screenshots showing (a) your MLflow run logged with params/metrics/artifacts and the registered model version, (b) your MODEL_CARD.md, (c) your RUNBOOK-<incident-name>.md, and (d) a screenshot or link to your working MkDocs documentation site with all three documents navigable.