🛠️ Lab 7 — Standing Up a Local Triton Model Repository (50 pts)

Goal: Run NVIDIA Triton Inference Server locally using only free, open tools, serve the official example model repository, verify it's healthy, and then add a new model of your own to the repository — giving you a repeatable template for standing up any Triton deployment at no cost. Steps:

1. Install Docker Desktop (or confirm it's already installed from earlier coursework) and clone the official tutorials repository: git clone https://github.com/triton-inference-server/tutorials.git. This gives you the "Conceptual Guide Part 1" materials referenced below.
2. Pull the free Triton server image from NGC: docker pull nvcr.io/nvidia/tritonserver:<xx.yy>-py3 (use the latest tag available to you, matching your CPU/GPU setup — a CPU-only machine can drop the --gpus=1 flag in the next step and Triton will still run, just without GPU-accelerated backends).
3. Locate (or clone) the official example model repository referenced in NVIDIA's docs, then start the server against it: docker run --gpus=1 --rm -p8000:8000 -p8001:8001 -p8002:8002 -v /full/path/to/docs/examples/model_repository:/models nvcr.io/nvidia/tritonserver:<xx.yy>-py3 tritonserver --model-repository=/models.
4. Confirm the server is ready before doing anything else: curl localhost:8000/v2/health/ready. You should get an HTTP 200 back. If you don't, check the container logs first — most early failures are path or permissions issues with the volume mount, not a broken model.
5. Work through NVIDIA's "Conceptual Guide Part 1" tutorial (in the tutorials repo you cloned) end to end: convert a model, hand-write its config.pbtxt, and query it via both the HTTP and gRPC endpoints.
6. Now do it yourself: choose a small ONNX model (or write a simple Python-backend model) and add it to your local model repository. Build the required directory structure by hand — a top-level folder named after your model, a numeric version sub-directory (e.g. 1/) containing the model file, and a config.pbtxt declaring the model name, backend, max batch size, and input/output tensors.
7. Restart (or reload) the server and confirm your new model loads successfully. Query it with a sample request over HTTP or gRPC and capture the response.
8. If Triton refuses to load your model, treat it as a diagnostic exercise: check first for a missing numeric version directory, then check that your config.pbtxt input/output shapes actually match what the model expects — the two most common beginner mistakes covered in this week's lecture.

Deliverables: Submit (a) a short write-up (1-2 pages) describing your steps, the curl health-check output, and any errors you hit and how you fixed them, and (b) your final config.pbtxt plus a screenshot or terminal capture of a successful query response from your self-added model.