🛠️ Lab 9 — Build Your Capstone's Model Layer (50 pts)

Goal: Build and demonstrate a working model-layer component for your own capstone project, using either the RAG track or the adaptation track below — both use free, local tools only, so no API keys or cloud costs are required. Choose your track based on what your capstone actually needs: pick RAG if your project's core gap is external or changing knowledge; pick Adaptation if your project needs a specific behavior, tone, or output format the base model doesn't already produce. Track A — RAG (local, no API keys): Steps:

  1. Install Ollama locally and pull a small local model (e.g., ollama pull llama3.2 or a similarly sized model that runs on your machine).
  2. Install ChromaDB and LangChain in a local Python environment (pip install langchain langchain-community chromadb).
  3. Following the itsFOSS tutorial "Tuning Local LLMs With RAG Using Ollama and Langchain," gather 3–5 documents relevant to your capstone (your own notes, project docs, a domain PDF, etc.).
  4. Chunk the documents on semantic boundaries (paragraphs or sections) with a small overlap — do not just slice by fixed character count.
  5. Embed the chunks and store them in a local ChromaDB collection.
  6. Wire up a LangChain retrieval chain that embeds an incoming query, retrieves the top-k relevant chunks from ChromaDB, and passes them plus the query to your local Ollama model for generation.
  7. Ask your pipeline at least 5 questions that can only be answered correctly using your documents (not general world knowledge), and record the model's answers.
  8. Write a short retrieval-quality note: for any wrong or incomplete answer, inspect what chunks were actually retrieved and explain whether the failure was a retrieval problem or a generation problem.

Track B — Adaptation (free Colab/Kaggle GPU): Steps:

  1. Open a free Unsloth QLoRA notebook on Google Colab (T4 GPU) or Kaggle, following the Unsloth fine-tuning guide.
  2. Select a small base model (e.g., a Gemma model or another \~3B-parameter model) that fits comfortably in the free-tier GPU's memory.
  3. Assemble a small training set (20–100 examples) that demonstrates the specific behavior, tone, or format you want your capstone to produce — write these yourself or adapt them from your project's domain.
  4. Run the QLoRA fine-tune using the notebook's defaults, adjusting only the training-set path and a couple of hyperparameters (epochs, learning rate) as needed.
  5. Export the trained LoRA adapter.
  6. Run the same 5–10 prompts against the base model and against the base model + adapter, and record both sets of outputs side by side.
  7. Write a short before/after comparison explaining, in your own words, what changed and whether the adapter achieved the behavior you targeted.

Deliverables: A short write-up (1–2 pages) describing your track, your capstone's model-layer requirement, and your results; your retrieval-quality note (Track A) or before/after comparison (Track B); and either your pipeline code/notebook link or a screen recording/screenshots showing the working endpoint answering questions over your own data (Track A) or the adapter in action (Track B).