🛠️ Lab 4 — Build a Provider-Agnostic LLM Wrapper with LiteLLM (50 pts)

Goal: Use the open-source LiteLLM Python SDK to call OpenAI, Claude, and Gemini behind one interface, and directly observe where a compatibility approach smooths over provider differences — and where it can't. You'll do this entirely with free tools: Gemini's free tier and/or a local Ollama model, so no paid API key is required. Steps:

  1. Set up your environment. Create a new project folder and virtual environment. Install LiteLLM with pip install litellm. If you plan to use local models, also install and start Ollama (ollama pull llama3.1 or similar small model) so you have a zero-cost provider available offline.
  2. Get a free Gemini API key. Sign up for a Google AI Studio account and generate a free-tier Gemini API key. Store it as an environment variable (e.g., export GEMINI_API_KEY=...) — never hard-code keys in your script.
  3. Write a baseline single-call script. Using LiteLLM's completion() function, call a Gemini model with a plain model string like gemini/gemini-2.5-flash. Confirm you get a normal response back and print the message content.
  4. Build your provider-agnostic wrapper. Write a Python function ask(model: str, prompt: str) -> str that calls litellm.completion() and returns just the text content, regardless of provider. Call your wrapper three times with three different model strings — one gemini/*, one ollama/* (pointing at your local Ollama server), and, if you have access to any free/trial Claude or OpenAI credits, one anthropic/* or openai/*. If you don't have paid keys for Claude or OpenAI, substitute a second Gemini or Ollama model instead so you still demonstrate the "swap the string, nothing else changes" pattern.
  5. Log and compare behavior. For each call, print the model string used, the response text, and (if available in the LiteLLM response object) token usage. Write a short code comment or docstring noting anything that surprised you — for example, differences in response length, tone, or refusal behavior across providers.
  6. Do the 3-line provider swap exercise. Separately, write a tiny script using the plain openai Python SDK (not LiteLLM) that calls a Gemini model through Gemini's OpenAI-compatible endpoint. This means changing only the base_url, the api_key, and the model name in code that otherwise looks exactly like a normal OpenAI SDK call. Confirm it works and note in a comment which three lines you changed.
  7. (Extension, optional for extra credit) Run the LiteLLM proxy locally (litellm --config your_config.yaml) as a self-hosted AI gateway. Configure at least one virtual key and confirm you can see per-key spend/usage tracking in the proxy's logs or dashboard. Briefly describe in your submission how this proxy pattern relates to the "AI gateway" concept from the lecture.

Deliverables: Submit your complete Python source file(s) (the wrapper script and the 3-line-swap script) plus a short written reflection (200–300 words, submitted as an online text entry) explaining what stayed the same and what had to change across your three provider calls, and how this hands-on experience connects to the compatibility-layer limitations covered in the lecture.