🛠️ Lab 5 — Build a Cost/Latency-Aware Multi-Provider Router (50 pts)

Goal: Get hands-on, free/low-cost experience authenticating against all three provider ecosystems from this week's lecture, then write a small router that picks a provider based on measured cost and latency — documenting the trade-offs you observe. Steps:

  1. Set up OpenRouter and hit its free tier. Create an OpenRouter account and API key. Set your model string to openrouter/free — this auto-routes to a randomly selected free model that supports tool calling. Make at least 3 calls and log which underlying model actually served each request.
  2. Experiment with provider preferences. Add explicit provider preferences (e.g., prefer a specific base provider, or add :floor / :nitro to a paid model slug you have credits for) and observe how the routing behavior changes versus the default.
  3. Force a fallback and observe it. Configure a provider preference that is likely to hit a rate limit or be temporarily unavailable (or simulate this by setting an artificially low provider allow-list), and confirm OpenRouter automatically falls back to another eligible provider/model. Capture the response metadata showing which provider actually served the fallback.
  4. Make your first Bedrock call. Using a new (or existing) AWS account's promotional credits (\$200, expiring after six months), go to the Bedrock console, enable model access for at least two foundation models in us-west-2, and make an InvokeModel call to each from the AWS CLI or SDK (boto3). Record the response latency and the token-based cost for each model side-by-side.
  5. Make your first Vertex AI / Gemini call. Compare two free paths: (a) the Gemini API free tier directly, and (b) Vertex AI using a GCP trial account's \$300/90-day credit. Authenticate with a service account (or gcloud auth application-default login for local testing) and make one call through each path. Note any differences in setup friction, latency, or response format.
  6. Build the router. Write a small Python script (a single function is fine) that takes a prompt and a priority ("cheapest" or "fastest") and picks among your three configured providers/models based on the cost and latency numbers you actually measured in Steps 1–5. It does not need to call a live API for grading — it can use your logged numbers as a lookup table — but it must actually route to real, working credentials for at least one live call per provider.
  7. Document your trade-offs. In a short write-up (300–500 words), summarize what you observed: which provider was cheapest, which was fastest, what setup friction you hit (auth errors, unenabled models, region issues), and how you'd change your router's logic for a production system.

Deliverables: