🛠️ Lab 1 — Build and Benchmark Your First Code-Writing Agent (50 pts)
Goal: Build a working code-writing agent using Hugging Face's free smolagents library, then extend it to run against three different free/local model providers so you can directly observe how the provider landscape affects latency, cost, and behavior — using only free or local tools. Steps:
- Open a free Google Colab notebook (or a local Jupyter environment if you prefer). Install the required library:
pip install smolagents duckduckgo-search. - Create a free Hugging Face account if you don't already have one, and generate an access token (Settings → Access Tokens) for the Serverless Inference API.
- Follow the structure of the Hugging Face Agents Course tutorial ("Build Your First Agent with smolagents," linked in Resources) to instantiate a
CodeAgent using the Qwen2.5-Coder-32B model via the free Serverless Inference API, and equip it with the DuckDuckGoSearchTool. - Give your agent a concrete, multi-step task that requires both reasoning and a web lookup — for example: "Look up the current population of the three largest cities in California and write a Python function that returns them sorted in descending order." Run it and read the full trace of tool calls and reasoning steps it produces.
- In a markdown cell, annotate the trace: identify each point where the agent plans, acts, observes, and adapts. Label at least three distinct loop iterations.
- Now extend the exercise. Point the same agent logic at two additional free providers so you can compare across the provider landscape:
- Groq (free tier, no credit card required, 30 requests/minute, 14,400 requests/day) — sign up for a free API key and swap in a Groq-hosted open model.
- Google Gemini free tier (Flash-Lite model, 15 requests/minute, 1,000 requests/day) — sign up for a free Google AI Studio API key.
- (Optional stretch): install Ollama locally and run a Qwen 3 model entirely on your own machine at zero API cost, for a true local baseline.
- Run the identical task against each provider. For each run, record: wall-clock latency (seconds from prompt to final answer), estimated cost (should be \$0 for all three), and any differences in output quality, tool-call count, or failure modes.
- Compile your results into a short comparison table and a 200-300 word written reflection on what the exercise revealed about the tradeoffs between providers.
Deliverables: Submit your annotated Colab notebook (as a downloaded .ipynb or PDF export) plus your comparison table and written reflection, either pasted into the text entry box or attached as a separate document.