Title: Discussion 7 — Scaling QLoRA Up to a 70B Model on a Single RTX 4090
Course: AIINFRA 201: Model Adaptation — Fine-Tuning & Quantization > Section: Week 07 — QLoRA and Training on Consumer GPUs with Unsloth

💬 Discussion 7 — Scaling QLoRA Up to a 70B Model on a Single RTX 4090 (20 pts)

Prompt: Imagine you have a 24GB RTX 4090 at home and you want to fine-tune a 70-billion-parameter model with QLoRA. Based on this week's lecture and resources:

- Would QLoRA actually let you attempt this on a single RTX 4090? Why or why not, referencing what you learned about how QLoRA reduces memory footprint versus what it does not change about compute.
- What would you need to adjust in your Unsloth/training configuration to make a 70B run realistic on 24GB (think about batch size, sequence length, gradient accumulation, and LoRA rank/target modules)?
- What's one thing you'd expect to sacrifice or compromise on (speed, dataset size, sequence length, etc.) to make it fit?

Post your initial response (150+ words) by the midweek deadline, then respond to at least two classmates — push on their reasoning, ask about a config choice they made, or offer an alternative trade-off they didn't consider.