Up to this point, you've hand-written training scripts and manually wired up datasets, tokenizers, and hyperparameters. This week we introduce Axolotl, an open-source post-training framework that flips that workflow on its head: instead of writing code for each new experiment, you write a single YAML config file that describes the entire run, and Axolotl executes it.
A single Axolotl YAML file specifies your base model, dataset(s), tokenization settings, adaptation method, hyperparameters, and hardware strategy. That one file drives every stage of the pipeline: dataset preprocessing, training, evaluation, quantization, and inference. This matters because it makes experiments reproducible and shareable — you can hand a teammate a 60-line YAML file instead of a 400-line training script, and they can reproduce your exact run. Axolotl supports a wide range of adaptation methods out of the box, so the same framework scales with you as your needs grow:
| Method family | Examples | When you'd reach for it |
|---|---|---|
| Full fine-tuning | Full-parameter updates | Small models, maximum quality, most compute |
| Parameter-efficient | LoRA, QLoRA | Limited GPU memory, fast iteration (what you used in Week 06–07) |
| Preference tuning | DPO, IPO, KTO, ORPO | Aligning a model to human preferences without a separate reward model |
| Reinforcement learning | GRPO | Reward-driven optimization for reasoning or agentic behavior |
Axolotl isn't a competitor to the Hugging Face tools you already know — it's an orchestration layer on top of them. It integrates directly with Hugging Face PEFT adapters (the same LoRA/QLoRA machinery from Week 06) and with distributed-training runtimes like DeepSpeed and DDP for when a job outgrows a single GPU. To lower the barrier to entry, the project ships premade example YAML configs for popular base models such as Llama, Mistral, and Mixtral — you rarely start from a blank file.
1. Always run axolotl preprocess before committing to a long training job. This is one of the most important habits to build this week. A huge share of data-preparation mistakes — a malformed prompt template, a missing end-of-turn token, a column with the wrong name — only surface during this preprocessing/validation step. Catching them in a 30-second dry run is a lot cheaper than discovering them three hours into a paid GPU job. 2. Know which performance flags you have to opt into. A common misconception is that Axolotl's defaults are already tuned for best performance. They aren't — several of the highest-impact flags are opt-in, not automatic:
| Config flag | What it does | Typical gain |
|---|---|---|
sample_packing: true |
Packs multiple short sequences into one training example instead of padding each separately | 3–5x throughput improvement on short-sequence datasets |
flash_attention: true |
Uses a memory-efficient attention kernel (Ampere-generation GPUs or newer) | 20–40% VRAM savings |
If you copy an example config and never touch these fields, you may be leaving a 3–5x speedup on the table without realizing it. Always check the flags section of any config you reuse.
Once you're running more than one experiment, you need a way to compare them — this is where Weights & Biases (W&B) comes in. Axolotl integrates with W&B directly through two config fields: wandb_project and wandb_entity. Set those two lines, and Axolotl automatically logs your training and validation metrics (loss curves, learning rate schedules, GPU utilization, and more) to a W&B dashboard as the run progresses — no manual logging code required. The real payoff is comparison: once you've logged a few runs, W&B lets you overlay their loss curves side by side, so you can see at a glance which hyperparameter combination converges fastest or generalizes best — instead of scrolling through separate terminal logs trying to remember which run used which learning rate.
sample_packing and flash_attention must be explicitly enabled. Defaults favor safety and compatibility over speed.axolotl preprocess before committing to a training job, so formatting errors are caught in seconds, not hours.This week's lab (linked in Resources) walks you through defining a full LoRA training run purely in YAML, then adding W&B tracking and a preprocessing check — giving you hands-on proof of how much friction config-driven training removes.