Training is the process of taking a model architecture, exposing it to a large dataset and iteratively adjusting its parameters until the predictions get good. It is where most of the cost and most of the magic of modern AI lives.
The basic loop is conceptually simple: present an input, let the model predict an output, compare the prediction to the correct answer with a loss function, compute the gradient of the loss with respect to every parameter, and nudge each parameter in the direction that reduces the loss. Repeat for billions of examples, days or months of wall-clock time, and you get a model that has internalised the patterns of the data.
There are several flavours of training that matter in 2026:
- Pretraining — the original, expensive step where a base model learns general language or vision from web-scale data. Costs run from tens of millions to over a billion dollars for frontier LLMs.
- Supervised fine-tuning (SFT) — a smaller second step where the base model is taught to follow instructions on curated example pairs.
- RLHF / RLAIF — a third step where humans (or another AI) rank outputs and the model is trained to prefer the higher-ranked ones, producing the polished assistants users actually interact with.
- Continued pretraining — taking an existing base model and further training it on a domain (legal, biomedical, code) to specialise it.
- Adapter / LoRA training — adding a tiny number of new parameters on top of a frozen base, dramatically cheaper than full fine-tuning.
For a US team building products, the right question is rarely "should we train a model?" — it is "do we need to train, fine-tune, or just prompt?" Prompting handles 80% of cases. Fine-tuning handles another 15% where format consistency or style matters. Full training handles maybe 5%, mostly inside frontier labs and well-funded specialists.
Three things make a training run succeed: enough clean data (quality beats quantity), a stable optimisation recipe (learning rate, batch size, schedules) and a rigorous evaluation harness so you know whether your new model is actually better. Skip any of those and you ship a worse product than the off-the-shelf alternative — at vastly higher cost.
