By Thorsten Meyer
Most people picture a language model as a single thing: the AI, sitting somewhere, thinking about your question. That picture is the source of nearly every confused question people have about these systems. Does it learn from talking to me? Where do its values come from? Why did it refuse that? Can it be fixed overnight?
The answers stop being mysterious the moment you stop seeing one thing and start seeing a pipeline that runs across three very different timescales. Capability is built once, over months. Behaviour is shaped afterward, over weeks. And every individual answer is assembled in seconds, from parts that learn nothing new in the process.
Get that shape in your head and the rest falls into place. Here is the whole thing, stage by stage — and, more usefully, what each stage explains about the model you actually talk to.
One map, three timescales. Capability is built once over months; behaviour is set over weeks; and every answer is assembled in seconds from parts that learned nothing new. Three points along the way are where alignment actually lives.
Three timescales, one system
Before the stages, hold onto the frame, because it does most of the work.
Pre-training takes months and happens once. It is enormously expensive and it builds raw capability — the model's knowledge of language, facts, code, and patterns. This is the part everyone means when they talk about "training a model," and it is the part that is essentially fixed by the time you meet the model.
Post-training takes weeks and has enormous leverage. It doesn't add much raw capability; it takes the raw thing and turns it into an assistant with manners, judgment, and limits. Pound for pound, this is where the model's behaviour is decided.
Inference takes seconds and happens on every message. This is the model answering you. Crucially, nothing is learned here. The model that answers your thousandth message is byte-for-byte identical to the one that answered your first.
Almost every misconception about these systems is a timescale confusion — attributing to one stage something that actually happens in another. So let's walk them in order.
The Sales Manager's AI Operating Kit: 40 Copy-Paste AI Prompts and 8 Conversation Scripts for Pipeline Reviews, 1:1s, Forecasts and Hiring
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Pre-training: building raw capability
It starts with data: trillions of tokens of text, deduplicated and filtered. The quality and composition of that corpus quietly determines much of what the model will be good and bad at, long before any "training" in the everyday sense begins.
Then comes the actual pre-training objective, and it is almost comically simple to state: predict the next token, at enormous scale. Show the model a sequence of text with the next piece hidden, have it guess, correct it, and repeat — trillions upon trillions of times, across a network with hundreds of billions of parameters. That's it. There is no notion of "helpfulness" or "truth" or "refusal" in this objective. There is only: what token comes next.
What you get out the other end is a base model — and this is the stage people almost never see, because base models are rarely shipped to the public. A base model is fluent but has no manners. It will happily continue any text you give it, but it doesn't reliably follow instructions, doesn't answer questions as questions, and has no concept of declining anything. Ask it "What is the capital of France?" and it might helpfully answer, or it might continue with three more trivia questions in the same style, because that's a plausible next-token continuation. It has the raw capability of an assistant and none of the behaviour of one.
That gap — between fluent capability and usable assistant — is the entire job of the next phase.
Fine-tuning Large Language Models Handbook: Customize GPT and Open-Source LLMs for Specialized AI Applications, Domain Adaptation, and Enterprise Solutions
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Post-training: where the assistant is actually made
This is the highest-leverage part of the whole pipeline, and the least understood. Four things happen here, and the order matters.
First, and upstream of everything else, is the model spec — or constitution: a written document of principles that everything below is judged against. This is the part people find surprising. A model's values are not some emergent mystery; at the labs doing this carefully, they are written down — how to be helpful, when to decline, how to weigh competing considerations — and then the rest of post-training is, in effect, the work of pressing those written principles into the weights. When you argue with a model's judgment, you are, at one remove, arguing with a document.
Second is instruction tuning, often called supervised fine-tuning (SFT): the model is shown curated example answers — good responses to a wide range of prompts — and learns, from imitation, to respond rather than merely continue. This is what converts the fluent-but-mannerless base model into something that treats a question as a question.
Third is the reward model: a separate model trained to learn which answer people — or the spec — prefer. Instead of hand-writing the ideal response to every possible prompt (impossible), you train a judge that can score any response, so the system has a scalable notion of "better" and "worse."
Fourth is reinforcement learning, and it's the engine that pulls it all together. The loop is: the model produces an answer, the reward model scores it, and the weights get nudged toward higher-scoring answers — on repeat, millions of times. Answer, score, nudge the weights, again. Over enough cycles, the model's behaviour converges toward what the spec and the preference data reward. This is where "be helpful, be honest, decline these specific things" stops being words in a document and becomes a property of the network.
The output of all this is the model people actually use. And then something happens that is easy to miss and important to understand.
Reinforcement Learning: Industrial Applications of Intelligent Agents
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The freeze: a deployed model does not learn
When the model is deployed, its weights are fixed. Frozen. From that moment on, everything that happens runs per request and changes nothing permanent.
I want to dwell on this, because it corrects the single most common misconception about these systems: the model does not learn from talking to you. It does not remember your last conversation because it "learned" from it. Whatever continuity you experience — a memory feature, a reference to an earlier chat — is achieved by feeding text back into the model at inference time, not by changing the model. The weights that answered your first message will answer your millionth, unchanged. Nothing you say updates them.
That single fact resolves a surprising amount. It's why a model can't be quietly "corrected" by users mid-conversation in any lasting way. It's why fixing a model's behaviour is slow — it requires another round of post-training, not a live patch. And it's why your data isn't silently becoming part of the model as you type. The learning door is closed at deployment. There is exactly one way back through it, and we'll get to it at the end.
Revell 15873 Messerschmitt BF 109G-10 1:48 Scale 40-Piece Skill Level 4 Model Airplane Building Kit
- Model Kit Number: Revell 15873
- Skill Level: Level 4
- Number of Parts: 40 pieces
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Inference: how an answer is actually assembled
So the weights are frozen. What happens when you send a message? A stack of parts gets assembled into the model's context, in a specific order of trust, and then the model does the only thing it ever does — predict the next token — now steered by all that training.
At the top sits the system prompt: hidden rules for this specific deployment. Not part of the model's weights, not written by you — instructions the operator wraps around every conversation to shape how the model behaves in this product. It's the fastest-changing layer in the whole pipeline; a deployment can rewrite its system prompt in an afternoon.
Then your user prompt — which is, by design, untrusted input that cannot outrank the system prompt. This ordering is deliberate and it is a security boundary: the model is trained to treat the operator's instructions as higher-priority than the user's. It's why you generally can't talk a well-built deployment out of its core rules just by insisting.
Both of those, plus conversation history and any retrieved documents, are gathered into the context window — the working memory for this single response. This is also, quietly, the soft underbelly. The system-prompt-beats-user-prompt hierarchy is clean, but retrieved documents — a web page, a file, an email the model was asked to read — enter the same context, and hostile text hidden in them can try to hijack the model's behaviour. That's the whole mechanism behind prompt-injection attacks: smuggling instructions into the context through the one door that isn't the user typing them.
Then comes generation: next-token prediction again — the very same operation as pre-training — but now steered by everything post-training pressed into the weights. The raw capability and the trained behaviour meet here, one token at a time.
The draft that generation produces then passes an output classifier: a separate check that either lets the response through or replaces it with a refusal. This is the last wall — an independent layer that can catch something the model itself produced but shouldn't send. And then the response is streamed back to you, token by token, which is why you watch it appear rather than receiving it all at once.
Where alignment actually lives
Step back from the stages and notice something. Across this entire pipeline, the points where a model's values and limits are actually set are surprisingly few and surprisingly identifiable. There are three of them, and they're worth naming, because "is this model aligned?" is too vague a question — the useful question is at which touchpoint, and who controls it.
The model spec / constitution is the deepest one — upstream of all of post-training, baked into the weights, the same everywhere the model runs. The system prompt is the shallowest and most flexible — set per deployment, changeable in an afternoon, different in every product built on the same model. And the output classifier is the last line — a check applied after the fact, independent of what the model intended.
That's a genuinely useful mental model. When a model behaves in a way you like or don't, it's worth asking which of these three is responsible, because they have completely different properties. The constitution is slow to change and universal. The system prompt is fast and local. The classifier is a blunt final filter. A refusal you run into might be coming from any of the three, and the fix — if there is one — is different for each.
It also, I'll note, sharpens what "sovereign" or "local-first" AI actually buys you. If you run open weights on your own infrastructure, you don't get to rewrite the constitution baked into the weights — but you do hold the other two touchpoints yourself. You write the system prompt. You choose whether there's an output classifier and what it does. Two of the three places where a model's behaviour is governed become yours rather than a vendor's. That is not a small thing, and it is exactly the part you give up when you rent a model through someone else's API.
The loop back into the weights
I said the learning door closes at deployment, and that there was exactly one way back through it. Here it is.
Every time you rate a response — a thumbs up, a thumbs down — and every time the output classifier trips and swaps a draft for a refusal, that becomes a data point. Not a change to the live model; the live model is still frozen. But those signals accumulate into preference data for the next round of post-training. They feed the reward model of the next version. Inference itself changes nothing — and yet, in aggregate, it quietly shapes what the next model will become.
So the pipeline isn't a line, it's a loop. Capability is built once. Behaviour is shaped afterward. Answers are assembled per request, learning nothing. And the residue of all those answers — the ratings, the refusals, the preferences — flows back to become the raw material for the next turn of the wheel. That loop is the only path from your keyboard back into the weights, and it runs on the scale of model releases, not conversations.
Once you can see the whole shape — three timescales, three alignment touchpoints, one loop closing the circle — the model stops being a black box you talk to and becomes a system you can reason about. And reasoning about it, rather than mystifying it, is most of what it takes to use it well.
Insights from a builder, founder, and post-labor economist running a local-first inference operation. This is a conceptual map of the standard training-to-inference pipeline used across the frontier labs; exact stage names, the presence and design of safety layers, and the specifics of post-training vary by lab and change over time. It is meant as a mental model, not a spec for any one system. Point-in-time as of 11 August 2026.
