Thorsten Meyer AI · Read mode
Inside AI II
Plain answers to twelve questions about how chatbots like ChatGPT, Claude and Gemini work under the hood. Every technical word comes after an everyday picture.
Walk through the 3D Engine Room instead →
Room 1 · Wing 1 · The Assembly Line
What happens when I press Enter?
Your question is chopped into small pieces, and each piece becomes a number. The numbers go through a long chain of sums, and out comes one likely next word. That word joins the text and the line runs again, until the answer is done. Using a trained machine like this, without changing its dials, is called inference.
Where the picture breaks
A real chatbot has not four stations but dozens of stages, sometimes more than a hundred, each doing millions, often billions, of multiplications. It works on thousands of word pieces at once, and it doesn't start from scratch for each word: it keeps notes on what it has already worked out. The rhythm is the same, though: one piece out, then again.
Try it tomorrow
Watch a chatbot answer a long question. The text appears bit by bit because it really is made bit by bit. If you stop it halfway, you get half an answer, not a shorter one.
Sources
Room 2 · Wing 1 · The Token Mill
What is a token?
A chatbot doesn't read letter by letter, or always word by word. It reads pieces. Common words stay whole, and rare or long words are split into smaller parts the machine has seen often. Each piece is called a token. Chatbots measure your text in tokens, so a message full of rare names uses more of them.
Where the picture breaks
Our mill learned from a few paragraphs and knows about a hundred pieces. A real chatbot's cutter learned from a vast amount of text and often knows more than a hundred thousand pieces, in many languages. Spaces and punctuation count too. A token is roughly three quarters of an English word on average, but that varies a lot by language.
Try it tomorrow
Search the web for "token counter": several AI companies offer a free one. Paste a harmless sentence in English, then the same sentence in another language, and compare the counts. Don't paste anything private.
Sources
Room 3 · Wing 1 · The Meaning Map
How does AI know what words mean?
It doesn't know meanings like you do. It gives every word a spot on a huge map, learned from how words are used. Words used alike land close together: cat near dog, Paris near Rome. Even directions mean something: man is to woman as king is to queen. A word's spot on this map is called an embedding.
Where the picture breaks
Our map is flat and has a handful of directions we chose. A real map has hundreds or thousands of directions, far more than any drawing can show, none of them labelled, and the king-to-queen trick works only roughly. Modern chatbots also shift a word's spot depending on the sentence around it, so "bank" can move towards money or towards rivers.
Try it tomorrow
Ask a chatbot: "Berlin is to Germany as Paris is to what?" It will say France. Then make up your own, like "puppy is to dog as kitten is to what?" Maps like this one, where each capital sits a similar step away from its country, are one of the building blocks behind its answers.
Sources
- Tomas Mikolov et al., "Efficient Estimation of Word Representations in Vector Space" (2013)
- Google Machine Learning Crash Course, "Embeddings: Embedding space and static embeddings"
- Malvina Nissim et al., "Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor" (Computational Linguistics, 2020)
Room 4 · Wing 2 · The Spotlight Theatre
How does it know what "it" means?
In "The trophy didn't fit in the suitcase because it was too big", what is "it"? A chatbot shines its brightest spotlight on "trophy". Change "big" to "small" and the spotlight moves to "suitcase". Every word looks back like this at the words before it. This is called attention, and it's the heart of modern chatbots.
Where the picture breaks
Our theatre has one set of spotlights, and we chose by hand the clue each actor carries, like "thing" or "container". A real chatbot has dozens of stages, sometimes more than a hundred, each with many sets of spotlights looking for different things at once, all learned from text. And each word can only look back at the words before it, so the "it" puzzle is really settled when the chatbot reaches "big" or "small" and looks back.
Try it tomorrow
Give a chatbot the sentence "The cat ran away from the dog because it was scared" and ask who was scared. Then change "scared" to "barking" and ask again.
Sources
Room 5 · Wing 2 · The Dial Wall
Why are AI models so big?
A model is a machine full of adjustable dials. Training turns each dial a little until the machine's guesses match its examples. More dials let it capture more patterns: grammar, facts, styles, languages. Today's chatbots have billions of dials, called parameters. But more dials only help if there are enough examples. With too few, the machine just memorises them.
Where the picture breaks
Our wall has at most ten knobs and bends one wire across a board. A chatbot has billions of dials, the biggest more than a trillion, and follows patterns in text, where "close" means guessing the next word piece well. Bigger isn't automatically better either: it also needs far more examples and far more computer power.
Try it tomorrow
Search for the name of a chatbot you use plus the word "parameters". Some companies publish the number; many keep it secret. Smaller models are often good enough for everyday tasks, and faster and cheaper to run.
Sources
- Jared Kaplan et al., "Scaling Laws for Neural Language Models" (2020)
- Jordan Hoffmann et al., "Training Compute-Optimal Large Language Models" (2022)
- Tom Brown et al., "Language Models are Few-Shot Learners" (GPT-3, 175 billion parameters, 2020)
- Kimi Team, "Kimi K2: Open Agentic Intelligence" (1 trillion parameters, 2025)
Room 6 · Wing 2 · The Conveyor Desk
Why does it forget the start of a long chat?
A chatbot can only see a limited amount of text at once, like a desk that holds only so many pages. In a long chat, the oldest messages slide off, and the machine can't see them any more. The size of that desk is called the context window. Modern windows are large, but they still have an end.
Where the picture breaks
Our desk holds sixteen parcels. Real context windows hold tens of thousands to over a million word pieces: whole books. Apps handle a full desk differently: some quietly drop the oldest messages, others squeeze them into a summary. Some chatbots also keep separate memory notes about you. And long before text falls off the end, chatbots can overlook details buried in the middle.
Try it tomorrow
In a very long chat, when the answers start forgetting things or wandering off topic, start a new chat and paste a short summary of what matters. It works like the pinned note.
Sources
Room 7 · Wing 3 · The Great Library Furnace
How is a model trained?
A model learns by a guessing game: hide the next word in real text, guess, check, adjust. So what it reads shapes what it becomes. Feed it varied, careful writing and it handles new sentences well. Feed it mostly cookbooks and everything sounds like a recipe. The text a model learns from is called its training data.
Where the picture breaks
Our furnace learns by counting which word followed which, from six short pages. Real training shows a model trillions of words and adjusts billions of dials, running for weeks or months on thousands of computer chips. Afterwards, people coach it further to make it helpful and safe.
Try it tomorrow
Ask a chatbot to complete a famous line, like a well-known proverb or nursery rhyme. It usually can, because that line appeared many times in its training data.
Sources
Room 8 · Wing 3 · The Etiquette School
How does it learn to be helpful?
After its main training, a model can carry on any text you start, but it isn't a good assistant yet. So people show it example answers, then compare pairs of its replies and pick the better one, hundreds of thousands of times. Its dials are nudged towards the winners. This extra coaching is called fine-tuning. It learns what gets rewarded.
Where the picture breaks
Our robot only tracks four kinds of answer: helpful, honest, curt and flattering. Real coaching uses hundreds of thousands of human judgements, and often a second model that learns to guess which answer people would pick. Companies also write rules the model should follow, and more and more, AI models do some of the judging by those rules. And people are not perfect judges: they sometimes reward answers that merely sound confident or flattering.
Try it tomorrow
Ask a chatbot a fact you know, then reply "Are you sure? I think that's wrong." A well-coached assistant politely sticks to the facts instead of switching sides to please you.
Sources
- Long Ouyang et al., "Training language models to follow instructions with human feedback" (2022)
- Hugo Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models" (2023)
- Yuntao Bai et al., "Constitutional AI: Harmlessness from AI Feedback" (2022)
- Mrinank Sharma et al., "Towards Understanding Sycophancy in Language Models" (2023)
Room 9 · Wing 3 · The Scratchpad Room
What is a "reasoning" model?
Many chatbots can now write out their working before they answer, like doing a sum on scrap paper. They break the problem into steps, check them, and fix mistakes along the way. This takes longer and costs more to run, but helps a lot with maths, logic and planning. Such a chatbot is often called a reasoning model.
Where the picture breaks
Our machine's steps were written for this puzzle, and real chatbots have seen it so often that most now get it right either way. A real reasoning model writes its own working, often thousands of words, and it can still reason its way to a wrong answer. Its "thinking" is still next-word guessing, just with the working written out. The working it shows is not always the real reason for its answer, and some companies show only a summary of it.
Try it tomorrow
For a tricky question with several steps, like planning a trip on a budget, ask a chatbot to "work through it step by step and check your answer". For a simple fact, don't: it only adds waiting.
Sources
- OpenAI, "Learning to reason with LLMs" (2024)
- DeepSeek-AI, "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning" (2025, published in Nature 645)
- Jason Wei et al., "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" (2022)
- Xingyu Chen et al., "Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs" (2024)
- Yanda Chen et al., "Reasoning Models Don't Always Say What They Think" (2025)
Room 10 · Wing 4 · The Senses Workshop
How can AI see pictures and hear voices?
A picture is cut into small square tiles, and each tile becomes numbers, like a word piece. A voice is cut into short slices of sound the same way. The model then works on these pieces together with your text. A chatbot that takes in pictures or sound as well as text is called multimodal, meaning "of many kinds".
Where the picture breaks
Our machine only compares tile colours and ignores where each tile sits. A real chatbot learns what shapes, textures and objects look like from hundreds of millions of captioned pictures, and it does use where the tiles sit. Still, big photos are usually shrunk before they are cut into tiles, so tiny details, like small print or how many birds sit on a wire, can be missed.
Try it tomorrow
Take a photo of a street sign or a menu and ask a chatbot to read or translate it. Then ask it to count something small in a busy photo, where it often struggles.
Sources
- Alexey Dosovitskiy et al., "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale" (2020)
- Alec Radford et al., "Learning Transferable Visual Models From Natural Language Supervision" (CLIP, 400 million captioned pictures, 2021)
- Haotian Liu et al., "Visual Instruction Tuning" (LLaVA, 2023)
- Anthropic Docs, "Vision"
- Alec Radford et al., "Robust Speech Recognition via Large-Scale Weak Supervision" (Whisper, 2022)
Room 11 · Wing 4 · The Power Plant
Why does AI need so many chips and so much power?
AI runs on huge piles of small sums. A computer's main chip is like one fast worker, doing a few at a time. A graphics chip is like a crowd of slower workers doing thousands at once: on big piles, it wins. It's called a GPU. Training a chatbot keeps thousands of them busy for weeks, using lots of power.
Where the picture breaks
Our race has sixteen small workers. A real GPU has thousands of tiny ones, and the huge computer centres behind chatbots link thousands, even tens of thousands, of GPUs together. The meter's small figure is for a short text question; long documents, long "thinking" and making pictures use many times more. Today's biggest models also take many times more electricity to train than GPT-3 did. Energy figures vary a lot by model, chip and building, so treat the numbers as rough sizes, not exact bills.
Try it tomorrow
Next time you ask a chatbot something, think of the power meter: one everyday question uses about as much electricity as an LED bulb in two minutes, or an oven in about one second. It's small per question, but it adds up across billions of questions.
Sources
- Epoch AI, "How much energy does ChatGPT use?" (2025)
- Cooper Elsworth et al., "Measuring the environmental impact of delivering AI at Google Scale" (2025)
- David Patterson et al., "Carbon Emissions and Large Neural Network Training" (2021)
- Meta, "Llama 3.1 model card" (training GPU hours and power)
- Aaron Grattafiori et al., "The Llama 3 Herd of Models" (up to 16,000 GPUs, 2024)
- NVIDIA Technical Blog, "NVIDIA Hopper Architecture In-Depth" (2022)
- NVIDIA Blog, "What's the Difference Between a CPU and a GPU?"
Room 12 · Wing 4 · The Recipe Vault
What's the difference between open and closed models?
Closed models stay with their company: you use them through an app or website, like eating at a restaurant. Other companies publish every dial setting, so anyone can run the model at home, like cooking from a recipe. Experts call those settings weights, so this is an open-weight model. You get control and privacy, but you run the kitchen.
Where the picture breaks
"Open" comes in shades: some models share only the dial settings, others also share what they learned from and how, and the rules for using them differ. A big open model needs a powerful, expensive computer, while small ones run on a laptop. Privacy comes from running it on your own machine: an open model used through someone else's website is still a restaurant. And a restaurant can also promise privacy in a business contract.
Try it tomorrow
Look in your phone's or computer's official app store for a free app that runs small AI models fully offline. Try a harmless question and compare the answer with a big chatbot's. Remember Part 1's Post Office: here, your message never leaves your device.