An LLM, short for large language model, is a program that has learned from huge amounts of text which piece of a word is likely to come next. When you ask it a question, it predicts the next piece, appends it and repeats that until the answer is finished. Because it follows probabilities and doesn't look things up in a database, its answers almost always sound convincing, but they aren't always right.
I work with Claude every day. My fourteen AI employees run on Claude Team, some also on open models or through the API. To give them sensible tasks, I need to know what happens inside. Here it is without formulas. All AI basics are on my AI Basics page.
What an LLM is
LLM stands for large language model. Behind ChatGPT is such a model from OpenAI, behind Claude one from Anthropic, behind Gemini one from Google. The app you chat in is therefore not the model itself, but the interface around it. I explain how an AI model differs from the app and the vendor in a separate article.
At its core, the model is a huge collection of numbers called parameters. Large models have many billions of them. These numbers are learned from examples during training; how that works is covered in my article on machine learning. The parameters determine how the model computes the continuation of a text. The blueprint for this is called the transformer: every piece of text can attend to the other pieces in the conversation.
These pieces of text are called tokens. A token is a word, part of a word or a punctuation mark.
Step 1: pretraining on huge amounts of text
At the start, the model can't do anything yet. In pretraining it reads enormous amounts of text. Anthropic describes it like this: the model is trained to predict the next word, given the text before it. When it is wrong, its parameters are adjusted a tiny bit. This happens an unimaginable number of times.
Vendors describe only roughly where the texts come from. For Claude, Anthropic names publicly available information from the internet, public and private datasets, user data (according to Anthropic, for example from feedback or with explicit permission), and data generated by other models.
Important: the model doesn't file the texts away like an archive it can look things up in later. It mainly learns patterns, meaning what language, facts and connections typically look like. That's why it can form new sentences but usually can't reliably say where it read something.
Pretraining ends at a cutoff date. Up to this knowledge cutoff, the model knows things best. For the current Claude models, Anthropic gives June 2026.
Step 2: post-training into an assistant
A model with pretraining alone continues texts but doesn't reliably answer questions. Anthropic writes that such models are not inherently good at following instructions. That's why post-training follows:
- Fine-tuning: the model is trained further on selected examples, such as good answers to questions.
- Feedback from people: people rate several answers and decide which is better. The model learns to prefer the better ones. The method is called RLHF.
Only then has the text continuer become an assistant that answers questions, follows rules and stays polite.
Step 3: the answer is built token by token
When you send a question, the model doesn't learn anything new. It only uses what it has learned. This is called inference. The same thing happens over and over:
- The model sees everything in its context window, meaning the space for everything it has in front of it at that moment: your question, the conversation so far, attached files and the vendor's instructions.
- It works out how well each possible next token fits.
- It picks one and appends it.
- With this new token it starts again, until the answer is finished.
An example: if the text reads "Dear Ms Miller, thank you for your", then "inquiry" is a very likely continuation, "banana" is not.
How strictly the model picks is set by the temperature. At a low temperature it almost always takes the most likely continuation; at a high one the answer becomes more varied. That's why you don't always get the same wording for the same question.
"Only predicting the next word" is a simplification, though. In one study, Anthropic showed that when writing poetry, a Claude model plans possible rhyming words in advance, before it writes the line. So the model writes token by token, but it can plan ahead while doing so. What fits into the context window and what happens when it is full is in Context and Context Windows in AI.
Thinking mode: reasoning models
Newer models can think first, before the actual answer. In Claude this is called thinking. According to Anthropic, Claude then restates in its own words what is being asked, tries approaches, checks intermediate results and drops what doesn't hold up. Technically these are tokens too, just ones that come before the answer. It helps with calculations, code and analysis, and it takes longer.
Why an LLM makes mistakes
The typical weaknesses follow from this design:
- It would rather guess than stay silent. Anthropic writes that training itself encourages hallucinations, meaning answers that sound convincing but are made up, because a model is always supposed to give a guess for the next word. Researchers at OpenAI and Georgia Tech come to a similar conclusion: training and evaluations reward guessing over admitting uncertainty.
- It only knows its knowledge cutoff. It doesn't know what happened after that date, unless it searches the web or you give it the information.
- It doesn't look things up. Without tools, the model answers from what it has learned. A rare name or an exact number then easily comes out wrong.
- Long conversations get blurrier. According to Anthropic, accuracy and recall degrade as more tokens sit in the context.
A hallucination is therefore not a one-off defect but a consequence of how the model works. How to spot hallucinations and what helps against them, I explain in my article on AI hallucinations.
An example from the office
An example: you ask an AI to draft a quote for a regular customer. It writes the text in seconds, friendly and well structured. But if your hourly rate isn't in the chat, it may insert a plausible amount, because a number simply belongs at that point in a quote. The model isn't lying. It continues with what typically goes there. If you give it your price list, it has the right number in its context and can use it.
What to watch out for
- Give it your material. What is in the context, the model doesn't have to guess.
- Allow "I don't know". Anthropic explicitly recommends giving Claude permission to admit uncertainty. According to Anthropic, this can drastically reduce false information.
- Ask for evidence. For important statements, have it quote the passage in the document.
- Check numbers, names and dates. That is where the model is most likely to be off.
- Start new tasks in a new chat. Then less old material sits in the context.
What comes after the model: tools and agents
An LLM on its own writes text. Give it tools such as web search, files or a mailbox, and let it take several steps in a row, and it becomes an agent. What comes next when the model gets tools, I explain in What Is an AI Agent? Explained Simply. Why the difference lies in the loop around the model and not in the model itself is in Agentic AI Explained: What It Is and Isn't.
Frequently asked questions
Does an LLM understand what it writes?
Nobody knows exactly. Anthropic itself writes that its developers don't understand most of what goes on inside their models, because the models learn their strategies on their own during training. What is established: an LLM computes with probabilities. And for Claude, Anthropic has shown that the model can plan ahead while doing so.
Does an LLM learn from my chats?
Not during the conversation. Its parameters don't change while you chat; it only keeps track of what is in the context window. Whether a vendor later uses your chats to train new models depends on its rules and your settings.
Why does the AI give me different answers to the same question?
Because with every token the model chooses among several fitting continuations, steered by the temperature. According to Anthropic, the answers are not fully identical even at temperature 0.
How many parameters does an LLM have?
It depends on the model. Open models for your own computer mostly have 20 to 35 billion parameters in 2026, large open models hundreds of billions to trillions. For Claude, Anthropic gives no parameter count in its model overview.
Can I run an LLM on my own computer?
Yes, with open models whose parameters you are allowed to download. Smaller models are a better fit for a normal computer. Which ones exist is in Open Source AI Models 2026 Compared, and what really works with them in Local AI: What Actually Works in 2026.
Where to go from here
If some words in this article were new to you, look them up in the glossary of AI terms. How much a model can keep in view at once is explained in Context and Context Windows in AI. Which models Anthropic currently offers and how they differ is in Claude Models: Opus, Sonnet, Haiku, Fable. How others work with language models day to day is what we discuss in my German-speaking community Claude Practitioners.
