AI Hallucinations: Why AI Makes Things Up

An AI hallucination is an answer from an AI chat such as ChatGPT, Claude or Gemini that sounds convincing but is wrong or simply made up. It might be a law section that does not exist, a delivery time that appears nowhere or a decision that was never taken. It has nothing to do with hallucinations in the medical sense. You are most likely to spot one in concrete details that come without a source. They become rarer when you give the AI your own material, allow it to say "I don't know" and have the result checked separately.

I work with AI tools such as Claude Code every day. My fourteen AI employees run on Claude Team, some also on open models or through the API. They are AI roles with fixed tasks, rules and their own knowledge. I ask for evidence with every result, because "done" is only an assertion. And I have a house rule: whoever produced something does not check it. All the basics about AI are collected on my AI Basics page.

Why AI makes things up

Behind an AI chat sits a language model, a large language model or LLM for short. Anthropic, the company behind Claude, describes the basic training like this: the model learns to predict the next word, given the text before it. So it writes what is most likely to come next after everything it has read. The model does not check whether that is true while it writes. I explain the details in my article on how an LLM works.

In 2025, researchers at OpenAI and Georgia Tech put forward a simple explanation for why the guessing persists. Language models behave like students facing a hard exam: when they are unsure, they often guess instead of leaving a gap. According to the study, most tests reward a guessed answer more than an honest "I don't know", so a model optimised for them learns to guess.

That leads to some typical triggers:

• The material is missing. You ask about something that is neither in the chat nor in the training data, for example your own company's prices.
• The knowledge is out of date. Every model has a cut-off date; Anthropic lists the date up to which each current Claude model's knowledge is most reliable. Anything later the model does not know unless it searches the web.
• The question assumes something. "Name three court rulings on …" sounds as if there are three. A model that prefers guessing is then more likely to deliver three than none.
• The chat is very long. Everything the AI has in front of it right now is called context. The fuller it gets, the more likely an early instruction gets lost. Anthropic says so itself for its tool Claude Code: when the context window is getting full, Claude may start "forgetting" earlier instructions or making more mistakes. What context means exactly is covered in my article on the context window.

Examples from everyday office work

The following cases are made-up examples of what can happen in any small office.

For minutes, for example, one simple rule helps: every statement gets a timestamp from the recording. I show what that looks like in practice in Have Minutes Written: ChatGPT, Teams. An AI phone assistant carries the same risk: if it finds nothing in its knowledge, it can still invent a plausible answer, see AI Receptionist: Providers, Pricing, Law.

How to spot a hallucination

A hallucination looks exactly like a correct answer; the OpenAI study speaks of plausible yet incorrect statements. These four checks help:

1. Watch the concrete details. Numbers, names, dates, case numbers, links and verbatim quotes carry the highest risk.
2. Ask for the source and open it yourself. Have the AI tell you where each important statement comes from. Then check the source yourself, because a citation can be made up too.
3. Ask the same question again from scratch. Ideally in a new chat. Anthropic recommends comparing several runs of the same task; if they differ, that points to a hallucination.
4. Do the maths. For totals, percentages and taxes, a cross-check is often enough, for example net plus tax equals gross.

Whether a text came from an AI at all, on the other hand, cannot be reliably detected. The reasons are covered in Can You Detect AI Text? What Detectors Do.

How to make hallucinations rarer

Anthropic states explicitly that its techniques significantly reduce hallucinations but do not eliminate them entirely. Steps 1 to 4 come from that guide, step 5 is my own house rule:

1. Give the AI your material. Put the price list, the transcript or the contract into the chat and add: "Answer only based on these documents, not from your general knowledge." How systems do this automatically is explained in my article on RAG systems.
2. Allow "I don't know". One sentence is enough: "If the documents say nothing about this, write: I have no basis for that." According to Anthropic, this alone can drastically reduce false information.
3. Ask for quotes before the answer. With long documents, have the AI pull out the relevant passages word for word first and only answer afterwards. That way it relies on the actual text.
4. Have every statement backed up. After the draft, the AI should name the supporting passage for each statement. Whatever it cannot back up, it removes and marks the gap with empty square brackets.
5. Have it checked separately. The role that wrote a text does not review it. Review happens in a second pass with its own assignment, often by a separate role with fresh context, sometimes with a different tool. For correction loops I regularly use Codex, an AI tool from OpenAI. A research finding fits this: language models do not reliably correct their own reasoning errors without outside feedback, and sometimes they even get worse. Outside means real sources, a cross-check, a test. The separate role is my rule for making sure that check does not get skipped.

What to watch out for

• Let the AI draft, not decide. Anything that gets expensive when it is wrong, you check yourself: prices, deadlines, promises to customers, figures on invoices and anything with legal effect.
• A visible gap beats a plausible filler. When my AI employee Ralf could not find anything about one person while building a website, he wrote visible placeholders into the text instead of inventing something. You can check a visible gap; a well-filled one may never catch your eye.
• Sources do not protect you completely. Even when the AI searches your own documents, sources can be outdated or matched to the wrong question. More on that in Build an AI Knowledge Base From Calls.
• Stop instead of arguing. When you discover an invention, don't argue with the AI in the same chat. Stop the result, make the gap visible and ask for evidence. How I go about this is in Using Claude Well: Six Habits.

Frequently asked questions

What is an AI hallucination, explained simply?

An answer that sounds right but is wrong or made up. The language model predicts which word is likely to come next. When it lacks the knowledge, it fills the gap with something plausible instead of saying it does not know.

Why does ChatGPT hallucinate?

For the same reason as other language models. According to a study by researchers at OpenAI and Georgia Tech, training and tests reward guessing more than admitting uncertainty. Missing material, out-of-date knowledge and questions that assume something add to it.

Can you prevent AI hallucinations completely?

No. You can make them much rarer: give the AI your material, allow "I don't know", ask for evidence and have results checked separately. Anthropic itself says these techniques do not eliminate hallucinations entirely. So you always check important information yourself.

Which AI hallucinates the least?

There is no reliable ranking for your day-to-day work. The vendors measure with different tests, and the result depends heavily on the task. My advice: take a typical task from your business whose correct answer you know, and let two tools solve it.

Does it help to ask the AI to check itself?

Only to a limited extent. A study presented at ICLR 2024 shows that language models do not reliably correct their own reasoning errors without outside feedback. What helps is a check against something outside the model: the source itself, a cross-check, your own knowledge. On top of that I have a separate role do the review; that is my house rule.

Where to go from here

Take a task you will give the AI this week anyway and add steps 1 and 2: your material and permission to say "I don't know". Then compare the result. If you want to understand where the guessing comes from, read first how an LLM works. The next step is the RAG system: an AI that looks things up in your documents before it answers. How hallucinations fit into the overall trade-off is covered in my article on the pros and cons of AI. You will find every technical term from this article in my AI glossary. How others check and use AI day to day is what we discuss in my community Claude Practitioners.