Why Does AI Hallucinate? How to Reduce It

AI hallucinates because a language model is built to predict plausible next words, not to look facts up, and because training and testing reward a confident guess over "I don't know." You can reduce it by giving the model sources, allowing uncertainty, asking for citations you then check, and verifying anything that matters.

That is the short answer. The longer one matters, because hallucinations are not random glitches that the next update will quietly fix. They come from how these systems are trained and scored, which means the people using them, and the people building with them, have to plan around them. This guide covers what a hallucination is, why it happens, where it does the most damage, and the habits that actually lower the risk.

What Is an AI Hallucination?

An AI hallucination is output that is false or unsupported but delivered with the same fluent confidence as a correct answer. The model is not lying in the human sense. It has no internal alarm that goes off when it moves from something it learned well to something it is making up.

The U.S. National Institute of Standards and Technology prefers the word "confabulation." Its Generative AI Profile of the AI Risk Management Framework (NIST AI 600-1, July 2024) defines it as "the production of confidently stated but erroneous or false content (known colloquially as 'hallucinations' or 'fabrications') by which users may be misled or deceived." NIST also counts outputs that drift away from the prompt, or contradict something the model said earlier in the same conversation.

Anthropic's documentation describes the same thing from the builder's side: even the most advanced models "can sometimes generate text that is factually incorrect or inconsistent with the given context," according to its guide to reducing hallucinations.

Why Do Large Language Models Hallucinate?

There are three causes worth understanding, and they stack on top of each other.

• They predict words, not facts. A large language model generates text one token at a time, choosing what is statistically likely to come next. NIST puts it plainly: confabulations "are a natural result of the way generative models are designed," because they "generate outputs that approximate the statistical distribution of their training data." A sentence can be highly plausible and completely wrong.
• Some facts barely appear in training data. A model learns patterns well when it has seen them many times. Rare, one-off facts, such as a private person's birthday or the title of an obscure dissertation, give it almost nothing to generalize from, so it fills the gap with something that looks right.
• Guessing gets rewarded. Most benchmarks score an answer as simply right or wrong. Under that scoring, a model that guesses will sometimes be right, while a model that says "I don't know" always scores zero. Over time, the system learns that bluffing pays.

The clearest recent explanation of the last two points comes from OpenAI. In September 2025, OpenAI researchers Adam Tauman Kalai, Ofir Nachum and Edwin Zhang, with Santosh Vempala of Georgia Tech, published Why Language Models Hallucinate. The paper opens with an analogy: "Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty."

Their argument has two parts. First, during pretraining, errors on rare facts are close to statistically unavoidable. The paper gives a worked example: "if 20% of birthday facts appear exactly once in the pretraining data, then one expects base models to hallucinate on at least 20% of birthday facts." That figure is an illustration of their math, not a measured rate for any product. Second, the habit survives later training because evaluations reward it. In the authors' words, "language models are optimized to be good test-takers, and guessing when uncertain improves test performance."

The paper's examples are telling. Asked for one of the authors' own birthdays, a state-of-the-art open-source model gave three different wrong dates on three attempts. Asked for his dissertation title, ChatGPT, DeepSeek and Llama each produced a different confident answer, and the authors note that none of them was correct. Their proposed fix is not another hallucination test but changing how mainstream benchmarks score answers, so that a well-placed "I don't know" stops losing to a lucky guess.

What Do AI Hallucinations Look Like?

Hallucinations tend to cluster in the places where a specific, checkable detail is expected. The most common types:

• Fabricated citations. Case law, academic papers and reports that sound real, with plausible authors, journals and page numbers, but do not exist. NIST notes that models can produce "confabulated logic or citations that purport to justify or explain the system's answer," which makes the error more convincing, not less.
• Invented URLs. Links that follow a real site's pattern but lead to a 404, or to a page that says something else.
• Made-up quotes. Words attributed to a real person, often in their style, that they never said.
• Wrong numbers. Statistics, prices, dates and percentages that are close enough to sound right. Numbers are especially risky because they rarely look suspicious.
• Confident wrong reasoning. A clean, step-by-step explanation that arrives at the wrong answer. NIST points out that models "sometimes provide logical steps for how they arrived at an answer even when the answer itself is incorrect."

If you want to train your eye, our free Spot the Hallucination game shows an AI answer each round and asks you to call it real or fabricated, with an explanation every time.

When Are Hallucinations Most Dangerous?

A wrong movie trivia answer costs nothing. A wrong answer acted on costs a lot. Risk goes up when:

• The output goes straight into a decision or a document without a person checking it. NIST warns that the danger comes when "users believe false content, often due to the confident nature of the response" and act on it.
• The topic is legal, medical or financial. NIST's example is healthcare, where "a confabulated summary of patient information reports could cause doctors to make incorrect diagnoses."
• The question is about something rare, recent or specific, which is exactly where the OpenAI paper says errors concentrate.
• The answer is long and open-ended. NIST notes confabulation is particularly relevant "when it comes to open-ended prompts for long-form responses."
• People over-trust the tool. NIST calls this automation bias, "excessive deference to automated systems," and says it can make the confabulation risk worse.

What Happened When Lawyers Trusted Fake AI Citations?

The best documented example is Mata v. Avianca, a personal injury case in the U.S. District Court for the Southern District of New York. In a March 1, 2023 filing, the plaintiff's lawyers cited court decisions that did not exist. As Judge P. Kevin Castel's Opinion and Order on Sanctions of June 22, 2023 records, the attorneys "submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT, then continued to stand by the fake opinions after judicial orders called their existence into question."

One detail is a lesson in itself. The lawyer asked ChatGPT whether one of the cases was real. According to the opinion, ChatGPT "responded that it had supplied 'real' authorities that could be found through Westlaw, LexisNexis and the Federal Reporter." Asking a model to vouch for its own output is not verification.

The court imposed a $5,000 penalty on the two lawyers and their firm, and ordered them to send letters to their client and to each real judge falsely named as the author of the six fake opinions. The judge was also clear that the tool was not the core problem: "there is nothing inherently improper about using a reliable artificial intelligence tool for assistance. But existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings."

Businesses are on the hook too. In February 2024, British Columbia's Civil Resolution Tribunal ordered Air Canada to pay a customer, Jake Moffatt, after its website chatbot wrongly said bereavement fares could be claimed after travel, CBC News reported. The airline argued the chatbot was responsible for its own actions. Tribunal member Christopher Rivers rejected that: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot."

How Can You Reduce AI Hallucinations When You Use AI?

You cannot switch hallucinations off, but you can make them much less likely and much easier to catch.

• Give it the source material. Paste the document, policy or data you want it to work from, and tell it to answer only from that text. Anthropic's documentation recommends exactly this kind of "external knowledge restriction," and for long documents suggests asking the model to pull out word-for-word quotes first and then work from those quotes.
• Explicitly allow "I don't know." Anthropic lists this first among its basic strategies: "Explicitly give Claude permission to admit uncertainty." A line such as "If the information is not in the material or you are not sure, say so" works with any assistant.
• Ask for citations, then check them. Ask the model to name the source for each claim, then open those sources yourself. A citation you have not opened is a claim, not evidence.
• Use search-grounded or retrieval tools for facts. Assistants that search the web or your files and show their sources give you something to check. Google's Vertex AI documentation describes grounding as the ability "to connect model output to verifiable sources of information," and says it reduces hallucinations and provides links to sources for auditability.
• Lower the stakes of the task. Use AI for drafting, brainstorming, summarizing material you supplied, and rewriting, and keep a human in charge of anything factual that leaves the building.
• Double-check every number. Recalculate math, and confirm statistics, prices and dates against the original source. Numbers are where small, believable errors hide.
• Ask twice. Anthropic's documentation suggests running the same prompt multiple times and comparing outputs, because "inconsistencies across outputs could indicate hallucinations."

How Do Builders Reduce Hallucinations in AI Products?

If you are building an assistant, chatbot or agent, the same principles become system design.

• Retrieval-augmented generation. RAG retrieves relevant passages from a trusted knowledge base and hands them to the model with the question, so it answers from real text instead of memory. The technique was introduced in a 2020 paper, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis and colleagues. It reduces hallucination; it does not eliminate it, because the model can still misread or overreach beyond what was retrieved.
• Grounding with citations. Require the model to tie each claim to a quote from the supplied material. Anthropic's guide suggests having the model check each claim against a supporting quote after drafting, and retract any claim it cannot support.
• Tool use instead of recall. Prices, inventory, appointment slots and account details should come from a live lookup or API call, not from the model's training data. A model that calls a tool for the answer has far less room to invent one.
• Evals. Build a test set of real questions with known correct answers, including questions the system should decline, and run it every time you change the prompt, model or data. Score abstentions fairly, which is the core recommendation of the OpenAI paper.
• Guardrails and human review. Restrict what the system is allowed to answer, route sensitive topics to a person, and log outputs so errors can be found and fixed.

How Do You Fact-Check AI Output?

A simple routine catches most problems:

• Pull out the checkable claims. Names, dates, numbers, quotes, citations and links.
• Open every source. Confirm the page or document exists and actually says what the AI claims. Check the quote word for word.
• Go to the primary source. For a statistic, find the original study or official dataset. For a court case, look it up in an official court record or legal database. For a product fact, use the vendor's own site.
• Do not ask the AI to confirm itself. As Mata v. Avianca showed, a model can confidently confirm its own fabrication.
• Cut what you cannot verify. If a claim cannot be traced to a source, leave it out or label it clearly as unverified.

The Bottom Line

Hallucination is a built-in tendency of today's language models, not a rare bug. They predict plausible text, they have thin knowledge of rare facts, and they have been trained in a system that rewards guessing. The practical response is the same whether you are asking a chatbot a question or shipping an AI product: give the model real sources, make "I don't know" an acceptable answer, check the specific claims, and keep a human responsible for anything that matters.

Sources

Frequently Asked Questions

What is an AI hallucination?

An AI hallucination is false or unsupported output delivered with the same confidence as a correct answer. NIST calls it confabulation, the production of confidently stated but erroneous or false content that can mislead users. It includes fake citations, invented quotes, wrong numbers and broken links.

Why do large language models hallucinate?

Language models predict plausible next words rather than looking facts up. Rare facts give them little to learn from, and a 2025 OpenAI paper argues that training and benchmarks reward guessing over saying "I don't know," so models learn that a confident guess scores better.

Can AI hallucinations be eliminated completely?

No. Techniques like grounding, retrieval and allowing the model to say it is unsure reduce hallucinations, but Anthropic's own documentation says they do not eliminate them. Critical information should always be checked against a primary source before anyone acts on it.

How can I reduce hallucinations when I use an AI chatbot?

Give the model your source material and tell it to answer only from that, explicitly allow "I don't know," ask for citations and open them, prefer search-grounded tools for facts, and double-check every number, date and quote yourself.

What happened in Mata v. Avianca?

In 2023, lawyers in a federal case in New York cited court decisions that ChatGPT had invented. Judge P. Kevin Castel's June 22, 2023 sanctions order imposed a $5,000 penalty on two attorneys and their firm and required letters to the judges falsely named as authors.

How do builders reduce hallucinations in AI products?

Builders use retrieval-augmented generation to answer from trusted documents, require citations tied to quotes, call tools or APIs for live facts like prices, run evals that reward correct abstentions, and add guardrails and human review for sensitive answers.

Build Your AI System

Tell us what is costing you the most time. We will map out exactly what your business needs. Free, no obligation.

Build An Agent