Why LLMs hallucinate

Hallucinations aren’t random — they’re a predictable failure mode of how language models generate text.

---

A large language model generates text by predicting the next token that looks most likely given the prompt and its training. That means it can sound confident even when it’s wrong.

The core reason

An LLM is not “retrieving facts” by default. It’s modelling plausibility. If your prompt nudges it toward an answer, it will produce one — even if the right response should be “I don’t know.”

Common triggers

1. Missing context: the answer depends on facts not in the prompt.
2. Ambiguous questions: multiple interpretations with no grounding.
3. Overly broad requests: “give me everything about…”
4. Incentives to be helpful: the model is optimised to respond.

What to do about it

Hallucination risk drops when you:

• constrain the task,
• provide grounded context (documents, policies, product data), and
• make abstention acceptable (e.g., “say you don’t know”).

In other words: treat hallucinations like a product quality attribute you can design for.

---