I thought I had AI hallucinations figured out.
At The Motley Fool, I was building fact-checking systems for financial content—the kind of stuff where getting numbers wrong can cost people real money (and get your lawyers very mad at you).
The advice everyone gives is simple: don't rely on the LLM's training data, bring in your own verified sources, and hallucinations disappear.
Turns out that's true… but not enough.
Yes, controlled data helps enormously. But even when I fed our systems pristine, real-time financial data, the AI still found creative ways to screw things up. After watching hundreds of these failures, I realized there are three distinct ways LLMs go wrong—and most people only think about the first one.
The Three Flavors of Wrong
1. Just plain wrong
This is what most people think of as hallucination. The LLM says Apple made $500 billion in revenue when it actually made $394 billion. Usually happens when the model gets confused between similar numbers or mixes up time periods. Annoying, but pretty easy to catch.
2. Right facts, wrong meaning
This one's sneakier. The LLM correctly pulls Apple's Q4 revenue of $89.5 billion from the data, but then says it was a "strong quarter" when the revenue was actually lower than expected. The raw number is correct, but the interpretation is off. These errors sound super confident and authoritative, which makes them dangerous.
3. Correct facts, correct analysis, terrible conclusions
The trickiest of all. The model gets the data right (margins improved), interprets it correctly (that's good for the company), but then jumps to conclusions nobody asked for ("this signals they're outperforming competitors and investors should buy immediately"). It's like having a smart intern who can't tell the difference between analysis and advice.
Why This Happens Everywhere
These same patterns can easily rear their heads in legal research, medical information, technical docs—basically anywhere someone deploys LLMs. The issue isn't that they're stupid. It's that they're really good at pattern matching but terrible at understanding boundaries.
They often just don't know when to stop.
What Actually Works
Here's what I've learned from building systems that need to be right:
Set boundaries, but not too many. You want to stop the LLM from giving investment advice while still letting it spot trends. Over-restrict and you lose what makes it useful in the first place.
Build different checks for different problems. Wrong facts need number validation. Wrong interpretations need logic checks. Wrong conclusions need boundary enforcement. One-size-fits-all detection misses too much.
Flag the warning signs. Watch for phrases like "this clearly shows" or "users should"—they often signal the model is about to overstep. Simple pattern matching catches a lot of overreach.
Human spot check strategically. You don't need humans to verify everything, but sample enough to catch when the system starts drifting off course.
Keep improving both sides. Work on your generation prompts until they're better than your checking systems, then flip and improve the checking until it's the stronger piece. Rinse and repeat.
The biggest lesson? Perfect data solves the obvious problems, but the subtle ones require understanding how these systems actually think—and more importantly, when they should stop thinking.
Building reliable AI isn't about making it perfect. It's about making it aware of its own limits.