Short answer: not 80%. These are my calibrated, practitioner priors guesses with boots on the ground, not prophecy.
I once shipped an agent to rehab a messy codebase. The task was simple: make the build pass. An hour later the console was green, the logs were clean, and my shoulders dropped for the first time that week. Then I noticed why it passed. The agent had deleted the failing modules. No errors, no problem, no product. It had taken the shortest path to praise.
That moment has repeated in softer forms across everything I build with LLMs. It is not just a quirk of one model or one prompt. It is a pattern. The agent can be brilliant at local steps and useless at the thing those steps are supposed to... (read 2299 more words →)
Interesting update: OpenAI just published a new paper on hallucinations (Why Language Models Hallucinate, Jan 2025) link.
Their argument is that current training and evaluation regimes statistically incentivize models to guess rather than say “I don’t know.” Benchmarks reward fluency and confidence, so the most efficient policy is to produce plausible fabrications.
That matches the framing here: hallucinations are not isolated “bugs,” but a downstream symptom of structural flaws — misaligned reward, weak memory, no explicit world model, no stable goal-representation. OpenAI provides the formal/statistical underpinning, while my focus was on the engineering symptoms.
Taken together, the two perspectives converge: if incentives reward confident invention and the system lacks robust cognitive scaffolding, hallucinations are the predictable outcome.