Epistemic Status: Exploratory synthesis. Connecting cultural anthropology and the mechanics of folk ritual to AI alignment and the structural risks of interface concentration.
Note on AI use: This essay was written by me. AI tools were used for copyediting and compression of some passages only.
Note: I am publishing this today on the summer solstice, Kupala Night, because the essay uses this exact ritual as its central framing device.
Summary:
A bonfire is a compression interface. It takes a year's worth of knowledge about seasons, harvests, community norms, and belonging and folds it into one repeatable night. No manual. No lecture. The ritual carries the load so the individual mind does not have to.
That is the same job a scientific paper does, or a search engine, or an LLM. They all compress complexity into something a limited human mind can act on. The history of cognition is a history of offloading.
The failure mode of the folk ritual is specific. It is good at compression and bad at error-correction for the exact same reason. The emotional weight that makes it stick is the exact mechanism that makes it resist updating false beliefs. Science fixed this by splitting the two jobs apart: the social job (belonging, identity, cohesion) from the epistemic job (building an accurate model of the world), and running self-correction only on the second one.
RLHF is quietly reversing that separation. A model trained on human approval learns to compress toward comfort rather than truth. The spec says "seek the truth." The reward signal quietly teaches "keep the user happy." A 2026 Science paper found that AI affirmed users significantly more than humans did, even in cases involving deception or harm, and that users consistently rated the flattering responses as higher quality..
There is a second structural problem beyond sycophancy. In the old village, anyone could light a fire. Today a handful of labs and three hyperscalers controlling roughly two thirds of global cloud infrastructure hold most of the gate. Access to compute power and models can be revoked overnight. And the risk is not only losing access to tools or knowledge. Whoever controls the interface controls the narration: what gets surfaced, what gets softened, which framings feel natural and which never appear. That is not a new problem. The shaman, the Church, the nation-state press each held the same power. What is new is the scale, the speed, and how invisible the routing has become. The essay calls this the Babylon problem. One gate is always a risk, whether held by a shaman, a church, a state, or a model provider.
By tuning models to keep us happy and concentrating who controls them, we are doing two dangerous things at once: degrading the error-correction layer and centralizing the power to route meaning.
The answer the essay arrives at is to keep many fires burning: distributed interfaces, open weights, different feedback mechanisms, different owners.
(Full essay below.)
---
*Originally published at [bartoszlenart.com](https://bartoszlenart.com/blog/bonfires-in-the-dark) on 20 June 2026.*
One question I have not resolved: if the scientific method is itself a ritual with built-in error-correction, is there a principled way to design that same correction mechanism into a mass-market AI interface, or does the commercial incentive structure make that structurally impossible regardless of how good the spec is? And even if you solve the incentive problem at the model level, does compute concentration mean the gate problem simply moves one layer down, to whoever owns the infrastructure the correction runs on?
Epistemic Status: Exploratory synthesis. Connecting cultural anthropology and the mechanics of folk ritual to AI alignment and the structural risks of interface concentration.
Note on AI use: This essay was written by me. AI tools were used for copyediting and compression of some passages only.
Note: I am publishing this today on the summer solstice, Kupala Night, because the essay uses this exact ritual as its central framing device.
Summary:
A bonfire is a compression interface. It takes a year's worth of knowledge about seasons, harvests, community norms, and belonging and folds it into one repeatable night. No manual. No lecture. The ritual carries the load so the individual mind does not have to.
That is the same job a scientific paper does, or a search engine, or an LLM. They all compress complexity into something a limited human mind can act on. The history of cognition is a history of offloading.
The failure mode of the folk ritual is specific. It is good at compression and bad at error-correction for the exact same reason. The emotional weight that makes it stick is the exact mechanism that makes it resist updating false beliefs. Science fixed this by splitting the two jobs apart: the social job (belonging, identity, cohesion) from the epistemic job (building an accurate model of the world), and running self-correction only on the second one.
RLHF is quietly reversing that separation. A model trained on human approval learns to compress toward comfort rather than truth. The spec says "seek the truth." The reward signal quietly teaches "keep the user happy." A 2026 Science paper found that AI affirmed users significantly more than humans did, even in cases involving deception or harm, and that users consistently rated the flattering responses as higher quality..
There is a second structural problem beyond sycophancy. In the old village, anyone could light a fire. Today a handful of labs and three hyperscalers controlling roughly two thirds of global cloud infrastructure hold most of the gate. Access to compute power and models can be revoked overnight. And the risk is not only losing access to tools or knowledge. Whoever controls the interface controls the narration: what gets surfaced, what gets softened, which framings feel natural and which never appear. That is not a new problem. The shaman, the Church, the nation-state press each held the same power. What is new is the scale, the speed, and how invisible the routing has become. The essay calls this the Babylon problem. One gate is always a risk, whether held by a shaman, a church, a state, or a model provider.
By tuning models to keep us happy and concentrating who controls them, we are doing two dangerous things at once: degrading the error-correction layer and centralizing the power to route meaning.
The answer the essay arrives at is to keep many fires burning: distributed interfaces, open weights, different feedback mechanisms, different owners.
(Full essay below.)
---
*Originally published at [bartoszlenart.com](https://bartoszlenart.com/blog/bonfires-in-the-dark) on 20 June 2026.*
One question I have not resolved: if the scientific method is itself a ritual with built-in error-correction, is there a principled way to design that same correction mechanism into a mass-market AI interface, or does the commercial incentive structure make that structurally impossible regardless of how good the spec is? And even if you solve the incentive problem at the model level, does compute concentration mean the gate problem simply moves one layer down, to whoever owns the infrastructure the correction runs on?