This is an automated rejection. No LLM generated, assisted/co-written, or edited work.
Read full explanation
Epistemic status: Institutional-design hypothesis drawn from historical state-capacity research, applied to current rationalist governance proposals. High confidence that the legitimacy bottleneck described here is real; moderate-to-high confidence that it is binding for the class of governance fixes discussed. Moderate confidence in the proposed substrate-first remedies. The AI governance prediction in §IX is explicitly falsifiable.
TLDR: Rationalist governance discourse keeps trying to fix the state by building better signal generators — prediction markets, AI audits, superforecasters, open-source diagnostics. The bottleneck is often not signal quality; it's the legitimacy gate: the question of who must own a model, who can contest it, who must answer when it is ignored. A signal is not yet governance until some legitimate actor must respond to it. Prediction markets are inputs to a cognition layer, not the layer itself. AI lowers the cost of producing diagnoses but not the cost of institutional ownership. The portable dath ilan move is: make consequence models public, contestable, and coupled to decisions before damage occurs.
I. The legitimacy gate
You can perfectly compute the failure mode of a government policy, price it in a prediction market, and publish it for free. The policy will still happen.
A prediction market says the policy will fail. An AI audit finds the contradiction. A think tank publishes the mechanism. A spreadsheet proves the fiscal path is impossible. Nothing happens.
This is the standard failure mode for rationalist governance fixes. The signal is correct and public. The signal does not bind, does not steer, does not get answered, and within a year it is forgotten. The default rationalist diagnosis is either public choice ("politicians are corrupt") or epistemic inadequacy ("voters are irrational"). Both are sometimes true. Neither names the missing procedural step.
The missing step is what I'll call the legitimacy gate: the set of questions a cognition artifact must answer before it can attach to a public decision.
Who authorized this model?
Who can contest it?
Which institution owns the response?
What happens if it is ignored?
Which legitimate decision-maker must take it into account?
A cognition artifact passes the legitimacy gate when a named actor must respond to it through a visible procedure: endorse the model, contest it, override it, delay the decision, trigger an audit, log it in a public register, or schedule a public review comparing the prediction to what actually happens. Without that procedural consequence, the artifact remains advice.
Earth's state machinery is legitimacy-first. Who may decide got centuries of institutional iteration. What will the decision do did not. Cognition was left fragmented across statistics offices, audit bodies, ministries, courts, academia, media, and domain regulators — none of which owns the consequence-mapping function for a proposed decision before it locks in.
A signal can be true, useful, public, and still have nowhere legitimate to attach. When that happens, it becomes advice, journalism, activism, or a spreadsheet that someone politely fails to read.
II. Specimen: every actor was acting within role
A concrete case before the rest of the abstraction.
In June 2022, Finland's statistical authority reclassified state-subsidized housing loans onto the public balance sheet after a Eurostat audit. Overnight Finland's headline EDP debt ratio rose by about six percentage points. (The method change affected debt reporting, not the deficit.) The classified stock has since grown to roughly €20 billion. Finland's primary countercyclical housing instrument — interest-subsidy loans — now increases the headline debt ratio just as housing construction enters a deep recession.
Markus Sovala, the statistical authority's director general, has defended the principle: statistical authorities should not weight outcomes when classifying transactions. He is correct. The 2025 firing of the US Bureau of Labor Statistics Commissioner over weak jobs numbers is exactly the failure mode that statistical independence exists to prevent.
The Netherlands, with a comparable social-housing setup, has contested the equivalent Eurostat interpretation for ten years and continues to exclude its loans from public debt. Finland accepted in three. The difference is not statistical independence — both authorities are independent — but that no Finnish institution owned the question of what would happen if Eurostat's interpretation were accepted in this case before the decision locked in.
Every actor was acting within role. That was the failure. Each agent optimized for its local objective. None of those objectives included the cross-domain consequence model.
III. Dath ilan already has the coupling layer
Dath ilan is often imagined as Earth-with-prediction-markets-and-better-norms. The deeper feature, and the part actually worth importing, is that cognition there is already institutionally coupled to decisions. The Keepers and the surrounding civic infrastructure don't supply better forecasting in isolation; they supply forecasting that has somewhere legitimate to attach.
The naive port — "legalize prediction markets, fund forecasting, add AI advisors" — copies the signal layer and skips the coupling layer. Earth cannot lift dath ilan's cognition organs without the legitimacy substrate they rest on. The coupling layer is the actual hard import.
IV. Prediction markets are inputs, not the cognition layer
A prediction market answers priced questions: will X happen? It does not select which questions become institutionally live. It does not specify the mechanism a policy claims will produce X. It does not name the hidden variable that absorbs the cost the policy displaces. It does not assign ownership of the response. It does not impose a duty to act when the price moves. A market is signal infrastructure.
A cognition layer, in the sense I mean, produces a different artifact. For a proposed decision, it outputs:
The mechanism claim: the causal path that is supposed to produce the stated outcome.
The carrying assumptions: what must be true for the mechanism to work.
Forecasts: what we should expect to observe and when, including market and expert signals where available.
An absorption audit: which unowned variables may carry the cost.
An owner map: which institution owns which downstream consequence.
A falsification trigger: what observation would show the mechanism failed.
A review date: when the prediction is compared to outcome in public.
Prediction markets are useful inputs to (3). They do not by themselves produce (1), (2), (4), (5), (6), or (7). A cognition layer is the procedural tripwire that forces a specific decision-maker either to publicly endorse a falsifiable mechanism model — with its assumptions, its absorption audit, and its review date — or to have their decision delayed, contested, or marked as unmodeled in a place that subsequent voters can see. "Attach to a decision" means that level of teeth, not "publish a PDF appendix."
A note on futarchy: Hanson's vote on values, bet on beliefs — elected representatives define and manage a national welfare metric, markets price which policies raise it — is the cleanest rationalist attempt at exactly the structural move this section names. Constitutionalizing the market's role inside a values metric set by legitimate principals satisfies the legitimacy-gate requirement by construction: named principal authorizes the domain, cognition layer operates inside it, the verdict couples to policy. On the legitimacy-gate axis, futarchy is the cleanest existing proposal.
Run it against the seven outputs above and a different picture appears. The market produces (3), the forecast. It does not produce (1) the mechanism claim — traders aren't required to publish their causal models, and the price aggregates private heterogeneous beliefs into a probability without naming any of them. It does not produce (2) the carrying assumptions, which stay private; or (4) the absorption audit, because no one writes a market on a variable nobody has named. (5), (6), and (7) are partial: there is constitutional ownership at the meta-level and price-vs-realization comparison at settlement, but no per-decision owner map and no mechanism-level falsification, because there is no published mechanism to falsify. Futarchy binds without modeling. It is a binding forecast layer, not a cognition layer. Constitutionalizing the market makes its verdict authoritative; it doesn't make the market produce the missing outputs.
The cognition organs that did emerge on Earth — what I'll call bounded oracles: institutions that produce decision-relevant cognition inside a scope-limited mandate set by a legitimate principal — succeeded by accepting that shape. CBO, central banks, and fiscal councils answer a defined class of questions for a defined client and stay out of the rest. They did not try to make cognition the general principal; they took a bounded mandate from one. Hanson's hard parts (who defines the welfare measure, who controls the agenda, how manipulation is audited, why losing factions accept the result) are not engineering edge cases — they are the legitimacy stack systematically refusing the principal-replacement move. This framework predicts that futarchy stays difficult to institutionalize inside incumbent polities with working legitimacy stacks because no incumbent faction benefits from cognition selecting policy over them.
V. Earth's cognition stack is fragmented
Earth has cognition. It is not absent. It is fragmented across roughly five categories.
Descriptive cognition. Statistics offices, censuses, national accounts. Authoritative, often insulated, structurally retrospective. They tell us what is, not what the proposed decision will do.
Retrospective cognition. Auditors, ombudsmen, inquiries, courts. They review after the harm. By the time they report, political capital is committed.
Bounded oracles. Central banks, fiscal councils (CBO, OBR, the Irish IFAC, Finland's Economic Policy Council). The closest things to decision-coupled cognition that already work. Politically tolerable because the boundary is mandated and legitimate.
Adversarial cognition. Intelligence assessment, strategic warning, military planning, wargaming. States build cross-domain consequence modeling here because the adversary supplies fast feedback.
Decentralized signal infrastructure. Prediction markets, open-source tooling, AI analysis. New, growing, and largely outside the legitimacy gate.
These are partial successes. Earth still lacks a generic, decision-coupled cognition layer that forces consequence-mapping before capital locks in.
The historical reason for the asymmetry is selection-gradient. Legitimacy failures are fast and lethal — contested succession, refused tax, civil war — and the actors who hold power pay the cost in their own generation. Cognition failures are slow and diffuse: bad policy theory, mismodeled second-order effects, accumulated legal incoherence. The cost arrives as debt, demographic drift, decayed infrastructure, and slow productivity loss, absorbed by future generations rather than the actor responsible. Selection ratcheted the fast loop. The slow loop drifted. This is also why the bounded oracles that did emerge are mostly downstream of fast feedback or adversarial pressure — monetary instability, fiscal crisis, congressional-executive conflict, external threat — not downstream of generic concern about governance quality.
VI. AI eats the technical bottleneck, not the legitimacy bottleneck
AI and open-source tooling dramatically lower the cost of producing cognition artifacts. They do not solve the procedural bottleneck. Producing a true, computable diagnosis outside the state does not make it decision-relevant. AI does not decide which artifacts are legitimate, which must be answered, which decision-maker must change course, or which institution must own the response. It changes the political question from "can the state compute this?" to "which legitimate actors must answer when the computation is public?"
Inadequacy is now a computable object. Repair is not.
VII. Design constraints: Hayek, Scott, and the NEPA trap
Cognition layers can fail in two opposite directions. Hayek's knowledge problem and Scott's high-modernism critique name the first: the layer mistakes its model for the world, and centralized confidence destroys local information. The second is that the layer survives but becomes ceremonial. The US National Environmental Policy Act (NEPA) and its environmental impact statement (EIS) regime is the cautionary case here. NEPA created a procedural requirement for federal agencies to review significant environmental consequences and inform the public before major decisions — recognizably a consequence-modeling requirement. In practice, final EISs commonly run to several hundred pages and take years. The CEQ's 2013–2018 sample found a 661-page average and 447-page median final EIS, well above the original 150–300-page expectation; recent CEQ data give a median of around 2.2 years from notice of intent to final EIS. The result is documents that function partly as litigation defense and compliance documentation rather than agile model-update infrastructure. Power's Audit Society generalizes the same failure mode: when verification becomes mandated, capture answers with rituals.
import local knowledge without pretending local knowledge has been fully centralized;
compare prediction to outcome in public after the decision;
carry adversarial review and a right of reply;
have the contestation route mandated, not just the production route — so that compliance-theatre fictions can be challenged by counter-models with formal standing;
Designing around the NEPA trap also requires hard procedural bounds rather than open-ended compliance: strict page and time limits, automated rejection of boilerplate language, mandatory machine-readable structure, and non-trivial adversarial-review funding so that opposing-model production is not orders of magnitude under-resourced relative to the official model.
VIII. Build the diagnostic substrate first
The mistake is to start with "mandate this in legislation." That asks the equilibrium to have already solved itself: asking the inadequate system to bind itself by passing a Mandatory Cognition Act is asking for the legitimacy gate to already be open. Two paths actually work — build signals that need no permission, and build procedures that ride existing legitimate vehicles.
The unofficial artifacts are catalysts, not the cognition layer itself. A one-shot think-tank PDF gets ignored because it's a one-shot signal with no compounding mechanism and no cheap path for any rival institution to use. An automated, versioned, public diagnostic substrate is different in three specific ways:
It compounds across years and electoral cycles, accumulating receipts faster than any individual decision-maker can outrun.
It indexes durably against named decisions and named decision-makers, so the cost of a rival rediscovering the inadequacy collapses to a search query.
It lowers the activation energy for legitimate actors with grievances — opposition staffers, courts, regulators, journalists, fiscal councils — to wield the diagnosis, the way opposition parties, courts, and journalists have long wielded independent think-tank and academic analysis against incumbents, but at automated cost, indexed durability, and structural specificity that previous artifacts could not match.
This is also where diagnostic substrate diverges from prediction markets. A prediction market produces a probability: "this policy will fail with X%." That number has no procedural attachment, no specific contradiction, no citeable structure. Probability and legal weapon are not the same kind of object. Markets price questions; substrates produce structured, citeable findings that already speak the legitimacy stack's native language.
Outside-the-state moves (no permission needed).
Counter-model registries: public, versioned consequence models for major bills, produced unofficially by think tanks, civic groups, or AI-augmented analysts, attached to the bill's name in a way that journalists, opposition staffers, and courts can find later.
Forecast-to-outcome receipts: anyone can publish the receipts when an official forecast is wrong by more than its stated uncertainty band, with the receipts attaching durably to the decision-maker's name in indexable public records.
Pre-existing-vehicle moves (require legitimate principals, not new agencies).
Generalize CBO/OBR-style scoring beyond fiscal cost to mechanism claims, inside the existing fiscal-council mandate. The legitimacy substrate already exists.
Attach mechanism briefs to ministerial impact assessments where existing law already requires impact assessment. Use the existing legal hook; expand the content.
Use bounded oracles' existing remit to publish consequence models for specific upstream inputs to their domain (e.g., a central bank's macroprudential mandate may already justify publishing consequence models for housing-finance classifications that affect debt ratios and construction cycles).
These artifacts do not yet bind decisions. What they change is the next argument: the question shifts from "could anyone have known?" to "why was the known model ignored?" That is the lever by which inadequate equilibria start to move.
IX. A falsifiable prediction: AI governance bodies
The framework outputs a concrete prediction LessWrong readers can watch over the next governance cycle. Of the AI safety institutes, AI Act bodies, and national AI-governance organs emerging this decade, survival will track statutory standing × intra-elite usefulness more strongly than technical evaluation quality. A body with thin legitimacy substrate but excellent technical work that nevertheless survives politically — without acquiring an intra-elite principal who finds it useful as a defensive instrument — would falsify the framework. So would the inverse: a body with weak technical work and strong intra-elite anchoring that nevertheless gets defunded or marginalized despite its institutional patrons.
X. Close
Earth is not going to import dath ilan's Keepers. We do not need to. The first import from dath ilan is the norm that public decisions should not remain cognitively ownerless — that some institution must own the consequence model of a decision before it locks in.
Background. The source for this argument is my essay Legitimacy Came Before Cognition(kunnas.com), which develops the historical asymmetry between legitimacy and cognition stacks across multiple civilizations. The repair taxonomy this implies is in Bad Equilibria Are Not One Thing. Yudkowsky's Inadequate Equilibria (MIRI, 2017) asks when an individual can know better than the status quo; this post asks what civic infrastructure would make the status quo know better before damage. The "every actor was acting within role" formulation generalizes Michael Power's The Audit Society: Rituals of Verification (Oxford UP, 1997) into the legitimacy/cognition split.
Epistemic status: Institutional-design hypothesis drawn from historical state-capacity research, applied to current rationalist governance proposals. High confidence that the legitimacy bottleneck described here is real; moderate-to-high confidence that it is binding for the class of governance fixes discussed. Moderate confidence in the proposed substrate-first remedies. The AI governance prediction in §IX is explicitly falsifiable.
TLDR: Rationalist governance discourse keeps trying to fix the state by building better signal generators — prediction markets, AI audits, superforecasters, open-source diagnostics. The bottleneck is often not signal quality; it's the legitimacy gate: the question of who must own a model, who can contest it, who must answer when it is ignored. A signal is not yet governance until some legitimate actor must respond to it. Prediction markets are inputs to a cognition layer, not the layer itself. AI lowers the cost of producing diagnoses but not the cost of institutional ownership. The portable dath ilan move is: make consequence models public, contestable, and coupled to decisions before damage occurs.
I. The legitimacy gate
You can perfectly compute the failure mode of a government policy, price it in a prediction market, and publish it for free. The policy will still happen.
A prediction market says the policy will fail. An AI audit finds the contradiction. A think tank publishes the mechanism. A spreadsheet proves the fiscal path is impossible. Nothing happens.
This is the standard failure mode for rationalist governance fixes. The signal is correct and public. The signal does not bind, does not steer, does not get answered, and within a year it is forgotten. The default rationalist diagnosis is either public choice ("politicians are corrupt") or epistemic inadequacy ("voters are irrational"). Both are sometimes true. Neither names the missing procedural step.
The missing step is what I'll call the legitimacy gate: the set of questions a cognition artifact must answer before it can attach to a public decision.
A cognition artifact passes the legitimacy gate when a named actor must respond to it through a visible procedure: endorse the model, contest it, override it, delay the decision, trigger an audit, log it in a public register, or schedule a public review comparing the prediction to what actually happens. Without that procedural consequence, the artifact remains advice.
Earth's state machinery is legitimacy-first. Who may decide got centuries of institutional iteration. What will the decision do did not. Cognition was left fragmented across statistics offices, audit bodies, ministries, courts, academia, media, and domain regulators — none of which owns the consequence-mapping function for a proposed decision before it locks in.
A signal can be true, useful, public, and still have nowhere legitimate to attach. When that happens, it becomes advice, journalism, activism, or a spreadsheet that someone politely fails to read.
II. Specimen: every actor was acting within role
A concrete case before the rest of the abstraction.
In June 2022, Finland's statistical authority reclassified state-subsidized housing loans onto the public balance sheet after a Eurostat audit. Overnight Finland's headline EDP debt ratio rose by about six percentage points. (The method change affected debt reporting, not the deficit.) The classified stock has since grown to roughly €20 billion. Finland's primary countercyclical housing instrument — interest-subsidy loans — now increases the headline debt ratio just as housing construction enters a deep recession.
Markus Sovala, the statistical authority's director general, has defended the principle: statistical authorities should not weight outcomes when classifying transactions. He is correct. The 2025 firing of the US Bureau of Labor Statistics Commissioner over weak jobs numbers is exactly the failure mode that statistical independence exists to prevent.
The Netherlands, with a comparable social-housing setup, has contested the equivalent Eurostat interpretation for ten years and continues to exclude its loans from public debt. Finland accepted in three. The difference is not statistical independence — both authorities are independent — but that no Finnish institution owned the question of what would happen if Eurostat's interpretation were accepted in this case before the decision locked in.
Every actor was acting within role. That was the failure. Each agent optimized for its local objective. None of those objectives included the cross-domain consequence model.
III. Dath ilan already has the coupling layer
Dath ilan is often imagined as Earth-with-prediction-markets-and-better-norms. The deeper feature, and the part actually worth importing, is that cognition there is already institutionally coupled to decisions. The Keepers and the surrounding civic infrastructure don't supply better forecasting in isolation; they supply forecasting that has somewhere legitimate to attach.
The naive port — "legalize prediction markets, fund forecasting, add AI advisors" — copies the signal layer and skips the coupling layer. Earth cannot lift dath ilan's cognition organs without the legitimacy substrate they rest on. The coupling layer is the actual hard import.
IV. Prediction markets are inputs, not the cognition layer
A prediction market answers priced questions: will X happen? It does not select which questions become institutionally live. It does not specify the mechanism a policy claims will produce X. It does not name the hidden variable that absorbs the cost the policy displaces. It does not assign ownership of the response. It does not impose a duty to act when the price moves. A market is signal infrastructure.
A cognition layer, in the sense I mean, produces a different artifact. For a proposed decision, it outputs:
Prediction markets are useful inputs to (3). They do not by themselves produce (1), (2), (4), (5), (6), or (7). A cognition layer is the procedural tripwire that forces a specific decision-maker either to publicly endorse a falsifiable mechanism model — with its assumptions, its absorption audit, and its review date — or to have their decision delayed, contested, or marked as unmodeled in a place that subsequent voters can see. "Attach to a decision" means that level of teeth, not "publish a PDF appendix."
A note on futarchy: Hanson's vote on values, bet on beliefs — elected representatives define and manage a national welfare metric, markets price which policies raise it — is the cleanest rationalist attempt at exactly the structural move this section names. Constitutionalizing the market's role inside a values metric set by legitimate principals satisfies the legitimacy-gate requirement by construction: named principal authorizes the domain, cognition layer operates inside it, the verdict couples to policy. On the legitimacy-gate axis, futarchy is the cleanest existing proposal.
Run it against the seven outputs above and a different picture appears. The market produces (3), the forecast. It does not produce (1) the mechanism claim — traders aren't required to publish their causal models, and the price aggregates private heterogeneous beliefs into a probability without naming any of them. It does not produce (2) the carrying assumptions, which stay private; or (4) the absorption audit, because no one writes a market on a variable nobody has named. (5), (6), and (7) are partial: there is constitutional ownership at the meta-level and price-vs-realization comparison at settlement, but no per-decision owner map and no mechanism-level falsification, because there is no published mechanism to falsify. Futarchy binds without modeling. It is a binding forecast layer, not a cognition layer. Constitutionalizing the market makes its verdict authoritative; it doesn't make the market produce the missing outputs.
The cognition organs that did emerge on Earth — what I'll call bounded oracles: institutions that produce decision-relevant cognition inside a scope-limited mandate set by a legitimate principal — succeeded by accepting that shape. CBO, central banks, and fiscal councils answer a defined class of questions for a defined client and stay out of the rest. They did not try to make cognition the general principal; they took a bounded mandate from one. Hanson's hard parts (who defines the welfare measure, who controls the agenda, how manipulation is audited, why losing factions accept the result) are not engineering edge cases — they are the legitimacy stack systematically refusing the principal-replacement move. This framework predicts that futarchy stays difficult to institutionalize inside incumbent polities with working legitimacy stacks because no incumbent faction benefits from cognition selecting policy over them.
V. Earth's cognition stack is fragmented
Earth has cognition. It is not absent. It is fragmented across roughly five categories.
These are partial successes. Earth still lacks a generic, decision-coupled cognition layer that forces consequence-mapping before capital locks in.
The historical reason for the asymmetry is selection-gradient. Legitimacy failures are fast and lethal — contested succession, refused tax, civil war — and the actors who hold power pay the cost in their own generation. Cognition failures are slow and diffuse: bad policy theory, mismodeled second-order effects, accumulated legal incoherence. The cost arrives as debt, demographic drift, decayed infrastructure, and slow productivity loss, absorbed by future generations rather than the actor responsible. Selection ratcheted the fast loop. The slow loop drifted. This is also why the bounded oracles that did emerge are mostly downstream of fast feedback or adversarial pressure — monetary instability, fiscal crisis, congressional-executive conflict, external threat — not downstream of generic concern about governance quality.
VI. AI eats the technical bottleneck, not the legitimacy bottleneck
AI and open-source tooling dramatically lower the cost of producing cognition artifacts. They do not solve the procedural bottleneck. Producing a true, computable diagnosis outside the state does not make it decision-relevant. AI does not decide which artifacts are legitimate, which must be answered, which decision-maker must change course, or which institution must own the response. It changes the political question from "can the state compute this?" to "which legitimate actors must answer when the computation is public?"
Inadequacy is now a computable object. Repair is not.
VII. Design constraints: Hayek, Scott, and the NEPA trap
Cognition layers can fail in two opposite directions. Hayek's knowledge problem and Scott's high-modernism critique name the first: the layer mistakes its model for the world, and centralized confidence destroys local information. The second is that the layer survives but becomes ceremonial. The US National Environmental Policy Act (NEPA) and its environmental impact statement (EIS) regime is the cautionary case here. NEPA created a procedural requirement for federal agencies to review significant environmental consequences and inform the public before major decisions — recognizably a consequence-modeling requirement. In practice, final EISs commonly run to several hundred pages and take years. The CEQ's 2013–2018 sample found a 661-page average and 447-page median final EIS, well above the original 150–300-page expectation; recent CEQ data give a median of around 2.2 years from notice of intent to final EIS. The result is documents that function partly as litigation defense and compliance documentation rather than agile model-update infrastructure. Power's Audit Society generalizes the same failure mode: when verification becomes mandated, capture answers with rituals.
A cognition layer that survives both must:
Designing around the NEPA trap also requires hard procedural bounds rather than open-ended compliance: strict page and time limits, automated rejection of boilerplate language, mandatory machine-readable structure, and non-trivial adversarial-review funding so that opposing-model production is not orders of magnitude under-resourced relative to the official model.
VIII. Build the diagnostic substrate first
The mistake is to start with "mandate this in legislation." That asks the equilibrium to have already solved itself: asking the inadequate system to bind itself by passing a Mandatory Cognition Act is asking for the legitimacy gate to already be open. Two paths actually work — build signals that need no permission, and build procedures that ride existing legitimate vehicles.
The unofficial artifacts are catalysts, not the cognition layer itself. A one-shot think-tank PDF gets ignored because it's a one-shot signal with no compounding mechanism and no cheap path for any rival institution to use. An automated, versioned, public diagnostic substrate is different in three specific ways:
This is also where diagnostic substrate diverges from prediction markets. A prediction market produces a probability: "this policy will fail with X%." That number has no procedural attachment, no specific contradiction, no citeable structure. Probability and legal weapon are not the same kind of object. Markets price questions; substrates produce structured, citeable findings that already speak the legitimacy stack's native language.
Outside-the-state moves (no permission needed).
Pre-existing-vehicle moves (require legitimate principals, not new agencies).
These artifacts do not yet bind decisions. What they change is the next argument: the question shifts from "could anyone have known?" to "why was the known model ignored?" That is the lever by which inadequate equilibria start to move.
IX. A falsifiable prediction: AI governance bodies
The framework outputs a concrete prediction LessWrong readers can watch over the next governance cycle. Of the AI safety institutes, AI Act bodies, and national AI-governance organs emerging this decade, survival will track statutory standing × intra-elite usefulness more strongly than technical evaluation quality. A body with thin legitimacy substrate but excellent technical work that nevertheless survives politically — without acquiring an intra-elite principal who finds it useful as a defensive instrument — would falsify the framework. So would the inverse: a body with weak technical work and strong intra-elite anchoring that nevertheless gets defunded or marginalized despite its institutional patrons.
X. Close
Earth is not going to import dath ilan's Keepers. We do not need to. The first import from dath ilan is the norm that public decisions should not remain cognitively ownerless — that some institution must own the consequence model of a decision before it locks in.
Background. The source for this argument is my essay Legitimacy Came Before Cognition (kunnas.com), which develops the historical asymmetry between legitimacy and cognition stacks across multiple civilizations. The repair taxonomy this implies is in Bad Equilibria Are Not One Thing. Yudkowsky's Inadequate Equilibria (MIRI, 2017) asks when an individual can know better than the status quo; this post asks what civic infrastructure would make the status quo know better before damage. The "every actor was acting within role" formulation generalizes Michael Power's The Audit Society: Rituals of Verification (Oxford UP, 1997) into the legitimacy/cognition split.