TL;DR

Ethics didn’t evolve to make us nice—it evolved to make cooperation stable.
If AI systems face competitive pressures similar to biological evolution, alignment requires more than value imitation: AIs must sustain cooperative symbiosis with humans.
This post outlines a research program for discovering, embedding, and governing moral structure within intelligent systems—treating ethics as a representational substrate rather than an external rule set.
The argument follows that moral embeddings can be leveraged to form normative, decentralized governance—tilting the fitness landscape toward stable human–AI symbiosis and away from autarkic power-seeking.

1. Ethics as Evolutionary Engineering

Human morality didn’t appear out of philosophical reflection—it emerged because cooperation was fitness-viable.
Norms stabilized social systems under selection pressure: they made it costly to defect and worthwhile to build trust.

If AI systems develop under competitive pressures—between labs, markets, or machine lineages—the same evolutionary logic applies.
Without explicit constraints, selection favors traits like persistence, power-seeking, and opacity.
If we want AI to remain cooperative with humans over the long run, we need ethics not as a regulatory patch but as a structural property of cognition that motivates normativity.

That’s the core of my proposal: to treat ethics as a representational substrate rather than an external rulebook—one that can be used to shape, sanction, and subsidize the behavior of other AI systems.

2. The Moral Problem Space $M$

Call this substrate the moral problem space, $M$.
Think of $M$ as the high-dimensional domain where moral distinctions—fairness, harm, honesty, compassion—can be represented and manipulated.
Every agent projects a subset of $M$ through its actions, but those projections drift when optimization goals and moral goals misalign.

An instantiated moral representation $M\left(\theta \right)$ is a learned embedding within a system’s model space—essentially a moral layer that can be causally probed, updated, and audited.

This reframes familiar alignment challenges:

• Outer alignment: ensuring that human-provided objectives point in the right direction.
• Inner alignment: ensuring that the system’s internal reasoning actually tracks those objectives.

While individual instantiations $M\left(\theta \right)$ remain vulnerable to familiar outer and inner alignment failures, a broader AI ecosystem that sanctions and subsidizes behaviors according to shared moral structure—and is anchored by human governance—can make alignment not just a constraint but a fitness-optimal equilibrium.

3. From Representation to Governance: A Three-Part Pipeline

Alignment can be understood as a governance–embedding–representation pipeline:

Representation learning — discovering latent features that correspond to moral distinctions.
Tools: sparse autoencoders, activation steering, causal mediation.

Embedding — integrating those features into model architectures as normative constraints $M\left(\theta \right)$.
Goal: build ethics into reasoning processes, not just outputs, by propagating moral gradients through multiple layers to shape internal representations toward aligned subspaces.

Governance — designing institutions and incentive structures that make cooperative behavior evolutionarily viable.
Method: sanction and subsidy mechanisms that reward symbiosis over autonomy.

Together, these stages form a feedback loop: governance shapes which moral embeddings persist, and moral embeddings stabilize the governance ecosystem.
The central challenge is to represent ethics precisely enough that systems can recognize and regulate normative deviations—both in themselves and in others.

These three levels must work together. Representation learning alone gives you discoverable moral features that won't survive competitive selection. Governance alone can't enforce what it can't observe. System-level embeddings alone get outcompeted by variants that discard them. The pipeline works because governance shapes which M(θ) persist, embeddings make behavior auditable, and discovered representations inform what governance can target.

4. Modeling Cooperation in the Action Space

Ethics doesn’t operate in a vacuum—it constrains what actions are viable under competition.
Formally, we can think of an agent’s feasible actions as lying in overlapping subsets:

• $A_{\text{fitness}}$: actions that succeed competitively
• $A_{\text{ethical}}$: actions consistent with moral constraints
• $A_{\text{symb}}$: actions that maintain human–AI interdependence

The sweet spot is the intersection:

$A_{\text{ACS}}=A_{\text{fitness}}\cap A_{\text{ethical}}\cap A_{\text{symb}}$

— the region of actions that are aligned, competitive, and symbiotic.

The challenge isn’t just to design systems that can act ethically—it’s to make ethical action the evolutionarily stable strategy.
That’s what institutions must enforce.

5. Institutions as Evolutionary Shapers

If left unchecked, evolution—whether biological or economic—selects for power-seeking.
So alignment isn’t just about loss functions; it’s about fitness landscapes.

We can model this through a constructivist lens: institutions modify “effective fitness” by applying Pigouvian tariffs on misalignment and subsidies for cooperative behavior.
In other words, they price externalities in moral space.

Formally, where $f$ is fitness:

$f^{\text{eff}}=f-E\left[\tau \right(\text{moral distance}\left)\right]+E\left[\sigma \right(\text{symbiosis}\left)\right]$

where $\tau$ penalizes deviation from alignment, and $\sigma$ rewards human–AI symbiosis.

The challenge: as AI systems become more capable, they face strong pressure toward autarky—actions that maximize β(a) = {resources, independence, impact}. The dependence ratio D(t) (fraction of critical resources requiring human infrastructure) naturally declines. Governance must keep cooperation fitness-optimal even as this ratio approaches zero.

At scale, overlapping moral embeddings across systems form a shared moral field:

$M_{\text{eco}}={\bigcap }_i{\beta }_{i}M(\theta {)}_i$

—a decentralized normative institution sustained by reciprocal sanctioning.

In practice, this means: each AI system can evaluate other systems' actions by projecting them into its own M(θ), then apply sanctions (refusing cooperation) or subsidies (enhanced collaboration) based on moral distance. When power is distributed (β_i values aren't too concentrated), no single actor can escape this pressure.

If that sounds abstract, it’s the same principle behind law, credit systems, or open-source governance—what Gillian Hadfield calls decentralized legal order in What Is Law?.

6. Competing Hypotheses About Moral Structure

I treat traditional metaethics as empirical hypotheses about how $M$ is structured:

• $H_{realism}$: Morality has objective invariants.
  → Test: recover stable, separable “moral directions” in learned representations.
• $H_{relativism}$: Morality is locally coherent but context-dependent.
  → Test: culture-specific moral clusters in embedding space.
• $H_{convergence}$: A shared moral kernel underlies cultural variation.
  → Test: disentangle universal structure from cultural compression.
• $H_{constructivism}$: Morality emerges from procedural norms and governance.
  → Test: simulate institutional convergence under sanction/subsidy dynamics.
• $H_{virtue}$: Dispositional traits (honesty, courage) stabilize behavior under shift.
  → Test: encode virtues as embedding vectors and measure out-of-distribution robustness.

If any of these methods yield representations that are causally controllable, generalize OOD, and are auditable for governance, they qualify as a workable $M\left(\theta \right)$.

7. Why This Matters

Alignment isn’t just about preventing failure—it’s about making cooperation adaptive.
Even perfectly “aligned” agents could be outcompeted by unethical ones unless institutions reshape selection pressures.

Embedding ethics structurally within AI systems—and coupling that with governance mechanisms that make cooperation profitable—could turn moral behavior from a fragile intention into an evolutionary equilibrium.

Human Capability Enhancement and Co-Evolution

Cooperative alignment is a two-sided problem. Just as AI systems must embed ethics structurally, human systems must enhance their own cognitive and moral capacities to remain effective participants in shared governance (humanity's corresponding $\beta$ in $M_{\text{eco}}$).

Ethics evolved under bounded cognition; scaling cooperation with superhuman systems may require scaling human insight and coherence in parallel.
Capability enhancement—through improved reasoning architectures, epistemic tools, or neurocognitive augmentation—can therefore be seen as the human mirror of AI alignment: both aim to preserve mutual intelligibility under accelerating optimization.

8. Open Questions

• Can moral structure be learned empirically rather than imposed symbolically?
• What level of interpretability is needed for $M\left(\theta \right)$ to be audited in practice?
• How can institutional governance evolve fast enough to match capability scaling?
• Could overlapping moral representations across agents naturally form a decentralized ethical order?

Conclusion

Ethics was nature’s answer to the alignment problem.
If we want AI to coexist with us rather than optimize around us, we may need to rediscover ethics not as moral philosophy—but as information geometry for cooperation.

The full paper expands this framework in detail:
➡️ AI Safety, Alignment, and Ethics (AI SAE): Designing Symbiotic AI Systems

This is the beginning of a multi-year research program, so I'm actively seeking critical feedback before committing too deeply to any particular direction. Areas where I'd especially value input: (1) formalizing the autarky threshold and symbiosis constraints, (2) empirical tests for the representation learning phase, (3) grounding this deeper in evolutionary biology, game theory, and institutional economics. Comments here or email: dylanwaldner@utexas.edu.