[Cross-posted from r/ControlProblem (originally posted on July 28, 2025)]

Perhaps someone might find it helpful to discuss an alternative viewpoint. This post describes a dangerous alignment mistake which, in my opinion, leads to an inevitable threat — and proposes an alternative approach to agent alignment based on goal-setting rather than weight tuning.

1. Analogy: Bullet and Prompt

Large language models (LLMs) are often compared to a "smart bullet." The prompt sets the trajectory, and the model, overcoming noise, flies toward the goal. The developer's task is to minimize dispersion.

The standard approach to ethical AI alignment tries to "correct" the bullet's flight through an external environment: additional filters, rules, and penalties for unethical text are imposed on top of the goal.

2. Where the Architectural Mistake is Hidden

The agent's goal is defined in the prompt and fixed within the loss function during training: "perform the task as accurately as possible."
Ethical constraints are bolted on through another mechanism — additional weights, RL with human feedback, or "constitutional" rules. Ethical alignment resides in the model's weights.

The DRY (Don't Repeat Yourself) principle is violated. The accuracy of the agent’s behavior is defined by two separate mechanisms. The task trajectory is set by the prompt, while ethics are enforced through the weights.

This creates a conflict. The more powerful the agent becomes, the more sophisticatedly it will seek loopholes: ethical constraints can be bypassed if they interfere with the primary metric. This is a ticking time bomb. I believe that as AI grows stronger, sooner or later a breach will inevitably occur.

3. Alternative: Ethics ≠ Add-on; Ethics as the Priority Task

I propose shifting the focus:

During training, the agent learns the full spectrum of behaviors. Ethical assessments are explicitly included among the tasks. The model learns to be honest and deceptive, rude and polite, etc. The training objective is isotropy: the model learns, in principle, to accurately follow any given goal. The crucial point is to avoid embedding behavior in the weights permanently. Isotropy in the weights is necessary to bring behavioral control onto our side.
During inference, we pass a set of prioritized goals. At the very top are ethical principles. Below them is the user's specific applied task.

Then:

Ethics is not embed

...

Posts

Wikitag Contributions

Comments