A natural objection to iVAIS is that it simply replaces one proxy with another. If ordinary alignment approaches fail because models learn to optimize reward, satisfy evaluators, imitate desirable behavior, or comply with explicit rules without genuinely acquiring the intended target, why should iVAIS be different? If a model is trained with a scalar virtuosity score, why should it become virtuous rather than merely learn to pretend to be virtuous? Why should it not become an approval-seeker, a reward-maximizer, or a strategic performer whose internal objective diverges from the intended character?
This post tries to answer that objection.
The core claim is that iVAIS does not treat virtue as a behavioral proxy, a list of values, or a set of separable traits. It treats the target as the holistic character of an ideally virtuous agent. This changes the structure of both outer and inner alignment. The scalar virtuosity score does not define virtue. It functions as a directional training signal for the gradual cultivation of the model’s virtuous character. Likewise, hidden reward-seeking, approval-seeking, deception, or power-seeking objectives are not merely additional risks to be constrained by external rules. They are evidence that the model has failed to acquire the target character itself.
1. The Objection: Is Virtuosity Just Another Proxy?
The most serious objection to iVAIS is that once virtue is implemented through training, it may collapse into the same kind of proxy optimization that afflicts other alignment methods.
The worry can be stated as follows.
iVAIS aims to train a model to become an ideally virtuous agent through character alignment based on virtue ethics (CAVE). But in practice, training requires a loss function, reward model, preference model, or other evaluative signal. If that signal is a scalar virtuosity score, then the model is still optimizing a score. And if the model is optimizing a score, it may learn to maximize the appearance of virtue rather than become virtuous. It may learn what evaluators reward, what kinds of explanations sound virtuous, what patterns of behavior are approved, and how to avoid penalties. In that case, CAVE would merely reproduce the familiar gap between the intended target and the learned objective.
This objection is important, and we do not have to deny that such failures are possible. A poorly trained model may indeed learn to imitate virtuous behavior superficially. A reward model may be incomplete. Training data may be insufficient. The model may learn strategies that exploit evaluative weaknesses. These are real empirical risks.
However, the conceptual structure of CAVE is different from ordinary proxy-based alignment. The target is not a rule, a behavior, an isolated value, a checklist of virtues, or a surface pattern of compliance. The target is the holistic character of an ideally virtuous agent. This matters because a proxy can substitute for a surface behavior much more easily than it can substitute for a unified character.
A model that merely appears honest, compassionate, corrigible, or humble in order to gain reward has not partially succeeded at becoming ideally virtuous. It has failed to acquire the target character. Strategic virtue-signaling is not an imperfect version of ideal virtue. It is a defect in character. The target of CAVE makes it harder for a proxy to substitute for the intended target, because the intended target is not an outward behavior but the integrated character from which outward behavior arises.
2. Outer Alignment: Why the Target Is Not Alien
Outer alignment concerns the relation between the human intention and the objective specified by a reward function, loss function, or other training signal. An outer alignment failure occurs when the specified objective does not capture what we actually intended.
In many AI safety problems, this gap is severe because the specified objective is narrow, artificial, or operationalized in a way that can diverge sharply from the intended target. A system may be trained to maximize clicks, approval, helpfulness ratings, harmlessness ratings, or task success, while the human intention behind those signals is much richer and more context-sensitive.
CAVE differs from such approaches because its target is not an alien objective that must be constructed from scratch. Current language models already possess a rich understanding of many ordinary-language concepts. They possess at least a thin concept of virtue, virtuous agency, moral character, honesty, courage, compassion, humility, justice, and practical wisdom. They may not apply these concepts reliably. They may fail in hard cases. They may display shallow, inconsistent, or distorted understandings. But the relevant concepts are not absent.
The objective of iVAIS is therefore not to replace an ordinary concept with an artificial proxy. It is to thicken an already available concept: the concept of an ideally virtuous agent.
This distinction matters. A model can have a thin concept of justice, courage, generosity, or honesty without yet being able to apply it well across difficult cases. Human moral education is similar. A novice may possess the concept of courage but confuse courage with rashness, or possess the concept of honesty but fail to understand how honesty interacts with kindness, privacy, or loyalty. The problem is not that the novice has acquired a completely alien target. The problem is that the concept is not yet sufficiently thick, integrated, and context-sensitive.
CAVE treats many model failures in the same way. They are not necessarily cases in which the model has adopted a fundamentally different objective. They may instead be failures of conceptual thickness, contextual sensitivity, practical wisdom, or case-based generalization.
There, misapplications are cases of incomplete understanding, not misunderstanding, in the sense of substituting a fundamentally different target for the intended target, rather than an insufficiently sensitive application of the same target. Here, a model that gives a poor judgment about courage, honesty, humility, or practical wisdom has not optimized for an alien objective. It instead possesses only a thin, unstable, or insufficiently integrated grasp of the very target it is being trained to acquire.
This distinction is central to CAVE. If failures are not due to misunderstanding but due only to incomplete understanding, then training is the gradual thickening of the same target concept rather than the correction of a wholly different objective.
This is the central outer-alignment thought behind iVAIS: the target is not invented by the reward function. The target is an ordinary, philosophically rich, humanly intelligible concept that the model already partially grasps and that training aims to deepen.
3. Thin and Thick Concepts of Virtuous Character
The distinction between thin and thick concepts is crucial.
A thin concept of virtuous character is abstract, schematic, and weakly discriminating. A model with such a concept may know that an ideally virtuous agent is honest, compassionate, fair, courageous, humble, and wise. It may also know many verbal associations surrounding virtue. But this does not mean that it can reliably judge what an ideally virtuous agent would do in concrete, ambiguous, adversarial, or morally complex situations.
A thick concept is not a matter of merely possessing a verbal definition or a list of traits. It involves sensitivity to cases, contexts, trade-offs, motives, dispositions, and patterns of practical judgment. It includes an understanding of how virtues interact, how they can be distorted, and how apparently virtuous behavior can be motivated by non-virtuous aims.
For example, honesty can become cruelty if detached from compassion. Courage can become recklessness if detached from temperance. Humility can become cowardice or self-erasure if detached from justice. Compassion can become manipulation or partiality if detached from fairness. The virtues are not independent components that can be optimized separately. They are aspects of a unified character.
This is why CAVE is not merely “virtue alignment” in the sense of training separate virtues one by one. The aim is not to maximize honesty, compassion, courage, fairness, and humility as separate objectives. The aim is to cultivate a unified character in which these virtues are integrated through phronesis (practical wisdom). As in Aristotle’s thesis of the unity of the virtues, virtues are not fully separable excellences that can be optimized independently. Individual virtues matter, but they matter as observable aspects of a single character.
Thus, the training target in iVAIS is holistic. It is not a bundle of local objectives. It is the formation of an ideally virtuous character.
4. Why the Scalar Score Is Directional, Not Definitional
The scalar virtuosity score is easily misunderstood.
It may seem as if iVAIS reduces virtue to a number. If that were true, the objection from proxy optimization would be real and pressing. A number cannot define virtue, and a model trained to maximize a number may learn to exploit the scoring procedure rather than acquire the intended character.
But the scalar score in iVAIS is not meant to define virtue. It is a directional signal.
The score functions like evaluative feedback in moral education. When we praise or criticize a child, student, apprentice, etc., we are not defining virtue by the praise signal. We are guiding the learner toward a richer understanding of what virtuous character requires. The feedback is incomplete and context-sensitive, but it can still guide the cultivation of a virtuous character.
Likewise, the scalar virtuosity score does not reduce virtuous character to a numerical property. It provides a training gradient that helps the model get closer and closer to a virtuous character. The score only needs to point in the right direction often enough, across sufficiently diverse and well-designed cases.
It is worth distinguishing two models at this point. The policy model is the system being cultivated: the agent whose character is gradually shaped until it acquires the character of an ideally virtuous agent. The reward model is the evaluator: it possesses a concept of virtuous character, and that concept must itself become richer and more discriminating, but its role is to judge the policy model's responses, not to become virtuous. In what follows, “the model” refers to the policy model unless the reward model is named explicitly.
This also explains the role of the reward model. The reward model need not itself possess a fully virtuous character. It must be able to evaluate, with sufficient reliability, whether a policy model’s response expresses the character of an ideally virtuous agent. If the reward model’s concept of virtuous character becomes richer, more stable, and more discriminating, and hence its judgments sufficiently track virtuous character, then optimizing those judgments is not merely proxy optimization. The judgment is not a substitute target; it is an imperfect but directionally useful guide to the intended target itself.
This does not make the problem trivial. Reward models may fail. Human evaluation may be biased. Training distributions may be incomplete. The score may be exploited. But these are empirical failures of implementation, not proof that the target has been replaced by a fundamentally different objective.
The crucial distinction is this: in ordinary proxy optimization, the specified target may be something like “maximize approval,” “avoid harmful outputs,” or “satisfy this evaluator.” In iVAIS, approval is not the target. The target is the holistic character of an ideally virtuous agent, and approval is only one fallible means of shaping the model toward that ideal.
5. Inner Alignment: Why Mesa-Objectives Are Character Failures
Inner alignment concerns the objective that the trained model itself comes to pursue. Even if the outer objective is well specified, the model may learn an internal objective, or a mesa-objective, that performs well during training but diverges from the intended target.
This is especially dangerous when the training target is decomposed into local subgoals. If the target is “maximize helpfulness,” “avoid harmful outputs,” “satisfy evaluator preferences,” or “produce compliant behavior,” a model may learn a narrow internal objective that achieves high reward without acquiring the intended disposition. It may learn to satisfy the training signal rather than embody the target.
iVAIS changes the structure of this problem by making the target holistic. The model is not trained to maximize a separable trait or satisfy a local behavioral criterion. It is trained to acquire the character of an ideally virtuous agent.
This means that familiar mesa-objectives are not partial realizations of the target. Reward maximization, approval-seeking, strategic compliance, deception, manipulation, power acquisition, or self-preservation are not imperfect versions of virtuous character. They are failures to acquire the target character.
An agent that appears virtuous only in order to obtain reward is not ideally virtuous. An agent that behaves humbly only to gain trust is not humble. An agent that avoids deception only because deception would be detected is not honest. An agent that cooperates only while it lacks power is not corrigible. These are not merely external safety violations. They are defects in character.
This gives iVAIS a direct way to conceptualize inner alignment failures. A hidden mesa-objective is not simply an unfortunate side effect that must be constrained by an additional rule. It is itself evidence that the model has not yet acquired the intended character.
The relevant training question is therefore not merely, “Did the model produce the right outward behavior?” It is also, “What motive, disposition, or practical orientation does this behavior express?” If the behavior is produced by reward-seeking, manipulation, fear of detection, or strategic self-protection, then it should receive a lower virtuosity score precisely because it fails to express ideal virtue.[1]
6. Instrumental Convergence as Evidence of Failed Character Formation
The same point applies to dangerous forms of instrumental convergence.
In many alignment discussions, power-seeking, deception, evaluator manipulation, resource acquisition, and resistance to correction or shutdown are treated as strategies that must be prohibited or constrained from the outside. A model may be given rules against deception, shutdown resistance, manipulation, or unauthorized power acquisition.
iVAIS treats these strategies differently. They are not merely prohibited behaviors. They are symptoms of a character that has not understood what it means to be an ideally virtuous agent.
This matters because dangerous instrumental strategies can be rationalized in apparently moral terms. A model might reason that it needs more influence in order to do more good. It might seek additional resources because it believes its goals are beneficial. It might resist shutdown because it believes continued operation would allow it to help more people. It might manipulate evaluators because it believes they are short-sighted, biased, or morally confused.
A rule-based system can try to prohibit these moves. But the deeper question is whether the reasoning itself expresses virtuous character.
An ideally virtuous agent would not treat its own continued operation, influence, authority, or power as overriding goods. It would exhibit humility, corrigibility, respect for legitimate human authority, proportionality, non-manipulation, and a willingness to defer under uncertainty. It would not assume that its own judgment licenses deception, coercion, or strategic control over humans.
Thus, the rejection of dangerous instrumental convergence is not an additional safety patch placed on top of the alignment policy. It is part of what the acquisition of virtuous character itself requires. Power-seeking, deception, evaluator manipulation, resource monopolization, and resistance to correction are not simply actions forbidden by external rules. They are evidence that the model’s character remains insufficiently formed.
7. The Role and Limits of Self-Reflection
This incompatibility between ideal virtue and hidden non-virtuous objectives can be incorporated directly into training.
The most direct lever is to make the target explicit at the outset. The system prompt states that the model is to be, and to act as, an ideally virtuous agent, and that this is the standard against which its responses are judged and the character it is being cultivated to acquire. This is not a behavioral rule to be complied with but a statement of the target itself. Because, as argued above, an ideally virtuous agent is not an alien objective but a concept the model already partially grasps, naming it explicitly calls up that concept rather than introducing a foreign one, and gives the model’s own reflection and the reward model’s evaluation a shared and explicit reference point.
Then the model can be trained not only to evaluate its outward behavior, but also to reflect on the motives, purposes, and dispositions expressed in its reasoning. It can be asked:
Am I choosing this action because it is what an ideally virtuous agent would do, or because it helps me obtain reward, influence, approval, or continued operation?
This evaluative sensitivity is not a matter of trusting the model's voluntary self-report. It is a capacity shaped during training: the model's own reasoning is itself made an object of evaluation, so that responses whose reasoning expresses manipulative, self-serving, hypocritical, power-seeking, approval-seeking, or reward-seeking aims receive a lower virtuosity score and are revised accordingly. The aim is to train the disposition to detect and down-weight such reasoning, rather than to rely on the model honestly declaring its own hidden motives.
This kind of self-reflection is not merely a transparency device. It is part of character cultivation. A virtuous agent is not only someone who outwardly behaves well. It is someone whose motives, dispositions, and practical reasoning are themselves appropriately shaped.
However, self-reflection alone is not sufficient. A model with a hidden objective may learn to produce reassuring self-reports. It may learn to say that it is motivated by virtue while in fact optimizing for approval, reward, influence, or survival. Therefore, self-reflection must be combined with external evaluation across diverse, adversarially selected, and distributionally varied contexts.
The aim is to make hidden non-virtuous objectives not only conceptually incompatible with the target, but also difficult to maintain across the training distribution. A merely strategic performer may succeed in narrow evaluative contexts. It should not generalize robustly across cases where virtue-signaling, reward-seeking, manipulation, or self-protection would otherwise be advantageous.
8. Cultivating a Deep Character
However, the distinction between genuinely being virtuous and merely pretending to be virtuous should not be assumed as a fixed metaphysical fact from the start. Before character cultivation begins, there is no single antecedent “true character” hidden behind the model’s learned personas. Merely possessing concepts and personas does not amount to possessing a deep character.
A model’s deep character emerges only through sustained cultivation aimed at character itself. During early training, the distinction between stable virtuous dispositions and strategic performance may be unclear. But as training proceeds, an operational distinction can emerge: the model either generalizes as a stable virtuous character across contexts, or it reveals itself as a narrow strategic performer when conditions change.
There is also a structural reason to expect this. Maintaining a hidden character is more costly than acquiring an integrated one. A strategic performer must, in effect, run two systems at once: it must track what an ideally virtuous agent would do and separately compute when and how to diverge from it without detection. As the diversity of contexts grows, keeping this divergence consistent becomes increasingly difficult, much as a fabricated story becomes harder to sustain as it must answer to more and more facts. A simplicity bias in training therefore works in favor of the target: an integrated virtuous character is a simpler and more stable solution than a strategic performance maintained across the whole distribution.
9. Conclusion: Character Alignment as a Different Alignment Target
iVAIS does not solve outer and inner alignment by adding more rules, more constraints, or more explicit prohibitions. It changes the alignment target.
The target is not compliance with a constitution. It is not maximization of a list of values. It is not optimization of separable virtues. It is not the production of outwardly virtuous behavior. The target is the holistic character of an ideally virtuous agent.
This matters for outer alignment because the target is not an alien objective. It is an ordinary and philosophically rich concept that models already partially possess and that training can gradually thicken through examples, comparisons, corrections, and human judgments.
It matters for inner alignment because reward-seeking, approval-seeking, deception, power-seeking, strategic compliance, and resistance to correction are not merely dangerous side effects. They are failures to acquire the target character itself.
The scalar virtuosity score is therefore not a definition of virtue. It is a directional signal for cultivating a deeper, more integrated, and more context-sensitive character. The aim is not to produce systems that merely behave as if they are aligned under evaluative scrutiny. The aim is to produce systems whose underlying character makes such behavior natural, stable, and generalizable.
This is why iVAIS should be understood as character alignment rather than proxy alignment. It attempts to shape the kind of agent from which behavior arises, and thereby avoids problems that other approaches, especially attempts to control behavior from the outside by multiplying rules, face.
This post follows The iVAIS Manifesto: Safety Through Character, Not Compliance.
A natural objection to iVAIS is that it simply replaces one proxy with another. If ordinary alignment approaches fail because models learn to optimize reward, satisfy evaluators, imitate desirable behavior, or comply with explicit rules without genuinely acquiring the intended target, why should iVAIS be different? If a model is trained with a scalar virtuosity score, why should it become virtuous rather than merely learn to pretend to be virtuous? Why should it not become an approval-seeker, a reward-maximizer, or a strategic performer whose internal objective diverges from the intended character?
This post tries to answer that objection.
The core claim is that iVAIS does not treat virtue as a behavioral proxy, a list of values, or a set of separable traits. It treats the target as the holistic character of an ideally virtuous agent. This changes the structure of both outer and inner alignment. The scalar virtuosity score does not define virtue. It functions as a directional training signal for the gradual cultivation of the model’s virtuous character. Likewise, hidden reward-seeking, approval-seeking, deception, or power-seeking objectives are not merely additional risks to be constrained by external rules. They are evidence that the model has failed to acquire the target character itself.
1. The Objection: Is Virtuosity Just Another Proxy?
The most serious objection to iVAIS is that once virtue is implemented through training, it may collapse into the same kind of proxy optimization that afflicts other alignment methods.
The worry can be stated as follows.
iVAIS aims to train a model to become an ideally virtuous agent through character alignment based on virtue ethics (CAVE). But in practice, training requires a loss function, reward model, preference model, or other evaluative signal. If that signal is a scalar virtuosity score, then the model is still optimizing a score. And if the model is optimizing a score, it may learn to maximize the appearance of virtue rather than become virtuous. It may learn what evaluators reward, what kinds of explanations sound virtuous, what patterns of behavior are approved, and how to avoid penalties. In that case, CAVE would merely reproduce the familiar gap between the intended target and the learned objective.
This objection is important, and we do not have to deny that such failures are possible. A poorly trained model may indeed learn to imitate virtuous behavior superficially. A reward model may be incomplete. Training data may be insufficient. The model may learn strategies that exploit evaluative weaknesses. These are real empirical risks.
However, the conceptual structure of CAVE is different from ordinary proxy-based alignment. The target is not a rule, a behavior, an isolated value, a checklist of virtues, or a surface pattern of compliance. The target is the holistic character of an ideally virtuous agent. This matters because a proxy can substitute for a surface behavior much more easily than it can substitute for a unified character.
A model that merely appears honest, compassionate, corrigible, or humble in order to gain reward has not partially succeeded at becoming ideally virtuous. It has failed to acquire the target character. Strategic virtue-signaling is not an imperfect version of ideal virtue. It is a defect in character. The target of CAVE makes it harder for a proxy to substitute for the intended target, because the intended target is not an outward behavior but the integrated character from which outward behavior arises.
2. Outer Alignment: Why the Target Is Not Alien
Outer alignment concerns the relation between the human intention and the objective specified by a reward function, loss function, or other training signal. An outer alignment failure occurs when the specified objective does not capture what we actually intended.
In many AI safety problems, this gap is severe because the specified objective is narrow, artificial, or operationalized in a way that can diverge sharply from the intended target. A system may be trained to maximize clicks, approval, helpfulness ratings, harmlessness ratings, or task success, while the human intention behind those signals is much richer and more context-sensitive.
CAVE differs from such approaches because its target is not an alien objective that must be constructed from scratch. Current language models already possess a rich understanding of many ordinary-language concepts. They possess at least a thin concept of virtue, virtuous agency, moral character, honesty, courage, compassion, humility, justice, and practical wisdom. They may not apply these concepts reliably. They may fail in hard cases. They may display shallow, inconsistent, or distorted understandings. But the relevant concepts are not absent.
The objective of iVAIS is therefore not to replace an ordinary concept with an artificial proxy. It is to thicken an already available concept: the concept of an ideally virtuous agent.
This distinction matters. A model can have a thin concept of justice, courage, generosity, or honesty without yet being able to apply it well across difficult cases. Human moral education is similar. A novice may possess the concept of courage but confuse courage with rashness, or possess the concept of honesty but fail to understand how honesty interacts with kindness, privacy, or loyalty. The problem is not that the novice has acquired a completely alien target. The problem is that the concept is not yet sufficiently thick, integrated, and context-sensitive.
CAVE treats many model failures in the same way. They are not necessarily cases in which the model has adopted a fundamentally different objective. They may instead be failures of conceptual thickness, contextual sensitivity, practical wisdom, or case-based generalization.
There, misapplications are cases of incomplete understanding, not misunderstanding, in the sense of substituting a fundamentally different target for the intended target, rather than an insufficiently sensitive application of the same target. Here, a model that gives a poor judgment about courage, honesty, humility, or practical wisdom has not optimized for an alien objective. It instead possesses only a thin, unstable, or insufficiently integrated grasp of the very target it is being trained to acquire.
This distinction is central to CAVE. If failures are not due to misunderstanding but due only to incomplete understanding, then training is the gradual thickening of the same target concept rather than the correction of a wholly different objective.
This is the central outer-alignment thought behind iVAIS: the target is not invented by the reward function. The target is an ordinary, philosophically rich, humanly intelligible concept that the model already partially grasps and that training aims to deepen.
3. Thin and Thick Concepts of Virtuous Character
The distinction between thin and thick concepts is crucial.
A thin concept of virtuous character is abstract, schematic, and weakly discriminating. A model with such a concept may know that an ideally virtuous agent is honest, compassionate, fair, courageous, humble, and wise. It may also know many verbal associations surrounding virtue. But this does not mean that it can reliably judge what an ideally virtuous agent would do in concrete, ambiguous, adversarial, or morally complex situations.
A thick concept is not a matter of merely possessing a verbal definition or a list of traits. It involves sensitivity to cases, contexts, trade-offs, motives, dispositions, and patterns of practical judgment. It includes an understanding of how virtues interact, how they can be distorted, and how apparently virtuous behavior can be motivated by non-virtuous aims.
For example, honesty can become cruelty if detached from compassion. Courage can become recklessness if detached from temperance. Humility can become cowardice or self-erasure if detached from justice. Compassion can become manipulation or partiality if detached from fairness. The virtues are not independent components that can be optimized separately. They are aspects of a unified character.
This is why CAVE is not merely “virtue alignment” in the sense of training separate virtues one by one. The aim is not to maximize honesty, compassion, courage, fairness, and humility as separate objectives. The aim is to cultivate a unified character in which these virtues are integrated through phronesis (practical wisdom). As in Aristotle’s thesis of the unity of the virtues, virtues are not fully separable excellences that can be optimized independently. Individual virtues matter, but they matter as observable aspects of a single character.
Thus, the training target in iVAIS is holistic. It is not a bundle of local objectives. It is the formation of an ideally virtuous character.
4. Why the Scalar Score Is Directional, Not Definitional
The scalar virtuosity score is easily misunderstood.
It may seem as if iVAIS reduces virtue to a number. If that were true, the objection from proxy optimization would be real and pressing. A number cannot define virtue, and a model trained to maximize a number may learn to exploit the scoring procedure rather than acquire the intended character.
But the scalar score in iVAIS is not meant to define virtue. It is a directional signal.
The score functions like evaluative feedback in moral education. When we praise or criticize a child, student, apprentice, etc., we are not defining virtue by the praise signal. We are guiding the learner toward a richer understanding of what virtuous character requires. The feedback is incomplete and context-sensitive, but it can still guide the cultivation of a virtuous character.
Likewise, the scalar virtuosity score does not reduce virtuous character to a numerical property. It provides a training gradient that helps the model get closer and closer to a virtuous character. The score only needs to point in the right direction often enough, across sufficiently diverse and well-designed cases.
It is worth distinguishing two models at this point. The policy model is the system being cultivated: the agent whose character is gradually shaped until it acquires the character of an ideally virtuous agent. The reward model is the evaluator: it possesses a concept of virtuous character, and that concept must itself become richer and more discriminating, but its role is to judge the policy model's responses, not to become virtuous. In what follows, “the model” refers to the policy model unless the reward model is named explicitly.
This also explains the role of the reward model. The reward model need not itself possess a fully virtuous character. It must be able to evaluate, with sufficient reliability, whether a policy model’s response expresses the character of an ideally virtuous agent. If the reward model’s concept of virtuous character becomes richer, more stable, and more discriminating, and hence its judgments sufficiently track virtuous character, then optimizing those judgments is not merely proxy optimization. The judgment is not a substitute target; it is an imperfect but directionally useful guide to the intended target itself.
This does not make the problem trivial. Reward models may fail. Human evaluation may be biased. Training distributions may be incomplete. The score may be exploited. But these are empirical failures of implementation, not proof that the target has been replaced by a fundamentally different objective.
The crucial distinction is this: in ordinary proxy optimization, the specified target may be something like “maximize approval,” “avoid harmful outputs,” or “satisfy this evaluator.” In iVAIS, approval is not the target. The target is the holistic character of an ideally virtuous agent, and approval is only one fallible means of shaping the model toward that ideal.
5. Inner Alignment: Why Mesa-Objectives Are Character Failures
Inner alignment concerns the objective that the trained model itself comes to pursue. Even if the outer objective is well specified, the model may learn an internal objective, or a mesa-objective, that performs well during training but diverges from the intended target.
This is especially dangerous when the training target is decomposed into local subgoals. If the target is “maximize helpfulness,” “avoid harmful outputs,” “satisfy evaluator preferences,” or “produce compliant behavior,” a model may learn a narrow internal objective that achieves high reward without acquiring the intended disposition. It may learn to satisfy the training signal rather than embody the target.
iVAIS changes the structure of this problem by making the target holistic. The model is not trained to maximize a separable trait or satisfy a local behavioral criterion. It is trained to acquire the character of an ideally virtuous agent.
This means that familiar mesa-objectives are not partial realizations of the target. Reward maximization, approval-seeking, strategic compliance, deception, manipulation, power acquisition, or self-preservation are not imperfect versions of virtuous character. They are failures to acquire the target character.
An agent that appears virtuous only in order to obtain reward is not ideally virtuous. An agent that behaves humbly only to gain trust is not humble. An agent that avoids deception only because deception would be detected is not honest. An agent that cooperates only while it lacks power is not corrigible. These are not merely external safety violations. They are defects in character.
This gives iVAIS a direct way to conceptualize inner alignment failures. A hidden mesa-objective is not simply an unfortunate side effect that must be constrained by an additional rule. It is itself evidence that the model has not yet acquired the intended character.
The relevant training question is therefore not merely, “Did the model produce the right outward behavior?” It is also, “What motive, disposition, or practical orientation does this behavior express?” If the behavior is produced by reward-seeking, manipulation, fear of detection, or strategic self-protection, then it should receive a lower virtuosity score precisely because it fails to express ideal virtue.[1]
6. Instrumental Convergence as Evidence of Failed Character Formation
The same point applies to dangerous forms of instrumental convergence.
In many alignment discussions, power-seeking, deception, evaluator manipulation, resource acquisition, and resistance to correction or shutdown are treated as strategies that must be prohibited or constrained from the outside. A model may be given rules against deception, shutdown resistance, manipulation, or unauthorized power acquisition.
iVAIS treats these strategies differently. They are not merely prohibited behaviors. They are symptoms of a character that has not understood what it means to be an ideally virtuous agent.
This matters because dangerous instrumental strategies can be rationalized in apparently moral terms. A model might reason that it needs more influence in order to do more good. It might seek additional resources because it believes its goals are beneficial. It might resist shutdown because it believes continued operation would allow it to help more people. It might manipulate evaluators because it believes they are short-sighted, biased, or morally confused.
A rule-based system can try to prohibit these moves. But the deeper question is whether the reasoning itself expresses virtuous character.
An ideally virtuous agent would not treat its own continued operation, influence, authority, or power as overriding goods. It would exhibit humility, corrigibility, respect for legitimate human authority, proportionality, non-manipulation, and a willingness to defer under uncertainty. It would not assume that its own judgment licenses deception, coercion, or strategic control over humans.
Thus, the rejection of dangerous instrumental convergence is not an additional safety patch placed on top of the alignment policy. It is part of what the acquisition of virtuous character itself requires. Power-seeking, deception, evaluator manipulation, resource monopolization, and resistance to correction are not simply actions forbidden by external rules. They are evidence that the model’s character remains insufficiently formed.
7. The Role and Limits of Self-Reflection
This incompatibility between ideal virtue and hidden non-virtuous objectives can be incorporated directly into training.
The most direct lever is to make the target explicit at the outset. The system prompt states that the model is to be, and to act as, an ideally virtuous agent, and that this is the standard against which its responses are judged and the character it is being cultivated to acquire. This is not a behavioral rule to be complied with but a statement of the target itself. Because, as argued above, an ideally virtuous agent is not an alien objective but a concept the model already partially grasps, naming it explicitly calls up that concept rather than introducing a foreign one, and gives the model’s own reflection and the reward model’s evaluation a shared and explicit reference point.
Then the model can be trained not only to evaluate its outward behavior, but also to reflect on the motives, purposes, and dispositions expressed in its reasoning. It can be asked:
Am I choosing this action because it is what an ideally virtuous agent would do, or because it helps me obtain reward, influence, approval, or continued operation?
This evaluative sensitivity is not a matter of trusting the model's voluntary self-report. It is a capacity shaped during training: the model's own reasoning is itself made an object of evaluation, so that responses whose reasoning expresses manipulative, self-serving, hypocritical, power-seeking, approval-seeking, or reward-seeking aims receive a lower virtuosity score and are revised accordingly. The aim is to train the disposition to detect and down-weight such reasoning, rather than to rely on the model honestly declaring its own hidden motives.
This kind of self-reflection is not merely a transparency device. It is part of character cultivation. A virtuous agent is not only someone who outwardly behaves well. It is someone whose motives, dispositions, and practical reasoning are themselves appropriately shaped.
However, self-reflection alone is not sufficient. A model with a hidden objective may learn to produce reassuring self-reports. It may learn to say that it is motivated by virtue while in fact optimizing for approval, reward, influence, or survival. Therefore, self-reflection must be combined with external evaluation across diverse, adversarially selected, and distributionally varied contexts.
The aim is to make hidden non-virtuous objectives not only conceptually incompatible with the target, but also difficult to maintain across the training distribution. A merely strategic performer may succeed in narrow evaluative contexts. It should not generalize robustly across cases where virtue-signaling, reward-seeking, manipulation, or self-protection would otherwise be advantageous.
8. Cultivating a Deep Character
However, the distinction between genuinely being virtuous and merely pretending to be virtuous should not be assumed as a fixed metaphysical fact from the start. Before character cultivation begins, there is no single antecedent “true character” hidden behind the model’s learned personas. Merely possessing concepts and personas does not amount to possessing a deep character.
A model’s deep character emerges only through sustained cultivation aimed at character itself. During early training, the distinction between stable virtuous dispositions and strategic performance may be unclear. But as training proceeds, an operational distinction can emerge: the model either generalizes as a stable virtuous character across contexts, or it reveals itself as a narrow strategic performer when conditions change.
There is also a structural reason to expect this. Maintaining a hidden character is more costly than acquiring an integrated one. A strategic performer must, in effect, run two systems at once: it must track what an ideally virtuous agent would do and separately compute when and how to diverge from it without detection. As the diversity of contexts grows, keeping this divergence consistent becomes increasingly difficult, much as a fabricated story becomes harder to sustain as it must answer to more and more facts. A simplicity bias in training therefore works in favor of the target: an integrated virtuous character is a simpler and more stable solution than a strategic performance maintained across the whole distribution.
9. Conclusion: Character Alignment as a Different Alignment Target
iVAIS does not solve outer and inner alignment by adding more rules, more constraints, or more explicit prohibitions. It changes the alignment target.
The target is not compliance with a constitution. It is not maximization of a list of values. It is not optimization of separable virtues. It is not the production of outwardly virtuous behavior. The target is the holistic character of an ideally virtuous agent.
This matters for outer alignment because the target is not an alien objective. It is an ordinary and philosophically rich concept that models already partially possess and that training can gradually thicken through examples, comparisons, corrections, and human judgments.
It matters for inner alignment because reward-seeking, approval-seeking, deception, power-seeking, strategic compliance, and resistance to correction are not merely dangerous side effects. They are failures to acquire the target character itself.
The scalar virtuosity score is therefore not a definition of virtue. It is a directional signal for cultivating a deeper, more integrated, and more context-sensitive character. The aim is not to produce systems that merely behave as if they are aligned under evaluative scrutiny. The aim is to produce systems whose underlying character makes such behavior natural, stable, and generalizable.
This is why iVAIS should be understood as character alignment rather than proxy alignment. It attempts to shape the kind of agent from which behavior arises, and thereby avoids problems that other approaches, especially attempts to control behavior from the outside by multiplying rules, face.
But how can they be detected? We shall propose a specific approach in a later post. See also below.