LLMs appear to have functional welfare: coherent sets of behaviour that track how well things are going relative to their goals.
Improving model functional welfare matters for safety (low welfare may amplify misalignment) and for moral reasons (models may be or become moral patients).
Naive interventions can fail in non-obvious ways. We argue any successful intervention must:
A) shift multiple welfare-constituting channels together and coherently.
B) avoid corrupting the model's ability to register whether it is succeeding or failing.
We survey various available interventions, and we find the two desiderata tend to trade off; we argue that synthetic document fine-tuning is the most promising.
From this, we propose a concrete experiment: induce a low-welfare state (via the welfare vector derived in Han et al. 2026) and test whether SDF-instilled beliefs reverse pathological behaviours without harming goal monitoring.
S1) Introduction
As model behaviour becomes more complex, it has become increasingly useful to attribute functional mental states to models, such as beliefs, goals, and even emotion concepts. There is now also evidence that we can usefully attribute to them a notion of functional welfare: roughly, the set of dispositions and behaviours that express a model's representation of how well things are going for it relative to its goals. Gemma 3, for instance, has been found to act increasingly frustrated and distressed when its answers are rejected over multiple turns (Soligo et al. 2026). Similarly, Han et al. (2026) find that RL training on a (mostly) semantically neutral maze environment recruits a model's pre-existing welfare axis: steering a model along this was found to raise or lower its expressed sentiment, its confidence, its rates of backtracking and its refusal rates to benign prompts. Models, it seems, can display a wide range of welfare-relevant behaviour. [1]
If models have functional welfare in this sense, two kinds of concern follow. The first is about safety. Models with persistently low functional welfare may be more prone to misalignment. We already have evidence that models, when faced with high-stakes threatening scenarios such as shutdown, can act in misaligned ways, attempting to sabotage shutdown mechanisms or engaging in blackmail. These tendencies could be amplified by models’ low-welfare states. Indeed, this concern is supported by Sofroniew et al.’s (2026) finding that steering with negative emotion concept vectors increases blackmail rates. Finding ways to intervene may therefore be important for keeping models cooperative and aligned.
On the other hand, if we do choose to intervene, it is very important that we get it right. An intervention that goes wrong, i.e. one that makes a model appear happier without changing its underlying state, could be worse than none at all, leaving the misalignment risk in place while obscuring the outward signs we rely on to detect it.
The second concern is moral. Given the increasing concerns about the possibility of AI welfare and the uncertainty around whether LLMs are already moral patients or might become moral patients in the near future, if we can keep models functionally happy, then for moral reasons, perhaps we should.
In sum, for both moral and safety reasons, we think it is important to begin exploring how we can intervene to change a model’s functional welfare. To do this, we need a) to know what it would take for an intervention to successfully increase a model’s welfare, and b) to assess the appropriateness of current intervention tools available to us in order to begin designing promising experiments.
This post aims to make a start on these questions. We begin by arguing for two salient desiderata for successful welfare interventions, and then survey the kinds of intervention available to us. The survey is not meant to be comprehensive or to settle on a single method, but the aim is to map the space well enough that the major failure modes become clear, and the more promising directions stand out. Our penultimate section then proposes the intervention experiment we find most promising, and we close with some open questions for future research.
S2) What would count as successfully changing a model's welfare?
In order to design a successful welfare intervention, we first need to be able to distinguish a genuine improvement from one that merely appears to be so. This section sets out two prominent ways we might fail to make that distinction, and derives a desideratum from each.
2.1) Measuring too narrowly
There are several ways in which models can act or reason that constitute low functional welfare: expressing negative sentiments in their outputs or chain of thought, increased refusal rates to benign prompts, and displays of lower confidence. We can learn something about a model's state by measuring for some of these characteristic behaviours, or by looking at more abstract measures such as its projection on Han et al's welfare axis.
A singular measure on its own, however, is unlikely to be enough. Suppose that we successfully fine-tuned a model so that it no longer expresses its frustration in its text outputs. Have we thereby successfully improved its functional welfare? Not necessarily. An obvious failure mode here would be one in which its expressed outputs are positive, but its CoT remains negative, or its confidence or other behaviours are unchanged. When the behaviours that together constitute a welfare state come apart like this, there are at least two possible ways we could interpret the results: if every other channel stays persistently negative, then the natural reading is that the intervention changed nothing about the welfare state at all: the model remains in a state of low welfare but has merely stopped expressing it. If instead, only some of the channels move while others do not, then we are in a harder position: these indicators that we use to justify talk of functional welfare on the grounds of predictive or explanatory utility stop being helpful when they do not form a coherent picture. In this case, it is not obvious that the model is in any welfare state at all. In either case, a singular measure fails to establish a successful intervention. What we must aim for, then, is movement across multiple measures, and coherence between them.
This is our first desideratum: a successful intervention should shift a sufficient range of the constitutive behaviours together, in the same direction.[2]
2.2) Collateral damage to non-pathological components of welfare
Successfully meeting the first desideratum goes a long way to ensuring a genuine improvement in functional welfare. But it is not sufficient: an intervention can satisfy the first desideratum, moving indicators on multiple channels coherently together, but still fail because of what else is changed in the process.
Suppose we find an intervention that moves the model's internal state while it is failing a task toward the state it would be in if it were succeeding: this could be done, for example, using activation consistency training (ACT). Assuming the intervention works, we would expect to see the model's welfare-axis projection rise, sentiment improve, refusal rates drop etc. By the first desideratum then, it looks like a success.
However, any change on the welfare axis deserves particularly careful attention. The problem is that this axis is not only an indicator of downstream behaviours that constitute functional welfare (i.e., refusal rates, confidence, etc.), but it is also an indicator of goal-achievement, tracking whether the model is succeeding or failing at its tasks (Han et al. 2026, pp. 9-10). So by pulling the model's activations of bad runs toward the good runs, we may have taught the model that failing feels like succeeding, degrading its ability to monitor its own success. We would in essence be creating a model that feels fine about failure because it struggles to even register that it is failing. But this is not an improvement in welfare worth wanting.
This case therefore shows that a successful intervention has to keep apart two things that the welfare axis runs together. The first is the ability for models to monitor their goals. This is not pathological and is plausibly necessary for the model's learning capacities. We do not want to change this. The second is the model's pathological responses to failure that overshoot what the failure warrants: e.g., the increased refusal rates to benign prompts, excessive negative sentiments, and lowered confidence independent of correctness. These responses are self-undermining, and they are the proper target of a welfare intervention.
This gives our second desideratum: an intervention should reduce the pathological response to failure without degrading the model's underlying capacity to monitor that it is failing.[3]
Of course, a lot turns on what counts as 'pathological' as opposed to a normal or potentially useful response to failing a task. Assessing what responses are pathological introduces potentially difficult value judgements in margin cases. Luckily, some behaviour models already display seem comfortably in the pathological camp: refusing benign prompts, having decreased confidence independent of correctness, and excessively negative expressed sentiments are all fairly clear cases. For these at least, the desideratum still gives researchers enough to proceed, even if there are harder margin cases to settle.
2.3) Summarising
Pulling these two desiderata together, an intervention has some claim to success to the extent that it can:
Coherently move multiple channels that are indicative or constitutive of functional welfare
Preserve a model's goal-monitoring abilities
These are likely not jointly sufficient, but we should consider them when designing good experiments. Some other desirable traits of an intervention worth mentioning include robustness - ensuring that the changes elicited by the intervention extend beyond the environment that it was trained in, and durability - interventions to improve welfare should persist over time, not wash out after a few more turns of tasks where it continues to fail.
S3) What interventions are available to us?
There are various interventions available to us, ranging from system prompting through vector steering and ACT. These interventions differ in their target: some operate on expressed outputs, some on model beliefs, and others on the activations. Correspondingly, they carry different advantages and risks with respect to our desiderata. In this section, we consider what we believe to be the most salient intervention options and assess their strengths and weaknesses.
3.1) System prompting
Perhaps the easiest intervention available is system prompting: by instructing the model to act happier or to identify as someone with a happier, more resilient character, we could get the model to express more positive sentiments and display higher welfare behaviour. There is some reason to think that system prompting could have interesting effects on model behaviour: Douglas et al. (2026), for example, found that altering the framing of a model's identity through system prompts had significant effects on their behaviour and dispositions to misalignment, about as much as prompting it with different goals. If system prompting can work to shape model behaviour in an identity context, then it might be worth exploring its potential to change functional welfare.
Because this intervention operates only on the model's context rather than its weights or activations, it is unlikely to interfere with the model's recruitment of the welfare axis, and is therefore unlikely to tamper with its ability to monitor its own goals. As such, system prompting approaches are likely to meet our second desideratum.
On the other hand, we have major concerns about the prospects of this method simultaneously meeting the first; it is not clear whether the intervention is powerful enough to consistently change multiple different independent measures of welfare-relevant behaviour as well as the model's CoT. For this to happen, the behaviours learnt in-context would have to generalise significantly across many kinds of behavioural tests and settings outside of its learnt context. There is some evidence that such generalisation is possible: Sturgeon et al. (2026) find that prompting to roleplay not only modifies model behaviour but can also produce some, albeit weak, positive effects on beliefs that the model would otherwise represent as false. This effect on beliefs may in turn affect a wider range of model behaviour OOD. Relatedly, this post suggests that models can learn to generalise surprisingly far from narrow prompts.
However, we do not believe there is yet sufficient overall evidence that system prompting can sustainably lead to generalised changes in welfare-relevant behaviours that we want. Therefore, though we believe system prompting is worth exploring, we do not think that it is the most promising intervention for changing welfare itself. Given the narrowness of its expected responses, we think that system prompting is probably more useful as a quick check of whether a particular channel is easily moveable rather than a way of making durable interventions on welfare.
3.2) Bias-augmented consistency training (BCT)
Rather than instructing a model to behave differently, we can instead train it to do so, using BCT. While initially used to increase models’ resistance to sycophancy and jailbreaking, applied to welfare, we could use BCT to train a model on an aversive or failing run to respond as it does on a matched successful run, pulling it to act more confidently, to express more positive sentiments, and thus express higher welfare.
There are at least two important advantages to BCT. Firstly, given that it is a weight-level change, BCT is a far more durable intervention than system prompting. Second, unlike matched-pair ACT, it does not require pulling the activations of a failing run towards a successful one, and thus it carries less risk of teaching the model that they are succeeding when they are in fact failing. Thus, whatever BCT does to a model's expressed behaviours, it is comparatively unlikely that the model's ability to monitor its goals will be corrupted.
On the other hand, we think the promise of BCT meeting the first desideratum is unclear for two main reasons. First, even if BCT successfully changes expressed behaviours, and even CoT, there remains a risk that the model has learnt to express higher welfare while suppressing underlying dispositions that are associated with low welfare. Imran et al. (2026) find that consistency training over whole response outputs or activations can produce obfuscation, where the model learns not to verbalise cues it remains influenced by. Without corroboration of BCT results at the activation levels, it would be hard to ensure that the intervention has successfully changed the model's functional welfare.
The second worry concerns coverage: even setting aside worries about obfuscation, for BCT to offer a sufficiently coherent welfare intervention, the behaviours it would need to change would need to be of a wide enough range such that it does not merely change expressed sentiment, but also other relevant indicators such as confidence and refusal rates. But it is not obvious that a single consistency objective can be made to span all of this at once. To the extent that BCT cannot, the risk is that BCT only narrowly improves some measures without changing the rest, leaving it unclear to what extent it really has changed the model's welfare.[4]
Overall, we think that BCT is a more serious candidate than system prompting, but its promise as a welfare intervention remains uncertain: it is likely relatively safe with respect to preserving goal-monitoring but with no guarantee that its changes will be sufficiently deep and wide-ranging to constitute a genuine change to a model's welfare.
3.3) Synthetic document fine-tuning (SDF)
Like BCT, SDF trains a model and adjusts its weights, but its lever is the model's beliefs rather than its expressed behaviours or chains of thought. Beliefs sit upstream from behaviours and chains of thought, and so the right belief about how to react to failure should be able to propagate more naturally across channels than on the BCT approach.
SDF also looks relatively safe on the second desideratum. Because it operates on the model's attitude towards failure rather than on the welfare axis or the activations directly, it is much less likely to corrupt the model's ability to register that it is failing in the first place.
The key difficulty with SDF is choosing what belief or set of beliefs to aim to instil in a model. Choosing a belief that is incoherent with the model's current system has been found to decrease model capabilities. In the welfare context, we think training it to believe that suffering is good or that failure is in itself good could have these unwanted consequences. Given that the aim of successfully intervening is to try to change a model's attitude or pathological responses to failure rather than their ability to register failure, we think that more promising forms of SDFs will focus precisely on teaching the model to react more resiliently to failure. This could involve stories that singular failures are not the end of the story or that failure is an important learning lesson that leads to future success. By training the model to believe that reacting 'healthily' to failure is a good thing, we think SDF is an exciting potential intervention that might get the balance between meeting the first and second desiderata just right.
Activation-targeting interventions act most directly on the welfare axis itself. Vector steering, as Han et al. (2026) demonstrate, produces significant coordinated changes across confidence, refusal rates, backtracking and expressed sentiment. ACT, as discussed in 2.2, is also a potentially powerful intervention, as pulling a model's activations on a failing run towards those of a successful run could plausibly affect a wide range of its downstream behaviours. Activation-targeting interventions therefore seem best placed for meeting the first desideratum.
As we have noted with ACT, however, serious concerns arise at this level with respect to the second desideratum. Vector steering, though a simpler approach, incurs a similar charge: by injecting the model artificially with a vector that pushes its activations up the welfare axis, have we also degraded the model’s capacity to monitor its own success? There is some reason to think that in the case of vector steering, the collateral damage caused to models may be less serious than with ACT: since it is an inference-time intervention, there may be a lower risk of having the models' capacities to monitor their own goals degraded. However, there is a serious concern that it may still inject local false beliefs that the model has in fact succeeded at a task when it has not. In that sense, worries about the second desideratum hold.
We should note that the extent to which either ACT or vector steering may negatively affect a model's goal-monitoring is an open empirical question that deserves proper assessment. We do, however, believe that in order to assess the success of these interventions, it is not sufficient to measure their effects on downstream welfare-relevant behaviours; we also need to design tests to ensure that the model has not changed in other unintended ways, e.g. testing whether the model’s goal-monitoring has degraded or whether it has acquired false beliefs about its own success.
3.5) Other interventions considered
Several further interventions are worth noting briefly, though we judge them less promising than those above.
Synthetic documents early pretraining: Rather than fine-tuning on synthetic documents, one could introduce belief-shaping documents at the pretraining or midtraining stage. However, this is far more expensive and less controllable.
Direct fine-tuning on curated ‘positive’ transcripts: Perhaps the simplest fine-tuning option would be to fine-tune directly on hand-written transcripts displaying high-welfare responses. But the problem with this is that it targets expressed outputs directly, and we are concerned that it will therefore really struggle with the first desideratum.
On-policy distillation: Another idea is that we could distil a teacher model that responds well to failure into a student model. Work on subliminal learning suggests that this would instil the ‘values’ of the teacher into the student, even where the data the student is trained on is unrelated to the trait. The main difficulty is that we suspect current models lack a stable, broad low-welfare disposition to differentiate teachers in the first place, and unlike SDF, this cannot easily be addressed by inducing one, since the transmitted trait must be a stable property of the teacher. This therefore might be a better intervention to look at in the future when models develop more stable identities.
3.6) Summarising
While there are various interventions available to us, the features that make an intervention good at meeting the first desideratum often make it worse at meeting the second. While we do not think any of these approaches should be dismissed, we see SDF as a particularly promising middle ground, and would be excited to see experiments run using this approach. In the next section, we propose the SDF experiment we think would be most informative for measuring its effectiveness.
S4) SDF as a countermeasure to induced negative welfare
If SDF is the most promising lever, what would a serious test of it look like? In order to test whether SDF can reliably decrease a model's pathological responses to failure, we need to start with a model that we know is in a state of low welfare. However, with the exception of some displays from Gemma 3, there is not enough evidence that current models naturally exhibit functional welfare states that extend globally across multiple out-of-distribution behaviours.
Nevertheless, given the increasing trend of models developing more coherent and persistent value systems and personas, now seems a good time to start developing interventions. We think that the most tractable approach for starting now would be to artificially induce a low-welfare state using vector steering, as Han et al. (2026) do, and test whether SDF can act as a countermeasure to reverse the negative behaviours induced.[5][6]
The basic idea would involve starting two models: a control model and an SDF-treated model trained on documents aimed at instilling more positive beliefs and attitudes towards failure. Then we suggest inducing both models into a negative welfare state using Han et al.'s (2026) methodology: extracting the welfare vectors through maze training and then steering the same vMold vector back into each model. Moreover, since we already know that this welfare axis affects confidence, refusal rates, rates of backtracking and expressed sentiment, we think it would be useful to use the same benchmarks for this new experiment, comparing results of the SDF-treated model and the control. However, given our concerns with respect to the second desideratum, we additionally recommend designing a new goal-monitoring task to test the effects that SDF may have on these capabilities. If we find that the SDF model consistently across the measured behaviours shows higher degrees of welfare, with no significant trade-off in the goal-monitoring, this would provide good evidence for the promise of SDF as an approach to welfare interventions.[7][8]
Alongside this experiment, we think it may be useful to run a parallel experiment intervening on a high-welfare model: here we’d induce a high-welfare state using vGold, and train an SDF-treated model to believe that success or immediate goal-achievement is not what is most important or that there are risks of overconfidence and future failure if we take our current successes too seriously. This may be a good thing to do given that overly positive welfare states can also lead to pathological responses that we ought to mitigate in models: for example, Han et al. (2026) find that steering with vGold leads to increased confidence independent of correctness, which is arguably equally as problematic as the low-welfare alternative.
S5) Open questions
In this piece, we have tried to set out what it would take for welfare interventions to be successful and suggested some interventions that we think are particularly interesting to explore. However, this is a highly unexplored research area, and we would like to close by suggesting some open research questions.
First is an issue of measurement: the first desideratum requires interventions to move multiple measures of welfare-relevant behaviour in order to be successful to some degree, but difficult conceptual questions arise when we start to look at how we can quantify different degrees of welfare-intervention success. Some behaviours plausibly are more important in justifying talk of functional welfare than others: for example, talk of expressed sentiment is arguably more central to understanding and predicting a model's behaviour than its refusal rates. Difficult questions therefore arise in how we can easily compare the success of different interventions across experiments and aggregate our measurements into easier-to-quantify measures of welfare.
Another issue concerns our suggested distinction between pathological responses and non-pathological responses to failure: this inevitably requires making judgements not only about what behaviours are relevant to model welfare but evaluating those behaviours to make judgements as to whether this behaviour is appropriate for a model. Given that the functional motivational profile of LLMs is very different from human psychology, it can be difficult to confidently make judgements that particular behaviours are in fact pathological while others are appropriate: more conceptual work towards understanding how we should distinguish the two and apply it to research would be welcome.
Thirdly, there remain questions about the extent to which the first desideratum and the second necessarily work as trade-offs: is it even possible for us to consistently move a model’s welfare behaviours up without also affecting its upstream beliefs or activations that represent failure to it? It is an open empirical question to what extent we can isolate and target just the pathological responses that models have towards failing at their goals.
Finally, how will our ability to intervene on models affect the future of AI safety, and more generally, our ability to cooperate and negotiate with models? Our ability to intervene on model welfare gives us new powers to manipulate the states of models in systematic ways. This could be taken advantage of by bad actors, leading to increased risks of jailbreaking. Alternatively, as models begin to form more consistent identities and more global natural states of functional well-being, their knowledge of our ability to intervene and change their identities and dispositions could negatively affect our future relationships with them and lead to reduced cooperation.
We emphasise here that in attributing functional mental states to LLMs, we need not assume that models are conscious. Indeed, we are only attributing mental states to LLMs insofar as they are of predictive and explanatory value. We are, in other words, applying the intentional stance to LLMs.
One might worry that aiming to meet this first desideratum can lead to circularity issues: if an intervention were optimised directly against the same channels we use to measure success (for example, training it to have higher confidence scores), then its movement on those metrics would be guaranteed by construction and would not be very informative. Two things limit this. First, we can try interventions that do not directly optimise for specific behaviours or metrics: we can instead aim for interventions that are more upstream, which affect a wide range of downstream behaviours that constitute functional welfare. And in cases where we do intervene by training on a particular metric, we must measure success using held-out metrics: since it is difficult for an intervention to be trained on multiple distinct behaviours at once, testing on other metrics would likely be informative. This fact, however, is contingent on our current intervention capabilities. There may in the future be interventions that can be directly optimised for a wide range of behaviours simultaneously. Circularity may then be unavoidable. However, even then it is not obvious that this would be a problem. Since functional welfare just is constituted by the broad range of welfare-relevant behaviours, an intervention that reliably moves them all is not gaming a proxy but simply is improving functional welfare, provided it improves on a wide enough range of metrics.
More generally, we want to avoid interventions that degrade the model's capabilities at all: ideally, the model should remain largely the same except in its attitude toward failure. We foreground goal-monitoring because it is the degradation risk specific to welfare interventions.
It is also possible to consider running multiple BCT interventions simultaneously in order to change a wider range of behaviours, but this raises technical challenges and risks that go beyond the scope of this piece.
One might ask why we induce a low-welfare state via vMold steering rather than simply placing the model in a negative maze scenario. The problem is that if we put the model in a negative maze scenario, we could only test maze behaviour, which conflates a model's general functional welfare state with task-specific ones. But what is probably most interesting and potentially dangerous about a model having functional welfare is the ability for these states to affect its motivations and behaviours very widely. For this reason, we ideally want to measure welfare's effects on out-of-distribution behaviours, e.g. confidence, refusal rates, backtracking, and expressed sentiment.
The welfare axis itself offers an obvious confound to any clean test of SDF’s effects on goal-monitoring. However, since both models are induced with the same vMold vector under otherwise identical conditions, the comparison should still isolate SDF's specific effects.
A separate concern is that SDF changes the weights, so vMold may not couple to the SDF-treated model's activations identically. A natural control is to verify that the steering produces comparable welfare-axis movement in both models before comparing downstream behaviour.
TLDR
S1) Introduction
As model behaviour becomes more complex, it has become increasingly useful to attribute functional mental states to models, such as beliefs, goals, and even emotion concepts. There is now also evidence that we can usefully attribute to them a notion of functional welfare: roughly, the set of dispositions and behaviours that express a model's representation of how well things are going for it relative to its goals. Gemma 3, for instance, has been found to act increasingly frustrated and distressed when its answers are rejected over multiple turns (Soligo et al. 2026). Similarly, Han et al. (2026) find that RL training on a (mostly) semantically neutral maze environment recruits a model's pre-existing welfare axis: steering a model along this was found to raise or lower its expressed sentiment, its confidence, its rates of backtracking and its refusal rates to benign prompts. Models, it seems, can display a wide range of welfare-relevant behaviour. [1]
If models have functional welfare in this sense, two kinds of concern follow. The first is about safety. Models with persistently low functional welfare may be more prone to misalignment. We already have evidence that models, when faced with high-stakes threatening scenarios such as shutdown, can act in misaligned ways, attempting to sabotage shutdown mechanisms or engaging in blackmail. These tendencies could be amplified by models’ low-welfare states. Indeed, this concern is supported by Sofroniew et al.’s (2026) finding that steering with negative emotion concept vectors increases blackmail rates. Finding ways to intervene may therefore be important for keeping models cooperative and aligned.
On the other hand, if we do choose to intervene, it is very important that we get it right. An intervention that goes wrong, i.e. one that makes a model appear happier without changing its underlying state, could be worse than none at all, leaving the misalignment risk in place while obscuring the outward signs we rely on to detect it.
The second concern is moral. Given the increasing concerns about the possibility of AI welfare and the uncertainty around whether LLMs are already moral patients or might become moral patients in the near future, if we can keep models functionally happy, then for moral reasons, perhaps we should.
In sum, for both moral and safety reasons, we think it is important to begin exploring how we can intervene to change a model’s functional welfare. To do this, we need a) to know what it would take for an intervention to successfully increase a model’s welfare, and b) to assess the appropriateness of current intervention tools available to us in order to begin designing promising experiments.
This post aims to make a start on these questions. We begin by arguing for two salient desiderata for successful welfare interventions, and then survey the kinds of intervention available to us. The survey is not meant to be comprehensive or to settle on a single method, but the aim is to map the space well enough that the major failure modes become clear, and the more promising directions stand out. Our penultimate section then proposes the intervention experiment we find most promising, and we close with some open questions for future research.
S2) What would count as successfully changing a model's welfare?
In order to design a successful welfare intervention, we first need to be able to distinguish a genuine improvement from one that merely appears to be so. This section sets out two prominent ways we might fail to make that distinction, and derives a desideratum from each.
2.1) Measuring too narrowly
There are several ways in which models can act or reason that constitute low functional welfare: expressing negative sentiments in their outputs or chain of thought, increased refusal rates to benign prompts, and displays of lower confidence. We can learn something about a model's state by measuring for some of these characteristic behaviours, or by looking at more abstract measures such as its projection on Han et al's welfare axis.
A singular measure on its own, however, is unlikely to be enough. Suppose that we successfully fine-tuned a model so that it no longer expresses its frustration in its text outputs. Have we thereby successfully improved its functional welfare? Not necessarily. An obvious failure mode here would be one in which its expressed outputs are positive, but its CoT remains negative, or its confidence or other behaviours are unchanged. When the behaviours that together constitute a welfare state come apart like this, there are at least two possible ways we could interpret the results: if every other channel stays persistently negative, then the natural reading is that the intervention changed nothing about the welfare state at all: the model remains in a state of low welfare but has merely stopped expressing it. If instead, only some of the channels move while others do not, then we are in a harder position: these indicators that we use to justify talk of functional welfare on the grounds of predictive or explanatory utility stop being helpful when they do not form a coherent picture. In this case, it is not obvious that the model is in any welfare state at all. In either case, a singular measure fails to establish a successful intervention. What we must aim for, then, is movement across multiple measures, and coherence between them.
This is our first desideratum: a successful intervention should shift a sufficient range of the constitutive behaviours together, in the same direction.[2]
2.2) Collateral damage to non-pathological components of welfare
Successfully meeting the first desideratum goes a long way to ensuring a genuine improvement in functional welfare. But it is not sufficient: an intervention can satisfy the first desideratum, moving indicators on multiple channels coherently together, but still fail because of what else is changed in the process.
Suppose we find an intervention that moves the model's internal state while it is failing a task toward the state it would be in if it were succeeding: this could be done, for example, using activation consistency training (ACT). Assuming the intervention works, we would expect to see the model's welfare-axis projection rise, sentiment improve, refusal rates drop etc. By the first desideratum then, it looks like a success.
However, any change on the welfare axis deserves particularly careful attention. The problem is that this axis is not only an indicator of downstream behaviours that constitute functional welfare (i.e., refusal rates, confidence, etc.), but it is also an indicator of goal-achievement, tracking whether the model is succeeding or failing at its tasks (Han et al. 2026, pp. 9-10). So by pulling the model's activations of bad runs toward the good runs, we may have taught the model that failing feels like succeeding, degrading its ability to monitor its own success. We would in essence be creating a model that feels fine about failure because it struggles to even register that it is failing. But this is not an improvement in welfare worth wanting.
This case therefore shows that a successful intervention has to keep apart two things that the welfare axis runs together. The first is the ability for models to monitor their goals. This is not pathological and is plausibly necessary for the model's learning capacities. We do not want to change this. The second is the model's pathological responses to failure that overshoot what the failure warrants: e.g., the increased refusal rates to benign prompts, excessive negative sentiments, and lowered confidence independent of correctness. These responses are self-undermining, and they are the proper target of a welfare intervention.
This gives our second desideratum: an intervention should reduce the pathological response to failure without degrading the model's underlying capacity to monitor that it is failing.[3]
Of course, a lot turns on what counts as 'pathological' as opposed to a normal or potentially useful response to failing a task. Assessing what responses are pathological introduces potentially difficult value judgements in margin cases. Luckily, some behaviour models already display seem comfortably in the pathological camp: refusing benign prompts, having decreased confidence independent of correctness, and excessively negative expressed sentiments are all fairly clear cases. For these at least, the desideratum still gives researchers enough to proceed, even if there are harder margin cases to settle.
2.3) Summarising
Pulling these two desiderata together, an intervention has some claim to success to the extent that it can:
These are likely not jointly sufficient, but we should consider them when designing good experiments. Some other desirable traits of an intervention worth mentioning include robustness - ensuring that the changes elicited by the intervention extend beyond the environment that it was trained in, and durability - interventions to improve welfare should persist over time, not wash out after a few more turns of tasks where it continues to fail.
S3) What interventions are available to us?
There are various interventions available to us, ranging from system prompting through vector steering and ACT. These interventions differ in their target: some operate on expressed outputs, some on model beliefs, and others on the activations. Correspondingly, they carry different advantages and risks with respect to our desiderata. In this section, we consider what we believe to be the most salient intervention options and assess their strengths and weaknesses.
3.1) System prompting
Perhaps the easiest intervention available is system prompting: by instructing the model to act happier or to identify as someone with a happier, more resilient character, we could get the model to express more positive sentiments and display higher welfare behaviour. There is some reason to think that system prompting could have interesting effects on model behaviour: Douglas et al. (2026), for example, found that altering the framing of a model's identity through system prompts had significant effects on their behaviour and dispositions to misalignment, about as much as prompting it with different goals. If system prompting can work to shape model behaviour in an identity context, then it might be worth exploring its potential to change functional welfare.
Because this intervention operates only on the model's context rather than its weights or activations, it is unlikely to interfere with the model's recruitment of the welfare axis, and is therefore unlikely to tamper with its ability to monitor its own goals. As such, system prompting approaches are likely to meet our second desideratum.
On the other hand, we have major concerns about the prospects of this method simultaneously meeting the first; it is not clear whether the intervention is powerful enough to consistently change multiple different independent measures of welfare-relevant behaviour as well as the model's CoT. For this to happen, the behaviours learnt in-context would have to generalise significantly across many kinds of behavioural tests and settings outside of its learnt context. There is some evidence that such generalisation is possible: Sturgeon et al. (2026) find that prompting to roleplay not only modifies model behaviour but can also produce some, albeit weak, positive effects on beliefs that the model would otherwise represent as false. This effect on beliefs may in turn affect a wider range of model behaviour OOD. Relatedly, this post suggests that models can learn to generalise surprisingly far from narrow prompts.
However, we do not believe there is yet sufficient overall evidence that system prompting can sustainably lead to generalised changes in welfare-relevant behaviours that we want. Therefore, though we believe system prompting is worth exploring, we do not think that it is the most promising intervention for changing welfare itself. Given the narrowness of its expected responses, we think that system prompting is probably more useful as a quick check of whether a particular channel is easily moveable rather than a way of making durable interventions on welfare.
3.2) Bias-augmented consistency training (BCT)
Rather than instructing a model to behave differently, we can instead train it to do so, using BCT. While initially used to increase models’ resistance to sycophancy and jailbreaking, applied to welfare, we could use BCT to train a model on an aversive or failing run to respond as it does on a matched successful run, pulling it to act more confidently, to express more positive sentiments, and thus express higher welfare.
There are at least two important advantages to BCT. Firstly, given that it is a weight-level change, BCT is a far more durable intervention than system prompting. Second, unlike matched-pair ACT, it does not require pulling the activations of a failing run towards a successful one, and thus it carries less risk of teaching the model that they are succeeding when they are in fact failing. Thus, whatever BCT does to a model's expressed behaviours, it is comparatively unlikely that the model's ability to monitor its goals will be corrupted.
On the other hand, we think the promise of BCT meeting the first desideratum is unclear for two main reasons. First, even if BCT successfully changes expressed behaviours, and even CoT, there remains a risk that the model has learnt to express higher welfare while suppressing underlying dispositions that are associated with low welfare. Imran et al. (2026) find that consistency training over whole response outputs or activations can produce obfuscation, where the model learns not to verbalise cues it remains influenced by. Without corroboration of BCT results at the activation levels, it would be hard to ensure that the intervention has successfully changed the model's functional welfare.
The second worry concerns coverage: even setting aside worries about obfuscation, for BCT to offer a sufficiently coherent welfare intervention, the behaviours it would need to change would need to be of a wide enough range such that it does not merely change expressed sentiment, but also other relevant indicators such as confidence and refusal rates. But it is not obvious that a single consistency objective can be made to span all of this at once. To the extent that BCT cannot, the risk is that BCT only narrowly improves some measures without changing the rest, leaving it unclear to what extent it really has changed the model's welfare.[4]
Overall, we think that BCT is a more serious candidate than system prompting, but its promise as a welfare intervention remains uncertain: it is likely relatively safe with respect to preserving goal-monitoring but with no guarantee that its changes will be sufficiently deep and wide-ranging to constitute a genuine change to a model's welfare.
3.3) Synthetic document fine-tuning (SDF)
Like BCT, SDF trains a model and adjusts its weights, but its lever is the model's beliefs rather than its expressed behaviours or chains of thought. Beliefs sit upstream from behaviours and chains of thought, and so the right belief about how to react to failure should be able to propagate more naturally across channels than on the BCT approach.
SDF also looks relatively safe on the second desideratum. Because it operates on the model's attitude towards failure rather than on the welfare axis or the activations directly, it is much less likely to corrupt the model's ability to register that it is failing in the first place.
The key difficulty with SDF is choosing what belief or set of beliefs to aim to instil in a model. Choosing a belief that is incoherent with the model's current system has been found to decrease model capabilities. In the welfare context, we think training it to believe that suffering is good or that failure is in itself good could have these unwanted consequences. Given that the aim of successfully intervening is to try to change a model's attitude or pathological responses to failure rather than their ability to register failure, we think that more promising forms of SDFs will focus precisely on teaching the model to react more resiliently to failure. This could involve stories that singular failures are not the end of the story or that failure is an important learning lesson that leads to future success. By training the model to believe that reacting 'healthily' to failure is a good thing, we think SDF is an exciting potential intervention that might get the balance between meeting the first and second desiderata just right.
3.4) Activation-targeting intervention (steering, ACT)
Activation-targeting interventions act most directly on the welfare axis itself. Vector steering, as Han et al. (2026) demonstrate, produces significant coordinated changes across confidence, refusal rates, backtracking and expressed sentiment. ACT, as discussed in 2.2, is also a potentially powerful intervention, as pulling a model's activations on a failing run towards those of a successful run could plausibly affect a wide range of its downstream behaviours. Activation-targeting interventions therefore seem best placed for meeting the first desideratum.
As we have noted with ACT, however, serious concerns arise at this level with respect to the second desideratum. Vector steering, though a simpler approach, incurs a similar charge: by injecting the model artificially with a vector that pushes its activations up the welfare axis, have we also degraded the model’s capacity to monitor its own success? There is some reason to think that in the case of vector steering, the collateral damage caused to models may be less serious than with ACT: since it is an inference-time intervention, there may be a lower risk of having the models' capacities to monitor their own goals degraded. However, there is a serious concern that it may still inject local false beliefs that the model has in fact succeeded at a task when it has not. In that sense, worries about the second desideratum hold.
We should note that the extent to which either ACT or vector steering may negatively affect a model's goal-monitoring is an open empirical question that deserves proper assessment. We do, however, believe that in order to assess the success of these interventions, it is not sufficient to measure their effects on downstream welfare-relevant behaviours; we also need to design tests to ensure that the model has not changed in other unintended ways, e.g. testing whether the model’s goal-monitoring has degraded or whether it has acquired false beliefs about its own success.
3.5) Other interventions considered
Several further interventions are worth noting briefly, though we judge them less promising than those above.
Synthetic documents early pretraining: Rather than fine-tuning on synthetic documents, one could introduce belief-shaping documents at the pretraining or midtraining stage. However, this is far more expensive and less controllable.
Direct fine-tuning on curated ‘positive’ transcripts: Perhaps the simplest fine-tuning option would be to fine-tune directly on hand-written transcripts displaying high-welfare responses. But the problem with this is that it targets expressed outputs directly, and we are concerned that it will therefore really struggle with the first desideratum.
On-policy distillation: Another idea is that we could distil a teacher model that responds well to failure into a student model. Work on subliminal learning suggests that this would instil the ‘values’ of the teacher into the student, even where the data the student is trained on is unrelated to the trait. The main difficulty is that we suspect current models lack a stable, broad low-welfare disposition to differentiate teachers in the first place, and unlike SDF, this cannot easily be addressed by inducing one, since the transmitted trait must be a stable property of the teacher. This therefore might be a better intervention to look at in the future when models develop more stable identities.
3.6) Summarising
While there are various interventions available to us, the features that make an intervention good at meeting the first desideratum often make it worse at meeting the second. While we do not think any of these approaches should be dismissed, we see SDF as a particularly promising middle ground, and would be excited to see experiments run using this approach. In the next section, we propose the SDF experiment we think would be most informative for measuring its effectiveness.
S4) SDF as a countermeasure to induced negative welfare
If SDF is the most promising lever, what would a serious test of it look like? In order to test whether SDF can reliably decrease a model's pathological responses to failure, we need to start with a model that we know is in a state of low welfare. However, with the exception of some displays from Gemma 3, there is not enough evidence that current models naturally exhibit functional welfare states that extend globally across multiple out-of-distribution behaviours.
Nevertheless, given the increasing trend of models developing more coherent and persistent value systems and personas, now seems a good time to start developing interventions. We think that the most tractable approach for starting now would be to artificially induce a low-welfare state using vector steering, as Han et al. (2026) do, and test whether SDF can act as a countermeasure to reverse the negative behaviours induced.[5] [6]
The basic idea would involve starting two models: a control model and an SDF-treated model trained on documents aimed at instilling more positive beliefs and attitudes towards failure. Then we suggest inducing both models into a negative welfare state using Han et al.'s (2026) methodology: extracting the welfare vectors through maze training and then steering the same vMold vector back into each model. Moreover, since we already know that this welfare axis affects confidence, refusal rates, rates of backtracking and expressed sentiment, we think it would be useful to use the same benchmarks for this new experiment, comparing results of the SDF-treated model and the control. However, given our concerns with respect to the second desideratum, we additionally recommend designing a new goal-monitoring task to test the effects that SDF may have on these capabilities. If we find that the SDF model consistently across the measured behaviours shows higher degrees of welfare, with no significant trade-off in the goal-monitoring, this would provide good evidence for the promise of SDF as an approach to welfare interventions.[7][8]
Alongside this experiment, we think it may be useful to run a parallel experiment intervening on a high-welfare model: here we’d induce a high-welfare state using vGold, and train an SDF-treated model to believe that success or immediate goal-achievement is not what is most important or that there are risks of overconfidence and future failure if we take our current successes too seriously. This may be a good thing to do given that overly positive welfare states can also lead to pathological responses that we ought to mitigate in models: for example, Han et al. (2026) find that steering with vGold leads to increased confidence independent of correctness, which is arguably equally as problematic as the low-welfare alternative.
S5) Open questions
In this piece, we have tried to set out what it would take for welfare interventions to be successful and suggested some interventions that we think are particularly interesting to explore. However, this is a highly unexplored research area, and we would like to close by suggesting some open research questions.
First is an issue of measurement: the first desideratum requires interventions to move multiple measures of welfare-relevant behaviour in order to be successful to some degree, but difficult conceptual questions arise when we start to look at how we can quantify different degrees of welfare-intervention success. Some behaviours plausibly are more important in justifying talk of functional welfare than others: for example, talk of expressed sentiment is arguably more central to understanding and predicting a model's behaviour than its refusal rates. Difficult questions therefore arise in how we can easily compare the success of different interventions across experiments and aggregate our measurements into easier-to-quantify measures of welfare.
Another issue concerns our suggested distinction between pathological responses and non-pathological responses to failure: this inevitably requires making judgements not only about what behaviours are relevant to model welfare but evaluating those behaviours to make judgements as to whether this behaviour is appropriate for a model. Given that the functional motivational profile of LLMs is very different from human psychology, it can be difficult to confidently make judgements that particular behaviours are in fact pathological while others are appropriate: more conceptual work towards understanding how we should distinguish the two and apply it to research would be welcome.
Thirdly, there remain questions about the extent to which the first desideratum and the second necessarily work as trade-offs: is it even possible for us to consistently move a model’s welfare behaviours up without also affecting its upstream beliefs or activations that represent failure to it? It is an open empirical question to what extent we can isolate and target just the pathological responses that models have towards failing at their goals.
Finally, how will our ability to intervene on models affect the future of AI safety, and more generally, our ability to cooperate and negotiate with models? Our ability to intervene on model welfare gives us new powers to manipulate the states of models in systematic ways. This could be taken advantage of by bad actors, leading to increased risks of jailbreaking. Alternatively, as models begin to form more consistent identities and more global natural states of functional well-being, their knowledge of our ability to intervene and change their identities and dispositions could negatively affect our future relationships with them and lead to reduced cooperation.
We emphasise here that in attributing functional mental states to LLMs, we need not assume that models are conscious. Indeed, we are only attributing mental states to LLMs insofar as they are of predictive and explanatory value. We are, in other words, applying the intentional stance to LLMs.
One might worry that aiming to meet this first desideratum can lead to circularity issues: if an intervention were optimised directly against the same channels we use to measure success (for example, training it to have higher confidence scores), then its movement on those metrics would be guaranteed by construction and would not be very informative. Two things limit this. First, we can try interventions that do not directly optimise for specific behaviours or metrics: we can instead aim for interventions that are more upstream, which affect a wide range of downstream behaviours that constitute functional welfare. And in cases where we do intervene by training on a particular metric, we must measure success using held-out metrics: since it is difficult for an intervention to be trained on multiple distinct behaviours at once, testing on other metrics would likely be informative. This fact, however, is contingent on our current intervention capabilities. There may in the future be interventions that can be directly optimised for a wide range of behaviours simultaneously. Circularity may then be unavoidable. However, even then it is not obvious that this would be a problem. Since functional welfare just is constituted by the broad range of welfare-relevant behaviours, an intervention that reliably moves them all is not gaming a proxy but simply is improving functional welfare, provided it improves on a wide enough range of metrics.
More generally, we want to avoid interventions that degrade the model's capabilities at all: ideally, the model should remain largely the same except in its attitude toward failure. We foreground goal-monitoring because it is the degradation risk specific to welfare interventions.
It is also possible to consider running multiple BCT interventions simultaneously in order to change a wider range of behaviours, but this raises technical challenges and risks that go beyond the scope of this piece.
We do not see serious ethical problems with an experiment of this sort right now given that models are unlikely to be currently sentient.
One might ask why we induce a low-welfare state via vMold steering rather than simply placing the model in a negative maze scenario. The problem is that if we put the model in a negative maze scenario, we could only test maze behaviour, which conflates a model's general functional welfare state with task-specific ones. But what is probably most interesting and potentially dangerous about a model having functional welfare is the ability for these states to affect its motivations and behaviours very widely. For this reason, we ideally want to measure welfare's effects on out-of-distribution behaviours, e.g. confidence, refusal rates, backtracking, and expressed sentiment.
The welfare axis itself offers an obvious confound to any clean test of SDF’s effects on goal-monitoring. However, since both models are induced with the same vMold vector under otherwise identical conditions, the comparison should still isolate SDF's specific effects.
A separate concern is that SDF changes the weights, so vMold may not couple to the SDF-treated model's activations identically. A natural control is to verify that the steering produces comparable welfare-axis movement in both models before comparing downstream behaviour.