It is widely known that cognitive behavioural therapy (CBT) is a standard treatment for depression. A little-known fact, however, is that its subcomponent, behavioural therapy (BT), is equally effective[1], according to a meta-analysis by Cochrane[2].
What the hell is that though?
If the term “reinforcement” makes you think of Sutton & Barto, or the newest shiny model release, then, ugh, fine, I’ll explain it in their terms:
Sequences of actions are graded according to a reward function. For example, DeepSeek spits out sequences of tokens, called chains-of-thought (CoT), and sequences that yield a correct answer are reinforced over those that do not.
Anything that is positively associated with getting the answer right, with getting the reward, such as correctly multiplying 322 by 214, is called a secondary reinforcer.
Behavioural therapy (BT) works by making effort a secondary reinforcer of your actions. That’s it.
To get started, you need morale[3], that is, you need to believe that effortful actions are associated with reward. If your level of morale is very low, you have come to believe there are no such actions, and taking very ambitious actions that then fail will only reinforce that belief[4]. Instead, morale needs to be raised in an incremental fashion. That is, you make a plan that involves effort, that you can achieve, and that gives you reward.
If you are unsure what gives humans reward, please consult Maslow’s Hierarchy of Needs. Surely things like “having sex” or “gaining status” may not sound quite as idealistic as “making the singularity go well” to you.
But again, we can reach for the power of association: Say there is a subculture with the aim of “making the singularity go well”, then “gaining status” may bring you closer to your terminal goal than initially assumed.
In fact, this principle extends way beyond any rigid DSM-5 definition[5], to all human behaviour. I posit the existence of two attractor states, existing alongside the middle ground of normalcy:
A laziness spiral[6], where one takes incrementally less effortful actions over time, which lowers morale recursively
An effort spiral[7], where one takes incrementally more effortful actions over time, which increases morale recursively
Any behaviour has a set of cues associated with it, e.g. a cue for DeepSeek is ‘322 × 214’. The environment is the set of all cues. Therefore, to change behaviour, a maximally disjoint environment needs to be created, whose cues must secondarily reinforce effort.
During the Manhattan Project, the Los Alamos Laboratory reliably created a strong effort spiral in its subjects, with ~75-hour work-weeks being common. Finding a modern equivalent is left as an exercise for the reader. Hint: Try to farm LessWrong karma, or, idk, do this “cold email” thing.
Finding a healthy, sustainable place on that spectrum, and avoiding the extremeties[8] associated with both attractors is by no means an easy task. It can be reduced to finding the correct amount of slack in one's life, and as much smarter ink has been spilled on this already[9], I invite you to find your own optimum.
Furthermore, similar to how once evolution ingrained the need for high-calorie food in humans to optimize inclusive genetic fitness, once effort becomes habitual, you can leverage inner misalignment, that is, you may no longer, and should no longer, only aim for e.g. status, you can exert effort towards more meaningful ends.
In other words, the secondary reinforcer is a proxy for reward, and this proxy can be manipulated towards one's own goals.
Unfortunately, other factors also play a significant role. For instance, we know executive function[10] mediates the degree of effort one puts forth. The trait involves the ability to self-regulate and is highly heritable. The most effective intervention, stimulants, suffers from unclear long-term effectiveness[11].
Another slap in the face is that interventional negative reinforcement (whose exertion by other people is called dominance) may be useful[12]. Fortunately, its sting may fade out in aggregation, that is, thanks to the hedonic treadmill.
A tiny bit of interventional negative reinforcement by others is both ethical (don't slap people!) and effective. A slight facial twitch signalling disapproval, or a downvote on a forum are good examples. This surgeon's knife of behaviourism[13] is, however, available only for use by other people, and thus many alignment problems find their origin here. In the behaviouristic worldview, negative reinforcement by the self is oxymoronic, and I want to strongly discourage from it.
All this leaves us with, well, my bad attempts at poetry:
At which point it is also prudent to clarify that this article does not aim to engage with the many subtleties of ‘depression’ as defined by the DSM-5.
Which John B. Watson, the father of behaviourism, had to of course use liberally by making a little child, Albert, afraid of a white rat, for science or something, you know? cf. Little Albert Experiment
It is widely known that cognitive behavioural therapy (CBT) is a standard treatment for depression. A little-known fact, however, is that its subcomponent, behavioural therapy (BT), is equally effective[1], according to a meta-analysis by Cochrane[2].
What the hell is that though?
If the term “reinforcement” makes you think of Sutton & Barto, or the newest shiny model release, then, ugh, fine, I’ll explain it in their terms:
Sequences of actions are graded according to a reward function. For example, DeepSeek spits out sequences of tokens, called chains-of-thought (CoT), and sequences that yield a correct answer are reinforced over those that do not.
Anything that is positively associated with getting the answer right, with getting the reward, such as correctly multiplying 322 by 214, is called a secondary reinforcer.
Behavioural therapy (BT) works by making effort a secondary reinforcer of your actions. That’s it.
To get started, you need morale[3], that is, you need to believe that effortful actions are associated with reward. If your level of morale is very low, you have come to believe there are no such actions, and taking very ambitious actions that then fail will only reinforce that belief[4]. Instead, morale needs to be raised in an incremental fashion. That is, you make a plan that involves effort, that you can achieve, and that gives you reward.
If you are unsure what gives humans reward, please consult Maslow’s Hierarchy of Needs. Surely things like “having sex” or “gaining status” may not sound quite as idealistic as “making the singularity go well” to you.
But again, we can reach for the power of association: Say there is a subculture with the aim of “making the singularity go well”, then “gaining status” may bring you closer to your terminal goal than initially assumed.
In fact, this principle extends way beyond any rigid DSM-5 definition[5], to all human behaviour. I posit the existence of two attractor states, existing alongside the middle ground of normalcy:
Any behaviour has a set of cues associated with it, e.g. a cue for DeepSeek is ‘322 × 214’. The environment is the set of all cues. Therefore, to change behaviour, a maximally disjoint environment needs to be created, whose cues must secondarily reinforce effort.
During the Manhattan Project, the Los Alamos Laboratory reliably created a strong effort spiral in its subjects, with ~75-hour work-weeks being common. Finding a modern equivalent is left as an exercise for the reader. Hint: Try to farm LessWrong karma, or, idk, do this “cold email” thing.
Finding a healthy, sustainable place on that spectrum, and avoiding the extremeties[8] associated with both attractors is by no means an easy task. It can be reduced to finding the correct amount of slack in one's life, and as much smarter ink has been spilled on this already[9], I invite you to find your own optimum.
Furthermore, similar to how once evolution ingrained the need for high-calorie food in humans to optimize inclusive genetic fitness, once effort becomes habitual, you can leverage inner misalignment, that is, you may no longer, and should no longer, only aim for e.g. status, you can exert effort towards more meaningful ends.
In other words, the secondary reinforcer is a proxy for reward, and this proxy can be manipulated towards one's own goals.
Unfortunately, other factors also play a significant role. For instance, we know executive function[10] mediates the degree of effort one puts forth. The trait involves the ability to self-regulate and is highly heritable. The most effective intervention, stimulants, suffers from unclear long-term effectiveness[11].
Another slap in the face is that interventional negative reinforcement (whose exertion by other people is called dominance) may be useful[12]. Fortunately, its sting may fade out in aggregation, that is, thanks to the hedonic treadmill.
A tiny bit of interventional negative reinforcement by others is both ethical (don't slap people!) and effective. A slight facial twitch signalling disapproval, or a downvote on a forum are good examples. This surgeon's knife of behaviourism[13] is, however, available only for use by other people, and thus many alignment problems find their origin here. In the behaviouristic worldview, negative reinforcement by the self is oxymoronic, and I want to strongly discourage from it.
All this leaves us with, well, my bad attempts at poetry:
The road from akrasia? — Well, it’s plain
and simple to express:
Slack
and slack
and slack again
but less
and less
and less[14].
The pharmacology people sometimes talk about “active ingredients” here, idk.
If the name “Cochrane” is unknown to you, just nod along approvingly, because they are very high-status wizards of meta-analytic black magic. cf. Behavioural therapies versus other psychological therapies for depression, Cochrane Database of Systematic Reviews
cf. Morale, J Bostock, LessWrong
A well-known psychiatrist says this is a common failure mode he experiences, so you better believe it, m’kay? cf. Peer Review Request: Depression, S. Alexander, AstralCodexTen, Point 2.1.5
At which point it is also prudent to clarify that this article does not aim to engage with the many subtleties of ‘depression’ as defined by the DSM-5.
cf. Laziness death spirals, PatrickDFarley, LessWrong
cf. Learned Industriousness, R. Eisenberger
For example, Oppenheimer dropped a substantial amount of weight due to working so much.
See for example On Slack, or Zvi's longer considerations of this.
cf. Executive Functions: What They Are, How They Work, and Why They Evolved, R.A. Barkley
Which is the subject of, and a shameless ad for, my last article
cf. Dominance: The Standard Everyday Solution To Akrasia, johnswentworth, LessWrong
Which John B. Watson, the father of behaviourism, had to of course use liberally by making a little child, Albert, afraid of a white rat, for science or something, you know? cf. Little Albert Experiment
cf. LessWrong, 2012-present