[This post is the second post in my thirty day writing challenge. Tune in for more if you enjoy it!]
If you reliably knew that you would make a bad decision during a certain time, would you let yourself make decisions during that time? Probably not, right? So why do you let yourself grocery shop when hungry, speak negatively when you have low blood sugar, and make key decisions when anxious?
A core epistemic skill that I’ve tried to cultivate over the years is the idea of knowing when I’m stupid, and not letting myself make decisions during these times. Like all core epistemic skills, I’ve cultivated this in response to getting repeatedly smacked across the face over and over again.
I.
Cindy is an anxious woman. Her resting heart rate is a bit high, she gets wigged out easily, and she tends to run an endless stream of disaster loops through her head at all hours of the day. Like all anxious people, her anxiety rises and falls. Maybe it rises when her blood sugar is low, when she’s under-slept, or when other things in her life are particularly stressful. Probably all three.
She, like all people, has a number of decisions to make throughout the day. Some of these decisions are big and some of them are small. Cindy reliably finds that when she’s flooded with anxiety, she repeatedly runs into certain failure modes. For example, maybe she makes elaborate plans to fix every single problem in her entire life, half-commits to it for, like, one day, and then decides that: no, what was she thinking? That plan is terrible! It’s time for another ground-breaking, earth-shaking plan that will fix every single problem in her entire life. She then half-commits to this plan, and then one day later comes to a terrible realization....
Cindy notices that this kind of planning loop stops her from making any actual progress. It’s like she’s in a car and wants to go in a certain direction, but instead she goes into a hard right turn, and then a hard left turn, and then a hard right turn, and then a hard... you get the idea. If Cindy just stuck with a plan, even if it was kind of mediocre, then she would be able to make some sort of progress over time. It’s similar to how a random walk has an expected displacement of zero, but a directed random walk has an expected displacement that grows like n, where n is the number of steps.
Since Cindy notices that she reliably gets the extremely strong urge to stop literally everything she’s doing and make a Plan to Rule All Plans when she feels very anxious, it would be a good idea for Cindy to ban herself from planning when she feels this way—to make a deontological bar[1] against planning when anxious. This would likely mitigate the failure mode, as Cindy only really feels the urge to stop, drop, and re-plan when she’s very anxious, and really not otherwise.
Another way of saying this is that Cindy realized that she’s invariably stupid in this particular way when she’s super anxious, and so she bans herself from making decisions during this time.
II.
Tom, like many members of technical staff at frontier labs, is highly stressed when he comes home from work. Tom pretty frequently takes this out on either himself or his partner. He takes it out on himself by binge eating food, and takes it out on his partner by (somewhat unintentionally) being cold, snippy, and short with her.
In the moment, this of course feels justified on his end. Those donuts aren’t going to eat themselves. Tom is, after all, a researcher on the pretraining team, and so he uses his brain all day; chess players have to consume lots of calories during chess tournaments to fuel their mental energy expenditure—the same probably applies to pretraining research, right? This thought, and many others, is one of the concomitant lines of justification that Tom’s +3SD-IQ brain cooks up when he’s hungry and wants to post hoc justify his strong urge to eat terrible food—food that he unfailingly regrets eating afterward.
Similarly, in the moment, it feels correct to be rude to his partner. She asked him a dumb question, and he gave an answer that is already more graceful than should be expected of him. It’s not his fault that she can’t act like the other pretraining researchers and “just get it” the first time. But yet, after Tom eats his nightly DoorDashed Mexican food, he can’t help but feel bad for how he acted when he walked through the door. She’s not a frontier AI researcher—of course she doesn’t know the persnickety details of what he’s doing during the day. Tom ends up apologizing whenever this happens, and resolves for it to not happen again. But yet, it reliably does happen when he comes home.
It would be useful here for Tom to recognize when he’s stupid, and place a deontological bar against taking certain actions during this time. Tom reliably eats poor-quality food and is rude to his partner when he gets home. So Tom decides to place the rule that he (a) can’t eat any junk food until after dinner, and (b) has to be Jesus-levels of kind to his partner before eating dinner. This is because through empirical experience, if Tom leaves the decision to the moment when he’s stupid, he’ll make the wrong one. So he takes away the decision-making power from himself until he’s out of his stupidity.
III.
Claire is a normal human being with a strong imagination. Maybe she also leans anxious, but it’s not really noticeable during the day. But Claire has the funny recurring experience of waking up in the middle of the night feeling super anxious, or feeling like something could go horribly wrong at any moment. This often causes her serious fear and anxiety, and sometimes it causes her to take certain actions, like check her phone, turn the lights on to scan the room, and carefully craft a salt ring around her bed to prevent any Catholic demons from getting her. Whenever she finally gets back to sleep and wakes up the next morning, she thinks back to how she acted during the night and morbidly laughs at how stupid her actions were. She thinks of the same things that brought her anxiety at 2am, and is completely unaffected by them.
It would serve Claire well to realize when she’s stupid, and bar herself from taking any remedial actions during this time. She has empirically found that she makes downright irrational decisions during the short time period after waking up, and so she bars herself from doing anything but ignoring her scary or anxious thoughts and going back to bed. She accepts the fact that she might come across a false negative—maybe her anxious thoughts about whether or not her parents were ok combined with her 0.7% credence that ESP is real were really telling her something that one time. But probably not.
So, Claire places a deontological bar on making any decisions during this time—she pre-makes the decision when she’s smart for the times where she’s stupid.
IV.
(non-fictional example)
Bryan Johnson—the most measured man on the planet—used to consistently binge-eat every night after getting back from building Braintree Venmo. Bryan was obviously a hyper-competent individual: extremely hard-working and intelligent, with focus so sharp you could cut diamonds with it. But during the hours of 5-10pm, “Evening Bryan” took over. Evening Bryan was stressed from the day, didn’t get great sleep the night before, had multiple young kids, and also had extreme depression. (Well, Work Bryan also had all of these—but Work Bryan was occupied with work, so he didn’t really have the chance to do something like this.) Evening Bryan was a caged animal, and an entire pan of Betty Crocker Chocolate Fudge brownies was his freedom. Bryan would always binge eat tremendous amounts of food regardless of what the better-meaning Bryans wanted him to do.
So Bryan placed a deontological bar against eating during the hours of 5-10pm. Evening Bryan would valiantly protest against this dictatorial ordinance, but it worked. He stopped binge-eating, and Bryan actually credits this as one of the first steps he took toward his current Blueprint lifestyle.
V.
Around 3200 years ago, when Odysseus crossed the Tyrrhenian Sea off the coast of Southern Italy, he knew that he had to sail past the Sirens that lived on the rocks in those waters. The Sirens were known for their song—a song that compelled all men who listened to it to claw their way off of the boat and dive into the water, swimming directly into the arms of the topless bird women singing that wondrous melody. These men were then inevitably killed by these topless bird women singing that wondrous melody. But the song sure was pretty.
Knowing this devilish information, Circe instructed Odysseus to stuff his sailors’ ears with wax, thus blocking the Sirens’ song from piercing their minds.
But Odysseus wanted more.
Odysseus wanted to hear the song himself. But he knew that upon hearing their sublime song, he would surely jump to his death. So Odysseus decided to bar himself from making a decision at this time by lashing himself to the mast of his black Achaean ship. In other words, Odysseus realized when he was going to be stupid, and then placed a deontological bar on making decisions during that time.
VI.
Will Lancer[2] is highly disagreeable, confident, and has strong opinions. He also considers himself pretty fair, in the sense that he will make sure to pass an Ideological Turing Test for his opponent's position before critiquing it. But even with this internal evaluation of fairness, he reliably finds that when he encounters a new view, he will frequently form very strong opinions about it extremely quickly. These opinions will usually be to the tune of “This view is completely and utterly insane, and it is mind-blowing to me that anyone rational holds it”. He will then propagate his views in a normal manner by telling others about them in-person and online.
But Will reliably finds that, after further inspection of these positions, they were actually stronger than he gave them credit for. Maybe they aren’t something he fully agrees with, but there are serious arguments for the position. For example, Will could imagine that if he had slightly different intuitive credences on a few key premises weighing on the issue, then he would believe the viewpoint he once thought was ludicrous.
So it would probably be useful for Will to bar himself from speaking harsh words about something unless he has rigorously investigated the opposing viewpoint, say, to the point of being able to write an essay on why it’s correct. Will realizes that he’s frequently wrong and stupid when the combination of relative ignorance + extreme dismissiveness arises in response to an idea, and so he shouldn’t speak with confidence about an issue before thoroughly steelmanning[3] the opposing side.[4]
We’ve seen a lot of instances where someone realizes that they predictably make bad decisions when they’re in a certain headspace, and so this person bars themselves from taking certain actions during this time. This bar can be on specific things, like eating food, or it can be a universal bar on decision-making in general, with a pre-filled decision made ahead of time (like Claire with her post-wake anxiety and fear).
In Googling around for this post, I found that this practice actually already has a neat name: a Ulysses contract. According to Wikipedia, a Ulysses contract is a “freely made decision that is designed and intended to bind oneself in the future.” Interestingly, its name is derived from Example V above—of Odysseus (Ulysses) strapping himself to the mast, thus making a binding choice to restrict himself in the future.
The key difference between a Ulysses contract and a good ol’ fashioned resolution is the strictness of it. You should think of your future self as a caged animal that will fight with its dying breath to get what it wants in the moment—the only way to fight a creature that has just as much intelligence as you, that’s just as good at arguing as you are, and that knows all of the rationality tricks you do is to just not fight it in the first place. Get rid of your ability to make a decision in the moment rather than imploring yourself to situationally make the correct one.
If you find yourself reliably making the same bad decisions at a certain time or while you’re in a certain headspace, consider making a Ulysses contract with yourself to avoid the temptation of the Sirens.
If you haven’t seen this term before, a deontological bar is simply a rule that says that it’s never ok to do something, regardless of the reasons you might have to do it. For example, Catholicism has a deontological bar against masturbation. Masturbation is always wrong, even if not masturbating meant that e.g. an infinite number of people were to be tortured for an infinite amount of time. Similarly, lying is a deontological bar in Catholicism. The Capuchin friar Padre Pio is quoted saying that “If I could end this war with a single lie, I would not tell it.” The war he’s referring to is WWII.
Unlike Eliezer, I do believe in the process of steelmanning. The reason I consider the Ideological Turing Test insufficient is because it’s too easy to fool yourself with it—at least, it was for me. I would articulate the position I disagreed with to a defender of that position, they would agree with it, I would point out fifteen errors with what they believe, and I would walk away vindicated in my belief. But that’s really not the standard you want to shoot for. Especially if you’re good at debating or thinking about things, I would strongly recommend making the best case possible for the opposing side using all of that brainpower you have. The truth of the matter is that you are frequently the strongest intellectual opponent you have immediate access to, and using those abilities for the opposing side is good intellectual practice. As a side note, I consider steelmanning your opponent to be an application of Jesus’ dictum of loving your enemies. I will probably write a blog post later about the virtues of steelmanning in general, as well as its virtues as opposed to passing an Ideological Turing Test.
Guys, I have something to admit to you. This “Will Lancer” character in the story, the one that I told you wasn’t me? Yeah. About that.... On the bright side, I’m finding that truly steelmanning the opposing side is both fun and epistemically helpful for finding flaws in my thinking. And I only really started doing this in response to the idea that I’m writing about right now!
[This post is the second post in my thirty day writing challenge. Tune in for more if you enjoy it!]
If you reliably knew that you would make a bad decision during a certain time, would you let yourself make decisions during that time? Probably not, right? So why do you let yourself grocery shop when hungry, speak negatively when you have low blood sugar, and make key decisions when anxious?
A core epistemic skill that I’ve tried to cultivate over the years is the idea of knowing when I’m stupid, and not letting myself make decisions during these times. Like all core epistemic skills, I’ve cultivated this in response to getting repeatedly smacked across the face over and over again.
I.
Cindy is an anxious woman. Her resting heart rate is a bit high, she gets wigged out easily, and she tends to run an endless stream of disaster loops through her head at all hours of the day. Like all anxious people, her anxiety rises and falls. Maybe it rises when her blood sugar is low, when she’s under-slept, or when other things in her life are particularly stressful. Probably all three.
She, like all people, has a number of decisions to make throughout the day. Some of these decisions are big and some of them are small. Cindy reliably finds that when she’s flooded with anxiety, she repeatedly runs into certain failure modes. For example, maybe she makes elaborate plans to fix every single problem in her entire life, half-commits to it for, like, one day, and then decides that: no, what was she thinking? That plan is terrible! It’s time for another ground-breaking, earth-shaking plan that will fix every single problem in her entire life. She then half-commits to this plan, and then one day later comes to a terrible realization....
Cindy notices that this kind of planning loop stops her from making any actual progress. It’s like she’s in a car and wants to go in a certain direction, but instead she goes into a hard right turn, and then a hard left turn, and then a hard right turn, and then a hard... you get the idea. If Cindy just stuck with a plan, even if it was kind of mediocre, then she would be able to make some sort of progress over time. It’s similar to how a random walk has an expected displacement of zero, but a directed random walk has an expected displacement that grows like n, where n is the number of steps.
Since Cindy notices that she reliably gets the extremely strong urge to stop literally everything she’s doing and make a Plan to Rule All Plans when she feels very anxious, it would be a good idea for Cindy to ban herself from planning when she feels this way—to make a deontological bar[1] against planning when anxious. This would likely mitigate the failure mode, as Cindy only really feels the urge to stop, drop, and re-plan when she’s very anxious, and really not otherwise.
Another way of saying this is that Cindy realized that she’s invariably stupid in this particular way when she’s super anxious, and so she bans herself from making decisions during this time.
II.
Tom, like many members of technical staff at frontier labs, is highly stressed when he comes home from work. Tom pretty frequently takes this out on either himself or his partner. He takes it out on himself by binge eating food, and takes it out on his partner by (somewhat unintentionally) being cold, snippy, and short with her.
In the moment, this of course feels justified on his end. Those donuts aren’t going to eat themselves. Tom is, after all, a researcher on the pretraining team, and so he uses his brain all day; chess players have to consume lots of calories during chess tournaments to fuel their mental energy expenditure—the same probably applies to pretraining research, right? This thought, and many others, is one of the concomitant lines of justification that Tom’s +3SD-IQ brain cooks up when he’s hungry and wants to post hoc justify his strong urge to eat terrible food—food that he unfailingly regrets eating afterward.
Similarly, in the moment, it feels correct to be rude to his partner. She asked him a dumb question, and he gave an answer that is already more graceful than should be expected of him. It’s not his fault that she can’t act like the other pretraining researchers and “just get it” the first time. But yet, after Tom eats his nightly DoorDashed Mexican food, he can’t help but feel bad for how he acted when he walked through the door. She’s not a frontier AI researcher—of course she doesn’t know the persnickety details of what he’s doing during the day. Tom ends up apologizing whenever this happens, and resolves for it to not happen again. But yet, it reliably does happen when he comes home.
It would be useful here for Tom to recognize when he’s stupid, and place a deontological bar against taking certain actions during this time. Tom reliably eats poor-quality food and is rude to his partner when he gets home. So Tom decides to place the rule that he (a) can’t eat any junk food until after dinner, and (b) has to be Jesus-levels of kind to his partner before eating dinner. This is because through empirical experience, if Tom leaves the decision to the moment when he’s stupid, he’ll make the wrong one. So he takes away the decision-making power from himself until he’s out of his stupidity.
III.
Claire is a normal human being with a strong imagination. Maybe she also leans anxious, but it’s not really noticeable during the day. But Claire has the funny recurring experience of waking up in the middle of the night feeling super anxious, or feeling like something could go horribly wrong at any moment. This often causes her serious fear and anxiety, and sometimes it causes her to take certain actions, like check her phone, turn the lights on to scan the room, and carefully craft a salt ring around her bed to prevent any Catholic demons from getting her. Whenever she finally gets back to sleep and wakes up the next morning, she thinks back to how she acted during the night and morbidly laughs at how stupid her actions were. She thinks of the same things that brought her anxiety at 2am, and is completely unaffected by them.
It would serve Claire well to realize when she’s stupid, and bar herself from taking any remedial actions during this time. She has empirically found that she makes downright irrational decisions during the short time period after waking up, and so she bars herself from doing anything but ignoring her scary or anxious thoughts and going back to bed. She accepts the fact that she might come across a false negative—maybe her anxious thoughts about whether or not her parents were ok combined with her 0.7% credence that ESP is real were really telling her something that one time. But probably not.
So, Claire places a deontological bar on making any decisions during this time—she pre-makes the decision when she’s smart for the times where she’s stupid.
IV.
(non-fictional example)
Bryan Johnson—the most measured man on the planet—used to consistently binge-eat every night after getting back from building Braintree Venmo. Bryan was obviously a hyper-competent individual: extremely hard-working and intelligent, with focus so sharp you could cut diamonds with it. But during the hours of 5-10pm, “Evening Bryan” took over. Evening Bryan was stressed from the day, didn’t get great sleep the night before, had multiple young kids, and also had extreme depression. (Well, Work Bryan also had all of these—but Work Bryan was occupied with work, so he didn’t really have the chance to do something like this.) Evening Bryan was a caged animal, and an entire pan of Betty Crocker Chocolate Fudge brownies was his freedom. Bryan would always binge eat tremendous amounts of food regardless of what the better-meaning Bryans wanted him to do.
So Bryan placed a deontological bar against eating during the hours of 5-10pm. Evening Bryan would valiantly protest against this dictatorial ordinance, but it worked. He stopped binge-eating, and Bryan actually credits this as one of the first steps he took toward his current Blueprint lifestyle.
V.
Around 3200 years ago, when Odysseus crossed the Tyrrhenian Sea off the coast of Southern Italy, he knew that he had to sail past the Sirens that lived on the rocks in those waters. The Sirens were known for their song—a song that compelled all men who listened to it to claw their way off of the boat and dive into the water, swimming directly into the arms of the topless bird women singing that wondrous melody. These men were then inevitably killed by these topless bird women singing that wondrous melody. But the song sure was pretty.
Knowing this devilish information, Circe instructed Odysseus to stuff his sailors’ ears with wax, thus blocking the Sirens’ song from piercing their minds.
But Odysseus wanted more.
Odysseus wanted to hear the song himself. But he knew that upon hearing their sublime song, he would surely jump to his death. So Odysseus decided to bar himself from making a decision at this time by lashing himself to the mast of his black Achaean ship. In other words, Odysseus realized when he was going to be stupid, and then placed a deontological bar on making decisions during that time.
VI.
Will Lancer[2] is highly disagreeable, confident, and has strong opinions. He also considers himself pretty fair, in the sense that he will make sure to pass an Ideological Turing Test for his opponent's position before critiquing it. But even with this internal evaluation of fairness, he reliably finds that when he encounters a new view, he will frequently form very strong opinions about it extremely quickly. These opinions will usually be to the tune of “This view is completely and utterly insane, and it is mind-blowing to me that anyone rational holds it”. He will then propagate his views in a normal manner by telling others about them in-person and online.
But Will reliably finds that, after further inspection of these positions, they were actually stronger than he gave them credit for. Maybe they aren’t something he fully agrees with, but there are serious arguments for the position. For example, Will could imagine that if he had slightly different intuitive credences on a few key premises weighing on the issue, then he would believe the viewpoint he once thought was ludicrous.
So it would probably be useful for Will to bar himself from speaking harsh words about something unless he has rigorously investigated the opposing viewpoint, say, to the point of being able to write an essay on why it’s correct. Will realizes that he’s frequently wrong and stupid when the combination of relative ignorance + extreme dismissiveness arises in response to an idea, and so he shouldn’t speak with confidence about an issue before thoroughly steelmanning[3] the opposing side.[4]
We’ve seen a lot of instances where someone realizes that they predictably make bad decisions when they’re in a certain headspace, and so this person bars themselves from taking certain actions during this time. This bar can be on specific things, like eating food, or it can be a universal bar on decision-making in general, with a pre-filled decision made ahead of time (like Claire with her post-wake anxiety and fear).
In Googling around for this post, I found that this practice actually already has a neat name: a Ulysses contract. According to Wikipedia, a Ulysses contract is a “freely made decision that is designed and intended to bind oneself in the future.” Interestingly, its name is derived from Example V above—of Odysseus (Ulysses) strapping himself to the mast, thus making a binding choice to restrict himself in the future.
The key difference between a Ulysses contract and a good ol’ fashioned resolution is the strictness of it. You should think of your future self as a caged animal that will fight with its dying breath to get what it wants in the moment—the only way to fight a creature that has just as much intelligence as you, that’s just as good at arguing as you are, and that knows all of the rationality tricks you do is to just not fight it in the first place. Get rid of your ability to make a decision in the moment rather than imploring yourself to situationally make the correct one.
If you find yourself reliably making the same bad decisions at a certain time or while you’re in a certain headspace, consider making a Ulysses contract with yourself to avoid the temptation of the Sirens.
If you haven’t seen this term before, a deontological bar is simply a rule that says that it’s never ok to do something, regardless of the reasons you might have to do it. For example, Catholicism has a deontological bar against masturbation. Masturbation is always wrong, even if not masturbating meant that e.g. an infinite number of people were to be tortured for an infinite amount of time. Similarly, lying is a deontological bar in Catholicism. The Capuchin friar Padre Pio is quoted saying that “If I could end this war with a single lie, I would not tell it.” The war he’s referring to is WWII.
No relation to the author whatsoever.
Unlike Eliezer, I do believe in the process of steelmanning. The reason I consider the Ideological Turing Test insufficient is because it’s too easy to fool yourself with it—at least, it was for me. I would articulate the position I disagreed with to a defender of that position, they would agree with it, I would point out fifteen errors with what they believe, and I would walk away vindicated in my belief. But that’s really not the standard you want to shoot for. Especially if you’re good at debating or thinking about things, I would strongly recommend making the best case possible for the opposing side using all of that brainpower you have. The truth of the matter is that you are frequently the strongest intellectual opponent you have immediate access to, and using those abilities for the opposing side is good intellectual practice. As a side note, I consider steelmanning your opponent to be an application of Jesus’ dictum of loving your enemies. I will probably write a blog post later about the virtues of steelmanning in general, as well as its virtues as opposed to passing an Ideological Turing Test.
Guys, I have something to admit to you. This “Will Lancer” character in the story, the one that I told you wasn’t me? Yeah. About that.... On the bright side, I’m finding that truly steelmanning the opposing side is both fun and epistemically helpful for finding flaws in my thinking. And I only really started doing this in response to the idea that I’m writing about right now!