I sometimes hear of EAs or rationalists thinking about their career plans and trying to account for their bias and the incentives they face. They want to compare their plans with some hypothetical ideal plan without their bias, and that was immune to social pressure or financial incentives. But they can’t do that, and then have nothing to compare to and measure against.
Trying to compare to the ideal case, and defend against The Bottom Line sounds real hard, but there's another case you can compare against! I haven’t yet heard of someone just writing up the plan as if they were biased and following their local social incentives. Write up the bad version of the plan you are trying to avoid, and now you have something to compare to. Something to measure against. This is my solution, and I call it the wormtongue test.
The wormtongue test is a sibling to Murphyjitsu. Murphyjitsu asks, "Suppose you get a message from the future that you failed, what do you think caused it?" Whereas the wormtongue test asks, "Suppose you get a message from the future that you succeeded, but it didn't matter because you were inadvertently making things worse or pointlessly toiling under a streetlight. What went wrong?"
Here's the procedure: write up your plan to achieve some goal. Career or otherwise. Include all the considerations you're making, the uncertainties you have, consider your resources and constraints.
Then take a break. Get up, walk around, get a snack. Or better yet, come back to this the next day, so you have some real space and are not anchored on what you just wrote.
Then write the wormtongue plan.
Think of a friend who can be a stand-in for you — similar situation and goals. Then imagine you are Gríma Wormtongue[1], and you need to write up a plan that is appealing to them. It must sound appealing and not be something they will easily dismiss. Make them feel like they are doing something important, especially if you can track progress. People love progress they can see. Bonus points if you can make the plan align with local social and financial incentives. Perhaps you make it rely on assumptions they can't test yet but seem reasonable. But the plan must not help. It must not actually solve the central difficulty. Even better if the plan makes things subtly worse in a way that's hard to trace back to the plan being flawed.
Then take the two different plans and compare them.
Is there a meaningful difference?
If your plan is too similar to the wormtongue plan, you’re in trouble. Your plan is probably self-serving or falling prey to financial or status incentives, and won't achieve your goal. Some similarities are expected; the wormtongue plan is trying to seem virtuous, but there must be crucial differences. Can you justify your real plan's similarities with the wormtongue plan? Can you point to places of divergence that are virtuous and costly?
If you want the hard version of the test, give the two plans to your friend, and ask them if they can tell which plan is which. Or better yet, write wormtongue plans for each other, and compare to your real plans. You can see incentives they are blind to or rationalizing away, and they can do the same for you.
To score yourself on the wormtongue test, treat each feature of your plan as evidence about two hypotheses: corrupt plan, vs virtuous plan. For each feature ask, . Features that are shared contribute nothing, only the differences matter. Features that spring from the same underlying trait shouldn't be counted twice. Sum the log-ratios and you get total bits for or against corruption. Lower is better, negative bits favor virtue.
Your score on the wormtongue test is measured in bits of corruption, by summing the log-ratios of each feature of your plan.
Write up the wormtongue plan, and keep your real plan free of corruption.
Gríma Wormtongue is a character in The Lord of the Rings who acts as the King of Rohan’s advisor. He is secretly in the employ of the evil wizard Saruman, and is giving the king advice that sounds good, but is steadily weakening his kingdom.
I sometimes hear of EAs or rationalists thinking about their career plans and trying to account for their bias and the incentives they face. They want to compare their plans with some hypothetical ideal plan without their bias, and that was immune to social pressure or financial incentives. But they can’t do that, and then have nothing to compare to and measure against.
Trying to compare to the ideal case, and defend against The Bottom Line sounds real hard, but there's another case you can compare against! I haven’t yet heard of someone just writing up the plan as if they were biased and following their local social incentives. Write up the bad version of the plan you are trying to avoid, and now you have something to compare to. Something to measure against. This is my solution, and I call it the wormtongue test.
The wormtongue test is a sibling to Murphyjitsu. Murphyjitsu asks, "Suppose you get a message from the future that you failed, what do you think caused it?" Whereas the wormtongue test asks, "Suppose you get a message from the future that you succeeded, but it didn't matter because you were inadvertently making things worse or pointlessly toiling under a streetlight. What went wrong?"
Here's the procedure: write up your plan to achieve some goal. Career or otherwise. Include all the considerations you're making, the uncertainties you have, consider your resources and constraints.
Then take a break. Get up, walk around, get a snack. Or better yet, come back to this the next day, so you have some real space and are not anchored on what you just wrote.
Then write the wormtongue plan.
Think of a friend who can be a stand-in for you — similar situation and goals. Then imagine you are Gríma Wormtongue[1], and you need to write up a plan that is appealing to them. It must sound appealing and not be something they will easily dismiss. Make them feel like they are doing something important, especially if you can track progress. People love progress they can see. Bonus points if you can make the plan align with local social and financial incentives. Perhaps you make it rely on assumptions they can't test yet but seem reasonable. But the plan must not help. It must not actually solve the central difficulty. Even better if the plan makes things subtly worse in a way that's hard to trace back to the plan being flawed.
Then take the two different plans and compare them.
Is there a meaningful difference?
If your plan is too similar to the wormtongue plan, you’re in trouble. Your plan is probably self-serving or falling prey to financial or status incentives, and won't achieve your goal. Some similarities are expected; the wormtongue plan is trying to seem virtuous, but there must be crucial differences. Can you justify your real plan's similarities with the wormtongue plan? Can you point to places of divergence that are virtuous and costly?
If you want the hard version of the test, give the two plans to your friend, and ask them if they can tell which plan is which. Or better yet, write wormtongue plans for each other, and compare to your real plans. You can see incentives they are blind to or rationalizing away, and they can do the same for you.
To score yourself on the wormtongue test, treat each feature of your plan as evidence about two hypotheses: corrupt plan, vs virtuous plan. For each feature ask,
Your score on the wormtongue test is measured in bits of corruption, by summing the log-ratios of each feature of your plan.
Write up the wormtongue plan, and keep your real plan free of corruption.
Gríma Wormtongue is a character in The Lord of the Rings who acts as the King of Rohan’s advisor. He is secretly in the employ of the evil wizard Saruman, and is giving the king advice that sounds good, but is steadily weakening his kingdom.