Even if the efforts of any particular person are likely to go to waste, if a large number of people follow the strategy of “do things that have a small chance of being hugely beneficial on net”, this may turn out to be more impactful than if they all tried to maximize their chance of making an individual impact.
As a counterpoint, I'm reminded of this post from the EA Forum:
The best cause will disappoint you: An intro to the optimisers curse
in which the author argues against naively trying to maximize for expected value and argues for choosing grounded causes over speculative causes instead. Curious for your thoughts on this.
I read this less as "counterpoint" and more "here's one thing to take into account when trying to figure out what to do". Though it's also pretty unclear to me how to apply the lesson from the post in practice, since if one is considering something like "what should I do with my career", I don't know what would count as a grounded vs. speculative career direction. (If I spend a year writing blog posts, is that grounded or speculative?)
Based on a conversation with Jukka Tykkyläinen and Kimmo Nevanlinna. The original framing and many of the ideas are stolen from them.
You can think of any job as having inputs, outputs, and valued outcomes.
The input is typically time you spend on doing something in particular, as well as any material resources you need. Outputs are the immediate results of what you do. Valued outcomes are the reason why you’re being paid to do the job in the first place.
In many jobs, these are closely linked:
Digging a tunnel
Store cashier in a busy store
Childcare
They’re not exactly the same. You could spend a lot of time on the job but do it poorly (dig lazily, ring up purchases wrong, be neglectful or abusive toward the children). But assuming that you are trying to do a reasonable job and aren’t grossly incompetent, you can generally get things done by just showing up and working for a given number of hours.
In other jobs, the three are different:
B2B email marketing
It’s not enough to just send any emails; you have to send out good emails that actually get you clients. In fact, it can be better to write one really good email than it is to write a hundred bad ones.
Scientific research
It’s possible to rack up lots of uninteresting publications that still get cited for whatever reason, without that actually contributing to the generation of any real knowledge.
In this second category of jobs, at least two things are different:
1. Outputs and valuable outcomes become much more weakly connected. You might genuinely do your best producing many real outputs and put a lot of work into them. And they might still completely fail to produce any valuable outcomes.
2. Because the connection between outputs and valuable outcomes weakens, it gets harder to tell when you’re doing a bad job and you may be actively rewarded for it. Previously, you could spend a lot of time working in a way that failed to be valuable, but this was usually relatively obvious. A salesperson who steals from the till might be unobvious in the sense of being good at hiding it, but at least everyone knows that a salesperson isn’t supposed to steal.
In contrast, if you can’t directly measure what leads to valuable outcomes but you do still need to reward performance, it’s easy to run into Goodhart’s Law. People may actively reward you for having lots of outputs separate from their connection to valued outputs.
For instance, hiring and grant committees may evaluate how many publications and citations a researcher has. This encourages maximizing the number of publications, such as by splitting up a single study into as many individual papers as possible, rather than just writing one really good and comprehensive paper. The reward process is actively pushing the researcher away from the kind of work that would produce value.
People may also be rewarded for outputs with genuine value, but in a way that neglects others that would be necessary for valuable outcomes. For instance, someone may be rewarded for their individual contributions, but get no credit for helping other team members, even if it would be more valuable for them to focus on that.
It might be tempting to conclude from this that “always focus on the outcomes”.
But.
What’s a valued outcome for you may not be the same as a valued outcome for whoever is employing you. Maybe you think that the company is doing something useless, or it’s organized in such a way that your personal contributions are useless. For instance, you are writing reports that will never be read. Or management keeps changing plans, so whatever work you just did will typically get thrown away in favor of the latest idea.
For you, the valued outcome is just getting paid, and your salary is not tied to the value of your work. In fact, even if you think that the company is doing something valuable and that you could do something valuable… your salary may still be anticorrelated with the value of your work if you get evaluated and promoted based on metrics that reward the wrong things, or fail to reward the right things.
In that case, it makes more sense to focus on whatever outcomes get you promoted. Maybe that will fail to further the company’s interests, but that’s the company’s problem for not developing a better incentive structure.
Many companies may not even want their rank-and-file employees thinking about outcomes. They just want the rank-and-file showing up and doing what they are told rather than trying to change established procedures. Then you might get something like the rank-and-file caring about inputs (“we come in and do our time”), middle management caring about outputs (“let’s get the employees to hit these metrics”), and the executives caring about outcomes (“let’s have these metrics that we expect to bring in profit”).
This argument feels different for researchers in fields using public funds. In those, there’s a stronger case for focusing on outcomes even if the incentives are bad. “Academia” doesn’t have a single decision-maker who could be held responsible for the incentives, and the work is supposed to benefit the public as a whole. In principle, if you are not willing to do what’s genuinely scientifically valuable, you should not be in the field at all.
In practice, the reality may also be that one has student loans and a family to support and a situation where they need to play the incentive game to keep their income.
Some relevant questions are “Why are you in this field in the first place?” and “What’s generally expected of you?”. If you are working at a Big Corporation just to make a living, it’s reasonable to just focus on whatever gets you money and let the management worry about how to best incentivize you into doing a good job. If you are working in something like academia, a hospital, the fire department, or a non-profit with a genuine intent to do good - you may still need to deal with the reality of the incentive structure, but you shouldn’t lose track of the real purpose of your job.
Philosopher C. Thi Nguyen says in an interview:
Sometimes when I get frustrated with work, it’s because I was expecting or hoping my inputs to equal outputs, and they don’t.
This might look like: I haven’t written a blog post in a while and I feel that I should get one out soon. I’m thinking, “let me just sit down and write a post”. What I’m hoping to experience is the feeling of just writing and making steady progress until I have a finished article.
But then things don’t go that way. Instead of being able to just sit down and write, I realize that my idea doesn’t work and I have to rethink it, and I have no idea of how long that will take. Spending all that time thinking about it is progress, but it often doesn’t feel like progress, because there’s no clear Number Going Up that would make the progress tangible.
Or rather, spending all that time thinking about it can be progress. It’s also possible to spend a lot of time thinking about something and just stay stuck; the input doesn’t translate to much in the way of outputs. And that’s part of why it’s stressful: you can’t necessarily distinguish between “I spent a lot of time thinking about this and then made valuable progress” from “I spent a lot of time thinking about this and most of that time was effectively wasted” in advance.
If it’s actually urgent to make progress on something and it’s not turning into tangible outputs, it can be rational to get frustrated and despair. It means that something is not working, and you might do better switching strategies to something that produces faster progress. (One failure mode of some AIs is never getting frustrated and then trying the same useless approach over and over again.)
There's an old article by @AnnaSalamon and @Duncan Sabien (Inactive) on why startup founders have mood swings - flipping from “I have the greatest product ever” and “my product is terrible and will never succeed” from one day to another. They note that other circumstances that can cause this kind of oscillation include
And they hypothesize that all of these have a common cause: high pressure and uncertainty about what the hell you even are supposed to do. In their words: “not just being unsure about which path to take, but also about whether the paths (and the destination!) are even real”.
In the language of this post, uncertainty about what kinds of inputs and outputs you should be aiming for to achieve the valued outcomes, and sometimes about what the valued outcomes even are. I’ll just call this “lack of clarity” for short.
Some people aren’t doing anything with high stakes, but still get stressed out when it’s not clear what to do. And for that matter, not everyone finds their first major artistic endeavor particularly stressful, either. Often when I get stressed about not making enough progress within a given time, there isn’t any objective reason to - I’m just placing high demands on myself and then getting stressed when I feel like I’m failing to meet them.
Thus, one’s ability to deal with lack of clarity has to do both with one’s personal ambiguity tolerance and the amount of external pressure they’re facing.
It’s then reasonable to be pulled toward things where the connection between the three is clear and straightforward. It means that it is always easy to know what to do, and the maximum amount of time and energy is spent on something valuable. Frustration about them being poorly connected is real information - it would be nicer if they were clearly connected, and a person could just optimize for one thing.
AI and machine learning also works best on tasks where you have a clear set of target variables and can unleash all of the system’s optimization power on exactly those variables. When the system can get clear feedback of how all of its actions are reflected in the variables of interest, it can rapidly iterate toward the actions that produce the best outcomes.
Similarly, many games are enjoyable because the outputs and valued outcomes are so clearly linked. If you are shooting enemies and the player with the greatest number of kills wins, then it is very straightforward to figure out what to do. You can just focus on finding the best way to execute it.
As this suggests, the desire for clarity affects people not only at work, but with their hobbies as well. While people may not have explicit goals for their hobbies, they might enjoy things like getting better at them - or thinking that they do. The popularity of things like the fallacious 10,000-hour rule for expertise - “do a skill for 10,000 hours to become a master at it” - may be in part because it suggests a direct relationship between a time input and a mastery outcome. Just keep doing a thing and you’ll become great at it, goes the idea.
Just as with jobs, in some hobbies the valued outcomes are more directly linked to the inputs and outputs than in others. There is a reasonably close connection between the amount of time spent weightlifting, the amount of weight you can lift, and your muscle mass. For something like writing good fiction or interesting essays, things are much more subjective and murky.
I used to work in AI strategy research, but I found it very challenging for a number of reasons, including lack of clarity. I wanted to do research that would prevent the world from being destroyed by out-of-control AI, but there was no way to know what the hell would contribute to that. I spent much of my time feeling that this was critically important to get right, and also paralyzed with indecision. Eventually, I quit in order to do something easier.
But I still want to contribute to the world becoming a better place, as vague of a goal as it is. And given how much progress AI has made recently, I do want to somehow make sure things go in a positive direction.
And it’s still impossible to know what would actually be useful for it.
So my goal has been: try to write blog articles regularly. Not just about AI, but about anything that I happen to find interesting. If I can’t stop the world from being destroyed entirely, maybe I can at least make it slightly better while we’re still alive to enjoy it.
Here, the question of what kinds of outputs to aim for gets interesting.
I’ve been aiming for at least one article a month (I’ve been able to stick to this), ideally once a week (I haven’t been able to stick to this). This is an output goal comparable to “publish scientific papers regularly” - it doesn’t directly lead to the world becoming better. It would be totally possible to write one essay a week for years with absolutely nobody reading them, and that having no impact on the world.
But it seems to me that it is useful for me to act as if the outputs directly lead to valuable outcomes. Throughout my life, I have written quite a few blog articles, often just for the sake of writing them. Later, various people have thanked me for it, saying that the articles were valuable for them. Apparently, if I pursue things that feel interesting to me, those things are often interesting for others as well.
While I don’t expect every blog article I write to be interesting and valuable - I expect that a large share of them will turn out to be of little value - each one of them has a probabilistic chance of turning out to be useful. There is also the effect that the reception I get for them affects what I end up writing in the future - if people respond particularly well to some of them, I’m more likely to write about that topic.
That said, while acting as if there was a direct connection between outputs and valued outcomes is useful to some extent, it’s important not to overdo it! If I started only optimizing for publication frequency with no regard for the content, the connection between the output and the outcome would disappear - Goodhart’s law again.
This is especially the case since some of my better posts take much longer to write than others - definitely more than one week. So the rule is something like - focus on just writing posts frequently and don’t worry too much about any individual post, except if spending more time on a post would produce a clearly better one, in which case, do sacrifice quantity for quality.
Interestingly, something like this also applies recursively within the goal of writing regularly. Specifically, there is the question of what my goal should be when I sit down to write.
I mentioned earlier that if I sit down with the goal of “getting a post written”, then that might be a counterproductive mindset if writing it takes longer than I thought. Those kinds of goals also easily put me into a mindset of “Getting Work Done”, which I’ve found generally counterproductive for creative work.
So even if I have decided that the output I’m aiming for is “a couple of articles per month”, there is the question of whether my output goal on some given day should be “finish an article” or “just spend some time writing”.
“Just spend some time writing (or outlining, or thinking about the topic of the article, etc.)” is often better - unless it’s close to the end of the month and I need to get something out, even if I won’t be totally happy with it. And it’s still possible for the “just spend some time writing” goal to become corrupt - if I then spend all my time doing something vaguely connected to writing but make no meaningful progress at it for an extended time.
And at the same time, it’s important to feel okay with the fact that some days are just going to be spent making no meaningful progress because my brain needs time to background process, or because I’ve gone down a path that turns out to be a dead end, or something similar.
David MacIver writes that he sometimes finds himself wanting read more, but isn’t sure what exactly he wants to read, so he needs to make himself read until he figures it out:
The way he describes his approach is similar to how I’m thinking of blogging: make up a number and treat that as the valuable outcome. Except that, when I notice that the number is distracting me from what I actually want to achieve, I should then stop treating it as the valuable outcome.
I mentioned that for blog posts, each post has a probabilistic chance of being valuable, so I might just focus on writing posts and trust them to be valuable in expectation, even if no particular one is. The outputs do not equal valuable outcomes, but I get good results by pretending that they do.
For something like making large-scale effects on the world, one might be faced with the dilemma where there’s only a probabilistic chance of their whole career turning out to be valuable. Maybe some researcher pursues a particular theory or invention that may turn out to be immensely valuable for humanity - or it might be a complete dead end. Maybe a reformer is working to change things in a particular country, where other forces may eventually turn out to completely eradicate all their work.
There, it may be best to just focus on outputs that have a probabilistic chance of leading to a good outcome. Even if the efforts of any particular person are likely to go to waste, if a large number of people follow the strategy of “do things that have a small chance of being hugely beneficial on net”, this may turn out to be more impactful than if they all tried to maximize their chance of making an individual impact. Many social reforms also take a really long time - decades or more - and tying one’s self-worth too much into visible results may be psychologically devastating.
But at the same time, it’s still bad to completely lose one’s sight of what all the work is meant to achieve, or it might just become an entirely lost purpose.
(And there’s the huge topic of “what if your work might have a negative impact”, which is outside the scope of this post.)
Though if you are tunnel man, just having dug the tunnel may be intrinsically valuable, regardless of how it is ever used.