Up until “Fuck The Symbols” I’m with you. And as an article for the general public, I’d probably endorse the “Fuck the Symbols” section as well.
In particular:
it’s usually worth at least thinking about how to do it - because the process of thinking about it forces you to recognize that the Symbol does not necessarily give the thing, and consider what’s actually needed.
To the extent this is advocacy, however, it seems worth noting that I think the highly engaged LW crowd is already often pretty good about this, (so I’d be more excited about this being read by new LWers). In fact, in my experience, the highly-engaged LW crowd’s bias is already too far toward “fuck the symbols”.
There’s a lot of information that can be gained by examining the symbols. For example, I think EA’s efforts toward global development are highly stunted by a lack of close engagement with many existing efforts to do good. Working at a soup kitchen is probably not the best use of a poverty-focused EA’s time. But learning about UN programs, the various development sectors and associated interventions, and the status and shortcomings of existing M&E, I think very likely are (for those who haven’t done so). Doing so revealed to me a myriad of interventions that I’d expect to be higher impact than those endorsed by GiveWell. The symbols often contain valuable information.
The symbols can also be useful. Ivy League MBAs probably have an easier time raising money for certain types of businesses than do others.
So ‘fuck the symbols’ just feels much too strong to me, and in fact in the opposite direction I’d advocate, for the particular audience reading this.
You're pointing to a good problem, but there's more to it than a slider between "pay more attention to Symbols" and "pay less attention to Symbols". I was thinking about writing a whole other post on this, but I'll give a short version here.
First, suppose some group of people repeatedly says "Fuck the Symbols" and then manages to do the corresponding things without the Symbols, across a bunch of different areas. How does this parse to others, through a social lens? Basically, it's countersignalling. It sends a message of "we are so good that we can do the things without the Symbols, those people with the Symbols are just a bunch of posers". If a group of people does this consistently across a whole bunch of areas, then we basically have Counterculture.
(To be clear, I'm not saying that the intent behind saying "Fuck the Symbols" is necessarily to countersignal. Everything in the post still holds; saying "Fuck the Symbols" is an object-level useful way to avoid Goodhart. But some people insist on interpreting everything as a social signal, regardless of intent, and countersignalling is an obvious way for Fuck the Symbols to parse.)
One interesting output of this model: it says that groups which focus on object-level things (at least to start) will naturally end up counterculture-loaded. The countercultural inclinations of e.g. the rationalist/EA community is not an accident of history, it's a natural result of the core approach.
But now there's a danger: Fuck the Symbols becomes a symbolic move in its own right. People start to say Fuck the Symbols not because it's object-level useful, but because it's a recognizable social signal. It's a status-granting Symbol within the counterculture. Once that happens, people will inevitably Goodhart on Fucking the Symbols, rather than doing it when it's object-level useful.
The solution to this is not to move a mental slider from "pay less attention to Symbols" to "pay more attention to Symbols". That would throw out the baby with the bathwater. Rather, the solution is to double-check whether Fucking the Symbols is actually useful in each particular situation. Think about how to solve the problem without the Symbol, in order to force yourself to consider the actual thing, but then go back and ask how the Symbol can improve the solution.
Or, go full meta: if Fucking the Symbols is itself a Symbol, then Fuck that Symbol in particular: ask how to solve the problem without Fucking the Symbols. (Culturally, this would parse as metacontrarianism.)
Doing so revealed to me a myriad of interventions that I’d expect to be higher impact than those endorsed by GiveWell.
Have you talked to GiveWell about this? Like, I don't know much of anything about charities or the people at GiveWell. But the standard rationalist reaction to "I found something you can read that helps explain the domain you're in and how to make the world a better place" is supposed to be "give me, give it now".
(And the reason I ask is the huge potential upside of making GiveWell donations more effective.)
Yes, and they are public, and others have highlighted similar things to them and publicly.
GiveWell is now starting to look into a subset of these things:
To date, most of GiveWell’s research capacity has focused on finding the most impactful programs among those whose results can be rigorously measured. ...
GiveWell has now been doing research to find the best giving opportunities in global health and development for 11 years, and we plan to increase the scope of giving opportunities we consider. We plan to expand our research team and scope in order to determine whether there are giving opportunities in global health and development that are more cost-effective than those we have identified to date.
We expect this expansion of our work to take us in a number of new directions,
Over the next several years, we plan to consider everything that we believe could be among the most cost-effective (broadly defined) giving opportunities in global health and development. This includes more comprehensively reviewing direct interventions in sectors where impacts are more difficult to measure, investigating opportunities to influence government policy, as well as other areas.
https://blog.givewell.org/2019/02/07/how-givewells-research-is-evolving/
The only reason ivy league MBA has easier time is because the symbol is used as a shortcut of the actual vetting process of qualifications, so are most social status symbols. They indicate certain qualities and expertise on certain topics because of the general process involved in getting those status symbols in the first place. Of course the actual usefulness of the individuals have to be vetted by doing real work. Outside of playing a role in the production of actual substance, they are mostly used in human social interactions and communications to induce specific positive emotions in said individuals.
The person who eats Good Food (possibly Healthy Food, possibly Expensive Food, possibly Ethical Food, depending on their social circles) but never really notices how much they enjoy the actual taste of different foods. I don't think this is a particularly good example, because it's perfectly coherent to eat Healthy or Ethical Food without caring about taste at any level. (In fact, I'd say taste is the only argument (besides maybe price) against eating food that's at least one of those.)
Nice post!
Y'know, I just figured I should try to comment more, even if I have nothing to say, as a strategy for increasing my LessWrong karma. :-P
Related to https://www.lesswrong.com/tag/goodhart-s-law, which has had a lot written about it, though in somewhat different contexts.
Unfortunately, the problem with "fuck the symbols" is that the universe is more complex than any participant can calculate, so abstractions and symbols are usually absolutely necessary to actually think or act on any topic. There is no "the thing" that fits in a brain - only different dimensions of compression and lossy abstractions.
Certainly "be sceptical of symbols", and "try multiple symbols for the same state of things". Being aware that over-focus on a simplified model is usually wrong in important ways is key here. But you can't do away with them entirely, you just don't have enough processing power.
But you can't do away with them entirely, you just don't have enough processing power.
I don't think that's what johnswentworth is advocating.
This isn’t always the best choice, but it’s usually worth at least thinking about how to do it - because the process of thinking about it forces you to recognize that the Symbol does not necessarily give the thing, and consider what’s actually needed.
I really like the structure of this post.
In particular, 1) starting off by giving the reader a sense of what the post is about. I think that sort of context is useful before proceeding to the section on examples. I also like how this was accomplished so quickly.
And then 2) diving into various examples before proceeding to the larger idea. Eliezer wrote about this in SotW: Be Specific, and it's always stuck with me.
(Also, note how this post starts off with a specific example - an instance of the concrete-abstract writing pattern in which you state the example first and the generalization afterward. This is one of the most common bits of nonfiction writing advice I dispense: "Open with the concrete example, not the abstract explanation!")
I'm a big fan of it. And I predict that over the course of time, this pattern will become more and more popular on Less Wrong, and then in the blogosphere more broadly.
Pulling it off is one of those things that is weirdly difficult though, I think. Same with coming up with examples. So well done!
I feel a bit like the title is a bit off. I might actually suggest a lot of what is said here is about Getting Grounded, rather than staying grounded. Staying Grounded implies one was grounded in the first place.
Great post though and I think it points to a number of examples one might consider and compare with their own life to see if perhaps one is as grounded as they think.
One concept that comes to mind is Lost Purposes. Maybe it explains why people fail to stay grounded.
Eg. maybe someone who pursues Good Grades initially does so because it is a means to the end of being Satisfied With Life, but somewhere along the way they start enjoying the feeling of getting Good Grades, and simultaneously forget that the end goal they had in mind was being Satisfied With Life.
I think this hypothesis is similar to some hypotheses you mention in the post, but is subtly different. Maybe.
I suspect that the Lost Purposes hypothesis is related to being Posioned By Ambition. When you pursue something hard, even if you logically understand that it is a means to some distinct end, your brain becomes convinced that it is an end in itself, and produces emotions accordingly. I further suspect that Staying Grounded is largely about preventing this from happening.
I wish I knew more about how to successfully do so. Saying "Fuck The Symbols" sounds somewhat promising. It's sorta like demonstrating to your brain, "See look! I don't actually care about this Symbol!". IIRC, there's evidence that your brain kinda observes your behavior and uses it to make inferences about what you care about, if that makes sense. But I'm not optimistic. I get the feeling that Staying Grounded is a hard problem to solve.
Make a point of not engaging in the standard academic research performances, and figure out how to cure cancer anyway.
I hear this a lot, but it doesn't seem to be very effective for people who do genuinely want to improve the world in tangible ways through academic research. The direction in fields like math and physics seems to be overwhelmingly in the direction of massive collaboration, and Symbols can be effective ways of organizing collective action on a hard problem. People joke about spending hundreds of dollars going to conferences just to spend the whole time falling asleep at talks and having coffee with strangers -- but everyone also has a story about the great collaboration that came out of a random hallway conversation at a conference.
That being said, there are some high-profile researchers who choose to communicate outside of papers and conferences. Chris Olah's blog and Grant Sanderson's YouTube channel come to mind. However, these all seem to be people for whom changing the way academics communicate is part of their core mission. For someone who just wants to do genuinely impactful research, I don't know if I can recommend "Fuck The Symbols."
The direction in fields like math and physics seems to be overwhelmingly in the direction of massive collaboration, and Symbols can be effective ways of organizing collective action on a hard problem.
Counterargument: both of these fields seem to have started producing actually-useful things at a much slower rate around the same time the shift toward large collaborations happened. Some people claim that the problems became harder in a way which required more people, but this does not seem more likely than theories in which the causal arrow goes the other way or is thoroughly confounded. There does seem to have been a more general cultural shift involved, and it's not clear that the large collaborations are actually better for progress as opposed to better for publishing high-profile papers.
Language is an extension and a more developed form of the drawings on the cave walls.
An example of social status without any attempt to provide anything of real substance: https://old.reddit.com/r/AskReddit/comments/p47gki/what_do_you_consider_the_biggest_threat_to/h8woqs7/
What does it mean to “stay grounded”? Why does it matter? How does one stay grounded?
Let’s start with a bunch of examples of grounding failures - i.e. people failing to stay grounded.
Note that these “failures” are not necessarily unintended/unwanted. Lots of hedge funds stay afloat mainly by bringing in investors, despite never beating the market. And hiring Brilliant People is a great way to look good to investors, regardless of whether they manage to beat the market.
In particular, many of the examples involve strategies which are good for winning social status, just not for whatever they’re nominally about. Graduates From The Best Schools or Volunteering to End Hunger or Publishing Papers or eating Good Food or Raising Awareness are all good ways to win social status, even in cases where they don’t actually help much with their nominal purpose. Humans often act as though social status is their “real” subconscious motivator, but they self-deceive about it.
In each of the examples above, some Symbols (capitalized in each example) have decoupled from what they represent. The Symbols are “ungrounded”, in the sense of the symbol-grounding problem.
There’s some socially-recognized Symbol of the thing - sometimes something which actually does help with the thing, sometimes a generic status symbol, but always something which people associate with the thing. Publishing Papers is a symbol of scientific progress, Good Grades are a symbol of a successful life in the making, Nice Clothes are a symbol of generic high status. People spend time and effort and resources on the Symbol, but obtaining the Symbol isn’t always sufficient to get them the thing. Publishing Papers isn’t enough to cure cancer, getting Good Grades isn’t enough to lead to a satisfying life, buying Nice Clothes isn’t enough to make one look good or feel comfortable, etc. Even if the Symbol can help somewhat, people often continue to pour resources into the Symbol long after it has ceased to be a bottleneck to obtaining the thing.
Another way to put it: grounding failure means Goodharting on the Symbol, and failing to actually get the thing as a result.
If your main goal is in fact to accumulate social status (regardless of whether you explicitly acknowledge this fact), then the situation is a bit different. The Symbols are far more important to your objective than the things they nominally represent; welcome to Simulacrum 3. So focusing on the Symbol over the thing is not a failure at all; the Symbol is what you actually want, on some level. If the suggestions in the next section about how to stay grounded sound aversive, unpleasant, or like they’re not really what you want, then consider that you may actually subconsciously want the social status more than the thing. If that’s you, then don’t worry about staying grounded - that’s not the game you’re playing.
But for those of us who are more interested in the thing than the Symbol, how can we stay grounded?
The most reliable way I know to avoid grounding failures is to say “Fuck The Symbols”. Make a point of not pursuing the Symbols, and try to obtain the thing anyway.
Make a point of not engaging in the standard academic research performances, and figure out how to cure cancer anyway.
Make a point of finding ways to be happy without doing the things which happy people do in commercials.
Make a point of not going out of your way to get good grades, and find a profitable/rewarding career path anyway.
Find ways to build a profitable business without Ivy League MBAs.
This isn’t always the best choice, but it’s usually worth at least thinking about how to do it - because the process of thinking about it forces you to recognize that the Symbol does not necessarily give the thing, and consider what’s actually needed.