Claude recommended that VCG as a kind of inverse Shapley often works better in the real world by encouraging truth telling. As a bonus, it handles damages (as opposed to benefits) slightly more easily.
I'm going to note that this is a proxy conversation for recent current events and I'm not really going to respond with recent current events in mind but instead focus on the abstract social coordination issues.
I think there's two very different aspects here both of which are important, (a) attribution of causal relevance and (b) attribution of reward & punishment to shape behaviour. When looking at the fire example from the perspective of (a) it might seem prudent to look at the psychology and upbringing of the "fire" shouter, and how society prepares or fails to prepare people for handling emergency situations such as fires. From the perspective of (b) on the other hand, it is more reasonable to want to have simple and clearly understandable rules and procedures. An appropriate rule in the fire example might be to ban shouting fire in a theater, making the "fire" shouter fully liable for harm caused, and make theaters liable for maintaining certain standard procautions and procedures in case of fire.
more broadly getting at "how do people who disagree about what's true and what's good, cooperate?"
Just wanna flag that I think this is a very important question for people to be focusing on.
I think it's true. I don't have a super principled answer other than "just, try not to be totalized about it."
My view is closer to: be totalized about it, but:
- Really deeply understand that other people are not totalized about it.
- Recognize and learn from examples of other totalized people. Many have caused significant problems. Most have probably been crazy.
- Try to argue yourself out of the position that is making you totalized. Make sure you understand your own position well. Think about the nuances and details.
- Try to adopt and integrate other totalizing beliefs.
- Try not to be a jerk about it.
I think it is correct for the social-judgment-sphere to have an immune system against totalizing beliefs.
I mostly disagree. I think something between "people should be able to hold totalizing beliefs without becoming totalized" and "we are living in a poly-crisis! There are too many totalizing problems now to easily catalogue all of them! Everyone needs to get totalized and organized and coordinated NOW!"
There are too many totalizing problems now to easily catalogue all of them! Everyone needs to get totalized and organized and coordinated NOW!"
I think I both disagree with the first claim, and also am not sure how this could make any sense. The thing that "totalizing" means is that your worldview becomes about a single thing.
If we have a lot of problems that are "totalizing-shaped", like, you can't be totalized about all of them! You might be saying "society should have a lot of totalized people, because this is a good way to solve problems that are very 'all-or-nothing' shaped." But, if part of what needs to happen at the societal scale is to figure out solutions to many all-or-nothing problems at once, it seems surprising that what you want is "totalized people" instead of "people who specialize in the thing, but, are still tracking at least some of the other problems."
I also just think most of the other problems don't make sense to be totalizing. Which ones are you thinking of?
(In my view, the reason this is bad to be "totalized" is that, even as an individual, it's more likely you accidentally neglect things that you needed, to either be personally healthy, or to make your local social/professional world healthy)
I think it's likely I misinterpreted your original sentiment "just, try not to be totalized about it." to mean something like "don't become very dedicated towards important well integrated aims" rather than what you probably meant which may be something more like "don't become panicked and stressed about a poorly considered issue and adopt overly narrow strategies that neglect important considerations".
I don't think either of us is using "totalize" like the merriam-webster definition:
1: to add up : total
2: to express as a whole
I feel like I'm picking on your use of the word more than is valuable at this point. Sorry about that. I'll try to explain my use of the word a bit by the following responses, but it's probably not super important we get on the same page, so if it doesn't make sense feel free to ignore.
you can't be totalized about all of them!
I agree you can't be totalized about all of them individually, but you can be totalized about the set of all of them together.
You might be saying "society should have a lot of totalized people, because this is a good way to solve problems that are very 'all-or-nothing' shaped."
Yes. I think this is indeed what I am saying. Also that those totalized people need to be better at organization and coordination with one another.
if part of what needs to happen at the societal scale is to figure out solutions to many all-or-nothing problems at once, it seems surprising that what you want is "totalized people" instead of "people who specialize in the thing, but, are still tracking at least some of the other problems."
I think you are very correct that specialization is very important, but coordination between specialists is also important. Specializing in helping to integrate and cross reference. IE specializing in totalizing. But also, I don't think a totalized person must have a totalized career, more that they have totalized motivations.
As an example, I could become an EA and think that some cause, like preventing malaria, is important enough that it should consume all of my focus, and then conclude that I should specialize in law, make lots of money as a lawyer, and donate most of it to the against malaria foundation.
A totalized view of what is important does not need to imply one must become an expert in all subjects.
I also just think most of the other problems don't make sense to be totalizing. Which ones are you thinking of?
Any claim that has extremely important implications if it is true. This includes ASI risk, climate change, the holocene extinction, societal collapse, global resource management, nuclear war and more generally geopolitical instability, pandemic risk, basically everything Toby Ord talked about in The Precipice, basically everything to do with communication and coordination, and also on the tail end weird intractable and difficult to make progress on things like philosophy of ethics, religion, consciousness, immortality, etc...
it's more likely you accidentally neglect things that you needed, to either be personally healthy, or to make your local social/professional world healthy
I agree with you about that, and I think it is possible for totalization to cause a person to neglect the things you mention in ways that harm themselves and the cause they are trying to aid, but it seems more like a fact about people's abilities to strategize and manage their executive functioning, not necessarily like a fact about totalization.
I don't know how you would measure it but I would expect there are more people accidentally neglect things they need who aren't totalized than those who are. I would expect totalization might have weakly positive effects, giving people motivation to do the things they must do to support themselves and their cause. This might be a case of general advice failing and needing to be reversed for some people.
Deepfates on twitter wrote:
I tried to give it a serious answer, and I wanted to check in with The LW Folk if this answer made sense to others here.
ii. Shapley Shares
My actual answer, within frame, is "You have some share of responsibility." The exact share depends on the exact situation.
(I think the first order thing to be checking is "was there a fire, or not?". But, for now, answering the question as-asked)
I think people have some responsibility for the second-order effects of their actions, and "how much?" depends on, like, how second-order it was.
Something something "shapley value?" I realize shapley value is intractable to calculate in most cases, but, is suggestive of what shape of answer you'd get if you were omniscient
If you were the only person shouting fire and there was exactly one person who did the trampling, maybe (I'm not actually 100% sure how shapley value works) they both get 50% of the credit/blame (if it turns out that if you remove either of you, it wouldn't have happened)
If you're in a different situation where lots of people are shouting things like "fire" all the time. (for example, some people shouting "AI will kill us" and some people shouting "billionaires are evil, eat the rich"), then the blame is more distributed. Exactly how distributed depends on the circumstances.
The question "okay, what is society supposed to do about that?" needs to take into account what sort of norms are practical to enforce. There's a theoretic answer to 'who is responsible' and a practical 'who are we going to prioritize holding accountable.'
I think it's a correct norm to be "people who shout 'Fire' are responsible for putting effort into doing so in a way that mitigates the tailrisk side effects." For example "Guys there's a fire, please walk calmly toward the exits". But, if the people are not listening to the warning there is a fire, and there _were_ a fire... you'd probably rather the guy shout "Fire!" more emphatically than not do that.
He is maybe 50% responsible for the guy getting trampled in the narrow simplified case (the rest of the responsibility distributed among the panicking people). But, also he gets credit for saving the other people.
...
I realize this is a lot trickier when smart people disagree about whether there's a fire, or will be, and some of the arguments are quite complicated.
I don't think there's a shortcut to looking into the details of the particular case, which includes "was there a fire or not?" and "what else was going on?", etc.
ii. Cooperation among people who disagree on what is true and what is good
I think your more broadly getting at "how do people who disagree about what's true and what's good, cooperate?"
I think this is also kinda confusing and hard. But, listing out the obvious background parts of my viewpoint here (I expect this will not feel as novel to you).
a) ~Everyone probably agrees that 'fully evaluate every claim and the consequences of every action' is intractable, so we need simplified rules
b) Currently, we have a distinction between "what gets punished via governmental force (i.e. law) and what gets punished via social censure." I'm not sure if this is a natural carving, but, it seems pretty good.
The downstream consequences of arguing "there is a danger!" are chaotic enough I don't think it usually makes sense to be a thing that gets legally prosecuted. But on the meta-level, it seems fine for there to be some public arguments and tug-of-war around what gets social-censure.
But another layer of confusion here is "what do you do, when you believe something to be true, that implies a kind of totalizing worldview?". I think the memeplex around "ASI is on it's way and how it shakes out will determine the course of the future" is somewhat intrinsically totalizing.
I think it's true. I don't have a super principled answer other than "just, try not to be totalized about it."
I think it is correct for the social-judgment-sphere to have an immune system against totalizing beliefs. I also think the social-judgment-sphere needs to have a way of processing arguments that are pretty plausibly true, and that skew totalizing for many modern human psychologies.
(I think the way most people should engage with "AI might kill everyone or ruin the future" is, mostly, to call their senators a couple times and mostly get back to their lives)
I think it's correct, for that immune system, to pressure the people saying the totalizing-prone-thing to go out of their way to do an extra good job ameliorating the damage.
But, there are still limits to what's reasonable to expect there.
I also think it's also the responsibility of the rest of society to be tracking what people actually said, not merely blaming the guy yelling "Fire" but also the guys making up stuff about what the guy yelling "Fire" said, etc.
And, while this isn't true for arbitrary arguments, I think the threat of negative superintelligence honestly is clear and obvious enough (at least in magnitude, and the risk being nontrivial), that the rest of the social-sphere has the responsibility of taking the argument seriously. That's the other side of the handshake on "put extra effort into not being totalizing."
It's sort of necessary for messages to get simplified at scale. I already had "make sure whatever political process is happening is sane and reasonable and non-polarized" as a primary goal, but, I've updated in the past few days about the importance of having a clear succinct message that simultaneously conveys the gravity while also having some "and don't be crazy about it" energy.