neo's Shortform

neo

This is a special post for quick takes by neo. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

What are some research directions for "improving coordination?"

In light of a recent post and comment, and several months of thinking, I have come to the position that one of our (humanity's) biggest problems is that we suck at precise coordination at every level.

This is not very specifically defined but I am trying to gesture at a problem area I think is super important. Some thoughts to convey my intuition here:

If the extreme risk of the AI development trajectory is as true and obvious as many believe (everyone's life at risk), humanity's thinking about it should appear a lot more sophisticated than it does now.
For the last few years Eliezer has basically been throwing his hands up in exasperation at the incompetence of the world and many have shifted to public-facing communication, presumably believing that trying to convince AI insiders is hopeless.

Broadly, I think there are two cases of problems with coordination:

Two people/groups genuinely agree to honest, rigorous exchange of information, but can't effectively coordinate.
Someone is withholding information or doesn't really want to coordinate in the first place.

I think the first problem is workable, and if improved sufficiently, makes progress on the second problem by clearly exposing parties that are avoiding productive exchange.

Specifically, I think there is a lot of progress to be made with augmenting the exchange of information between people. I think LessWrong, the knowledge commons arguably at the frontier of ensuring humanity's survival, is lacking in features for this purpose. Maybe because most users here are already conscientious and strongly value truth-seeking, which makes improvement seem less necessary.

Hopefully I'm making this line of thought clear enough. Key points:

Trustworthy, robust, and future-proof governance is the ultimate problem for humanity and anything else is a band-aid on a bullet hole.
Highly effective coordination is part of that problem, and better information exchange/clarity is a subset of that.
- I think LessWrong can become an exceptionally effective prototype of this, and this could be very high leverage because of how proximate it is to the frontier of AI. Happy to expand more here.

I am interested in situating my thinking better here. Who is working on this sort of thing? I know TsviBT has explored improvements to debate, Richard Ngo / Samo Burja are exploring broader political manifestations, Forethought has published adjacent work. Is there anything I'm missing? Very interested in contributing here and think it's a clear place where the ball is being dropped.

Apologies for the delayed response.

I should note that the post was somewhat hastily written – I agree that my categorization was not comprehensive, and yours is probably better.

I was mainly trying to point at a dynamic I see often online where influential voices present arguments regarding AI risk that completely ignore years of back-and-forth discussion on similar topics, but whose positions are interpreted as the "forefront" of the debate – leading to offshoot discussions that again, miss years of relevant literature and discourse. I think this leads to,... (read more)

9Kaarel8mo

Here's a bunch of stuff off the top of my head, in no particular order, including people who aren't thinking much about the issue in full generality, but are addressing aspects: [1] * Daniel Schmachtenberger * Michael Vassar * ACS * economics has the subfields of social choice theory and mechanism/incentive/institution design. public economics is also relevant. internalizing externalities * there’s a bunch of econ stuff on people coordinating in/as a firm * there is a lot of political philosophy/theory/science on what sorts of political institutions we ought to have. eg see here for a bunch of pointers to contemporary thinking on sortition, or see communist proposals for how we should coordinate, or see anarcho-capitalists proposals * there are groups trying to reform the US voting system * various groups are trying to get money out of politics, eg trying to get Citizens United v. FEC overturned. there are various anti-corruption groups and pro-transparency groups * there have been various attempts to establish a world government * the legal system is one of the main instruments society has for acting on its values and determining facts in specific cases. there’s a lot of work on what it should be like * i think a bunch of sociologists are studying polarization and social media echo chamber stuff * there’s a bunch of work on how to inform people / how to get people to pay attention / how to get people to believe something. eg advertising research, work on how to run propaganda campaigns, theory of journalism * there’s a bunch of work on how to make people able to understand stuff: education theory, designing curricula, teaching * retroactive funding, impact markets * there are various forecasting and (specifically) prediction market initiatives, eg metaculus and manifold * people who created and run twitter community notes, fact-checking in general * people running wikipedia * people running the alignment forum and lesswrong * work on reputation

6Mateusz Bagiński8mo

Also Game-B. I think Jan Kulveit also thinks about this as a part of the theory of change of FABRIC (?).

neo's Shortform

2

neo's Shortform

2

What are some research directions for "improving coordination?"