As I understand it – with my only source being Ben's post and a couple of comments that I've read – Drew is also a cofounder of Nonlinear. Also, this was reported:

Alice and Chloe reported a substantial conflict within the household between Kat and Alice. Alice was polyamorous, and she and Drew entered into a casual romantic relationship. Kat previously had a polyamorous marriage that ended in divorce, and is now monogamously partnered with Emerson. Kat reportedly told Alice that she didn't mind polyamory "on the other side of the world”, but couldn't stand it right next to her, and probably either Alice would need to become monogamous or Alice should leave the organization. Alice didn't become monogamous. Alice reports that Kat became increasingly cold over multiple months, and was very hard to work with. (footnote) After this, there were further reports of claims of Kat professing her romantic love for Alice, and also precisely opposite reports of Alice professing her romantic love for Kat. I am pretty confused about what happened.

So, based on what we're told, there was romantic entanglement between the employers – Drew included – and Alice, and such relationships, even in the best-case scenario, need to be handled with a lot of caution, and this situation seems to be significantly worse than a best-case scenario.

I don't think "we're currently living in a simulation" or "ASI would have effects beyond imagination, at least for the median human imaginer" are such weird beliefs among this crowd that them proving true would qualify for OP to win the bet. Of course, they specifically say that if UAP are special cases in the simulation that counts, but not the mere belief in simulation.

Would you mind sharing how much you will win if the bet goes your way and everyone pays out?

Also, I would like to see more actions like yours, so I'd like to put money into that. I want to unconditionally give you $50; if you win the bet you may (but would be under no obligation to) return this money to me. All I'd need now is an ETH wallet to send money to.

I would like this to be construed as a meta-level incentive for people to have this attitude of "put up or shut up" while offering immediate payouts; not as taking a stance on the object-level question.

I hear you, thank you for your comment.

I guess I don't have a clear model for how big is the pool of people who:

• have malicious intent;
• access LessWrong and other spaces tightly linked to this;
• don't yet have the kind of ideas that my research could provide them with.

As soon as someone managed to turn ChatGPT into an agent (AutoGPT), someone created an agent, ChaosGPT, with the explicit goal to destroy humankind. This is the kind of person that might benefit from having what I intend to produce: an overview of AI capabilities required to end the world, how far along we are in obtaining them, and so on. I want this information to be used to prevent an existential catastrophe, not precipitate it.

Thank you for your post. It is important for us to keep refining the overall p(doom) and the ways it might happen or be averted. You make your point very clearly, even in just the version presented here, condensed from your full posts on varios specific points.

It seems to me that you are applying a sort of symmetric argument to values and capabilities and arguing that x-risk requires that we hit the bullseye of capability but miss the one for values. I think this has a problem and I'd like to know your view as to how much this problem affects your overall argument.

The problem, as I see it, is that goal-space is qualitatively different from capability-space. With capabilities, there is a clear ordering that is inherent to the capabilities themselves: if you can do more, then you can do less. Someone who can lift 100kg can also lift 80kg. It is not clear to me that this is the case for goal-space; I think it is only extrinsic evaluation by humans that makes "tile the universe with paperclips" a bad goal.

Do you think this difference between these spaces holds, and if so, do you think it undermines your argument?

I couldn't get things to work here, but thank you Elizabeth, Raymond and Ben for trying to help me! Have fun!

I'm thinking a few things that are perhaps not super important individually, but ought to have at least some weight in such an index:

Standardization and transportation

• What's the progress of adoption of the metric system?
• Relatedly, can we all (including Chile, where I live) ditch US paper sizes and switch to ISO sizes?
• Standardizing electric plugs and outlets, as well as domestic alternating current frequency and voltage
• Low priority, but probably still desirable if one wants a truly unified world: everyone driving on the same side of the road
• For rail transport: reduce variety in track gauges, electrification (which is different from residential use), and signaling systems. It is probably impractical to make *all* rail be standard gauge (1435 mm), as there are use cases where broad or narrow gauge is useful (e.g. in mountain railways there is often not enough space for standard-gauge tracks). But you could still say, for example: "okay, all new narrow gauge is going to be 1000 mm from now on", while converting railroads in countries like Portugal and Spain (1668mm) to standard.
• Aviation freedom: to what extent the https://en.wikipedia.org/wiki/Freedoms_of_the_air are granted between countries. A unified world would probably have something like all eight freedoms everywhere; countries could proceed unilaterally with this, granting foreign carriers parity with local ones in regulatory terms.
• Passport strength: how many countries can a holder of a given passport travel to without a visa? I am not sure how to weigh this with regards to the population of the issuing country, the destination countries, their GDP...

Legal cooperation/integration

• More countries acceding to treaties that are open to signature by all states: the UN Charter, the two Human Rights Covenants (the US hasn't signed the one on Economic, Social and Cultural rights and China hasn't signed the one on Civil and Political Rights; Saudi Arabia has signed neither), several specialized human rights treaties (e.g. Rights of the Children)
• International jurisdiction: expanding the jurisdiction of the International Court of Justice (which tries cases between states) and the International Criminal Court (which prosecutes crimes of genocide when national courts fail to do so); expanding and standardizing extradition treaties, enshrining caveats against the death penalty and requiring a fair trial, etc.
• International private law: expanding the Hague Conference on International Private Law and its Conventions, such as the Apostille Convention that facilitates recognition of foreign documents
• Trade and integration organizations have been mentioned, but I'd like to mention a few specific initiatives: the OECD, which groups the developed countries and the top developing ones (like Chile and Mexico), and the African Continental Free Trade Area (plus Regional Economic Communities)
• Cooperation in global issues like the Montreal Protocol, which bans ozone layer-harming chemicals, and global warming. Carbon taxes/tariffs probably should count toward this.

A caveat: while I've phrased all of these in a positive light, this does not preclude there being trade-offs. For example, expanding the freedoms of the air would likely boost air travel, which has bad environmental impacts.

AlphaGo used about 0.5 petaflops (= trillion floating point operations per second)

Isn't peta- the prefix for quadrillion?