I think Moskowitz's funding of malaria nets and Moskowitz's funding of AI policymakers - both come from the same mistake of thinking like a billionaire instead of thinking like a politician.

The actual bottlenecks to fixing both African politics and US-China geopolitics around AI are not money but political leaders who are skilled in public persuasion, coalition building, finding successors, inventing ideology, etc

I am glad you atleast recognise the benefits of open source.

My preference order is:

• For capabilities of AI labs: Ban AI > Open source AI > Closed source AI
• For values and decision-making processes of people running AI labs: Open source (i.e. publicly publish) everything

As you say, I think open source today will atleast help build the proof required to convince everyone to ban tomorrow.

I think we should go further, and instead of hoping a benevolent leader to livestream the lab by choice, we should incentivise whistleblowers and cyberattackers to get the data out by any means necessary.

See also: Whistleblower database, Whistleblower guide

I like that you're discussing the question of purpose in a world where intelligences way smarter than you are doing all the useful knowledge work, and you are useless to your civilisation as a result. The frontier intelligences might have purpose (or they might not even care if they do), but you might not.

• I was disappointed by Bostrom's answer of "let's just create more happy beings".
  • Also, what makes those beings happy in the first place? Have the purpose of making even more beings? I don't actually care that much about populating the universe with beings with self-replication as their only purpose.
  • His entire book "Deep Utopia" seems to not provide a good answer.
• I was also disappointed by Yudkowsky's answer of "let's ask the AI to run a gazillion mind simulations and have it figure it out on our behalf".
  • This answer might be a good answer, but it is too meta, and does not tell me what the final output of such a process will be, in a way I as a human from 2025 can relate to.
• I like Holden Karnofsky's take on why describing utopia goes badly. I was not very satisfied with his answer to what utopia looks like either.
  • What is the use of social interactions if there's no useful project people can do together? Producing art and knowledge will both be done better by frontier intelligence, anything these people produce will be useless to society.
  • For that matter, why will people bother with social interactions with other biological humans anyway? The AI will be better at that too.
  • (Mandatory disclaimer - I think Karnofsky is accelerating us towards human extinction, me endorsing an idea of his does not mean me endorsing his actions.)

I support more advancements in cyberhacking capabilities so that companies and govts are incapable of keeping secrets. Secrecy enables them to act against the wishes of the majority to an extent that couldn’t otherwise.

Can you give an example in the real world? (Prefer historical examples if you dont wanna be too controversial) Both your comments are abstract so I'm unclear what you have in mind.

In some hypothetical game theory puzzle sure. In the real world it does necessitate it with like >95% probability.

And here we are talking about positive sum stuff like growing a business.

Pause AI movement is explicitly a zero sum political battle.

Positive sum games still involve a lot of zero sum moves! Just because the pie is growing doesn't mean it doesn't matter who gets more of the pie. If you are a company CEO in a growing industry, you will end up taking adversarial moves against lots of people. You will sue people, you will fire your employees, you will take away profit from your competitors if you succeed, and so on.

1. The situation is fundamentally adversarial. People want different things and are willing to go to extreme lengths to get it.

2. I think my statement is true of basically every major political or economic change in human history.