Haiku — LessWrong

Doom as a bad method not a utopia trade-off

That makes sense; it was easy to read your comment as applying to individual actors, so I appreciate the clarification. (And for extra clarity on my part, I wasn't meaning to making an accusation against your character even in my initial reading of your statement, which is analytical and neutral. My comment was very pointed, but was not intended to be pointed at you.)

Doom as a bad method not a utopia trade-off

Haiku5d87

That makes sense for personal risk, but when it comes to other people, it's not your call. Unilaterally and knowingly exposing the entire world to extreme amounts of additional risk is deeply immoral, of the "doesn't believe in consent" variety.

testingthewaters's Shortform

Haiku7d76

It doesn't have to be up to them, and by no rights should it be. If the US government understood what was happening, OpenAI et al would be shut down overnight.

I'm just some guy, and one of many doing grassroots lobbying about AI extinction risk. And yet, if there had been just 200 more people having my very modest level of impact on US lawmakers over the past 2 years, I think we would have a global AI treaty today, or be very close.

Other countries have less impact, but far from zero! The impact of multiple middle powers formally recognizing that superintelligent AI is an extinction level threat, may be very close, and should be banned would be very substantial.

Ian McKenzie's Shortform

Haiku8d10

I think you can get people unstuck from that prior if you offer them a line of retreat toward an even bigger reason to hate the AI companies.

My current go-to snappy retort for use in low-quality discourse (e.g. YouTube comments) is: "Exactly. Just like how oil companies invented climate change to make oil sound cool to investors." (I don't have any evidence that this works, but I predict that little nudges and intuition pumps like this would work over time if enough people offer them.)

Ian McKenzie's Shortform

Haiku8d42

But then why doesn't Cal get into PauseAI?

Warning: Incautious psychoanalysis ahead

PauseAI is trying to solve a problem, and people like Cal aren't in the business of trying to solve problems.

It's useful to notice here that Cal Newport and Ed Zitron are very similar figures. Cal has expertise in non-AI computer science and can sometimes make a coherent argument, so he's the shiny variant. But realistically, neither of them have a clue about AI, and they both give off an aura of religious zeal that prevents things like "evidence" from having any effect on... (read more)

Let's talk about the AI coordination problem

Haiku9d32

The issue is a willful lack of curiosity about verification mechanisms, which are viable in the very near term.

Daniel Kokotajlo's Shortform

Haiku10d1-2

I can imagine a scientific reason for doing so: learning more about how cyber capabilities scale, in order to get empirical data on the details of their emergence, in order to better design training runs that don't result in dangerous cyber capabilities.

(Though I really doubt this is what they were doing.)

Thomas Kwa's Shortform

Haiku16d1415

Why not just throw all of your effort into making RSI illegal?

Charbel-Raphaël's Shortform

Haiku16d31

Many of the people who claim "marketing hype" will continue to make that claim even if the companies are completely shut down by the government for safety reasons. ("They just don't want people to know that they failed; the government is covering it up.") I would be moderately surprised if ~anyone in a position of influence is making decisions based on that memetic disease. This especially applies to the AI companies, who benefit from a skeptical/dismissive public in terms of a lack of regulation.

neo's Shortform

Haiku16d75

I plan to keep my gas tank above half full. If "the hackening" does occur and it hits essential services, I would like optionality, rather than being stuck in a suburb of a metropolitan area in the middle of the desert.

I already stock water, and I'm beginning to look into emergency food supplies as insurance. (Thankfully, even the far-right lunatics selling bunker kits offer gluten-free options these days. That's genuinely very considerate of them.)

AI Tweets

Haiku16d31

Yep, that's the situation.

Like most people, she isn't strictly focused on maximizing expected value. (Which is fine! It's okay to be human in ordinary human ways! That's part of what I'm fighting to protect.) She seems to accept the arguments for extinction risk, but then compartmentalizes hard, and commits to being agnostic, saying "I don't understand those claims" about anything at the existential level.

She listens to me all the time and has been very supportive of my activism, so I haven't pushed for her to have more involvement, since I'm fortunate to ... (read more)

AI Tweets

Haiku18d123

Lots of people have plans to make the world better by improving X to improve Y to improve Z, where it's Z that matters. If Z is more than 3y out, consider finding a different plan.

This really hits on an emotional level, but I ultimately take a different perspective:

I think solving the relevant safety problems for any AI systems that could disrupt the world at this scale will be extremely difficult, and that most surviving worlds are ones in which AI progress has been ~halted via a long-term globally coordinated pause. That means the only futures I should e... (read more)

bpomo's Shortform

Haiku24d104

But I don't know where else to jump in.

Consider volunteering with an AI governance advocacy community. We can do a lot better then marginal technical safety work (most of which is just milling and justification, rather than actually tackling fundamental hard problems in AI safety). A global AI treaty is feasible.

The org I volunteer with (>100 grassroots meetings with Congressional offices so far this year, with many more coming soon): https://www.pauseai-us.org/

A sampling of other orgs with essentially the same mission: https://pauseai.info/ https://... (read more)

leogao's Shortform

Haiku1mo43

I don't think "applause light" is being used correctly here. Leo's claim is not devoid of content, it's just a claim that most people agree with.

Karl Krueger's Shortform

Haiku1mo337

https://x.com/blueblimpms/status/2086574031529247183 It turns out this allegedly happened in April, and Pangram says that the original blogpost is AI generated.

leogao's Shortform

Haiku1mo60

Waymo has a feature where you can pay a few bucks for the car to wait extra time, which you can send while the car is waiting for you. I've used it once when I slightly mistimed a ride hail from the airport. I think Uber and Lyft could implement this, but it's definitely lowest friction to implement when there is no driver and the fee is just set by the company.

Joseph Miller's Shortform

Haiku1mo93

Superintelligence safety isn't "far worse" in China. You can't get worse than zero superintelligence safety, which is what we currently have in both the US and China.

Arguments for P

Haiku1mo10

That argument for not-P doesn't account for observation O.

In general, in my all-things-considered forecast, I expect that there is a significant chance of P. I think the basis for P is pretty clear at this point given the empirical data, and I believe these earlier arguments for P were correct. P is also just a direct extrapolation from R, so this ultimately shouldn't be surprising. We've known P for a long time now, frankly.

A subset of the broader community is in favor of P, but I think that this is bad of them and that P is evil. Join me in calling your political representatives about P.

Existential Risk from AI: An Exposition for Mathematicians

Haiku1mo34

Being empathetic and being nice aren't always the same thing. Loving actions are sometimes adversarial. When I imagine myself in the same position, I would want a friend to confront me and demand to know what the hell I'm thinking.

"We are doomed and there is nothing to be done, so I might as well doom us harder" can be made psychologically unsafe. It's plenty of reason to end a friendship. They should feel pressure and shame for doing such a taboo thing, not to punish them, but because the lack of hard pushback acts as tacit approval, as if we can all be friends at the end of the day, as if friendship is more important than the death of all things.

Noah Birnbaum's Shortform

Haiku2mo20

This doesn't work if cognitive ability itself is the dangerous part. AI killing us via macroscale manufacturing is a plausible floor for the strangeness of doom, not the ceiling. In reality, if a superintelligence can communicate with anyone, that on its own is exceedingly dangerous. If it gains internet access, it already sees "checkmate in 4058," so to speak. It would need to be aligned before that point, which probably means it would need to be aligned before it finally gets good at philosophy.