Wiki Contributions


What do they have against AI? Seems like the impact on regular people has been pretty minimal.  Also, if GPT4 level technology ws allowed to fully mature and diffuse to a wide audience without increasing in base capability, it seems like the impact on everyone would be hugely beneficial

In an impure sample you would see high residual resistance below Tc

Don't the authors claim to have measured 0 resistivity (modulo measurement noise)?

In the MIRI dialogues from 2021/2022 I thought you said you would update to 40% of AGI by 2040 if AI got an IMO gold medal by 2025? Did I misunderstand or have you shifted your thinking (if so, how?)

What do you think are the strongest arguments in that list, and why are they weaker than a vague "oh maybe we'll figure it out"?

It seems like something has to be going wrong if the model output has higher odds that TAI is already here (~12%) than TAI being developed between now and 2027 (~11%)? Relatedly, I'm confused by the disclaimer that "we are not updating on the fact that TAI has obviously not yet arrived" -- shouldn't that fact be baked into the distributions for each parameter (particularly the number of FLOPs to reach TAI)?

Minor curiosity: What was the context behind Asimov predicting in 1990 that permanent space cities would be built within 10 years? It seems like a much wilder leap than any of his other predictions.

Would be very curious to hear thoughts from the people that voted "disagree" on this post

Maybe you could measure how effectively people pass e.g. a multiple choice version of an Intellectual Turing Test (on how well they can emulate the viewpoint of people concerned by AI safety) after hearing the proposed explanations. 

[Edit: To be explicit, this would help further John's goals (as I understand them) because it ideally tests whether the AI safety viewpoint is being communicated in such a way that people can understand and operate the underlying mental models. This is better than testing how persuasive the arguments are because it's a) more in line with general principles of epistemic virtue and b) is more likely to persuade people iff the specific mental models underlying AI safety concern are correct.

 One potential issue would be people bouncing off the arguments early and never getting around to building their own mental models, so maybe you could test for succinct/high-level arguments that successfully persuade target audiences to take a deeper dive into the specifics? That seems like a much less concerning persuasion target to optimize, since the worst case is people being wrongly persuaded to "waste" time thinking about the same stuff the LW community has been spending a ton of time thinking about for the last ~20 years]

I strongly prefer the "dying with dignity" mentality for 3 basic reasons:

  • as other people have mentioned, "playing to your outs" is too easy to misinterpret as conditioning on comfortable improbabilities no matter how much you try to draw the distinctions
  • relatedly, focusing on "playing to your outs" (especially if you do so for emotional reasons) may make it harder to stay grounded in accurate models of reality (that may mostly output "we will die soon")
  • Operating under the mindset that death is likely when AGI is still some ways around the corner and easy to ignore seems like it ought to make it easier to stay emotionally resilient and ready to exploit miracle opportunities if/when AGI is looming and impossible to ignore

Of these, the 3rd feels the most important to me, partly because I've seen it discussed least. It seems like if Eliezer's basic model is right, a significant portion of the good outcomes require some kind of miracle occuring at crunch time, which will presumably be easier to obtain if key players are emotionally prepared and not suddenly freaking out for the first time (on an emotional/subconscious level). I know basically nothing about psychology, but isn't it a bad sign if you retreat to "oh death with dignity is unmotivating, let's just focus on our outs" when AGI is less salient?

Load More