Just listened to the talk and I'm confused about what people think Dario said. The only mention I noticed of RSI at the end was when the moderator said: "Last question. What will have changed in a year."
Dario said, paraphrasing, that the most important thing to watch this year is AIs building AIs, which could lead to wonders or a great emergency in front of us if it turns out that works and causes a meaningful speedup.
Is that what you're talking about? I'm pretty sure I agree that this is one of the most important things to pay attention to in the next year.
What percentage of humans do you think will be capable of providing supervision to AGIs?
This is adorable!
What underlying model are you using?
What's the advantage of the extra doors versus having a sliding door? One advantage of a sliding door is that it's a lot easier to get something really big in through the side.
Fair enough. My goal was to explain the GDM position, not to defend it. I agree it would be better to screen out anything with the canary string (but not sufficient to prevent eval contamination).
"AI is going to move a lot of things to lower levels of friction. That is by default bad..."
Did you mean by default good?
This is super fun! Thanks for putting it together.
It seems pretty unlikely that there should be two things on the list at bigness 9 (one of the most important discoveries of all time), and 6 at 8 (tech of the century) from a single year. Especially since the p(generalizes) is around .6 for a bunch of them. I feel like you might be systematically biased upwards in size of impact at least at the top end.
This was fun to read through!
I would rewrite the conclusion of the proof as follows, curious to see if you would endorse this:
Either there are no perfect properties, or God exists.
To me, this makes it much more clear that this is a pretty continent claim on there being perfect properties. I realize it was just one of a list of axioms, but it seems like a natural one to be skeptical of if you were trying to translate this into the real world.
Nit: a bachelor is an unmarried man.
Does Dario's new essay make you feel better, or worse?