This is a special post for quick takes by Kaj_Sotala. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Similar to other people's shortform feeds, short stuff that people on LW might be interested in, but which doesn't feel like it's worth a separate post. (Will probably be mostly cross-posted from my Facebook wall.)
For me the math and CS research progress is a bigger update than the METR/Anthropic reporting… The ability to outperform large communities of human researchers at the task of designing and improving algorithms is a core part of an intelligence explosion, so I think that's a really important indicator.
This was also a bigger update for me, and it seems like a similarly-sized one, so perhaps we don’t disagree that much. I’m curious, though, if you are differentiating between kinds of algorithm improvement/design. It seems like AI improving AI R&D could fall anywhere between “improving dataset quality” to “coming up with an AIXI-like algorithm from scratch.” My own sense is that while AI systems are getting better at a subset of algorithm improvement/design it does not yet extend to the latter kind of thing, nor does it seem obvious that they will get better at this latter kind of thing, and I think this plausibly matters for RSI. I’m not sure if/where we disagree though. Is your claim that AI systems are already outperforming humans at all types of algorithm design, or that they are not outperforming us at the “designing AIXI-like algorithms from scratch” type but will soon, or that they are not outperforming us at this and won’t but that this doesn’t bear on RSI? (Or another option I didn’t list)?
Similar to other people's shortform feeds, short stuff that people on LW might be interested in, but which doesn't feel like it's worth a separate post. (Will probably be mostly cross-posted from my Facebook wall.)