This is a special post for quick takes by Daniel Tan. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I've been seeing takes that when models reach "drop-in remote worker" level in 6-12 months time, the majority (maybe all?) of human employees at labs will be replaced. This seems pretty implausible to me conditioned on no substantial improvements in fuzzy task performance, but I'd be interested in arguments to the contrary
Do you mean "substantial off-trend improvements in fuzzy task performance"? The default does seem like the models are getting better at fuzzy task performance.
hmm this is reasonable. I guess it hasn't felt (to me, based on personal usage) like there has really been improvement but I also guess there's no reason to think this isn't scaling (albeit slower than non-fuzzy tasks).
Automating creation of the RL envs that plug the capability gaps as they come up is the most important step in what I call prosaic RSI that automates most of the things humanity does, but might remain too slow at conceptual invention (with model-building being the speed bottleneck in the invention-learning-invention loop), and so the jobs of the humans specializing in conceptual invention might remain valuable for some time (also, this probably doesn't by itself cause a software singularity, for the same reason).
This hasn't happened yet because the models weren't smart enough. But Astra 6 is plausibly borderline smart enough to be taught the relevant skills (to automate RL env and task creation in response to observing contexts where it fails for comprehensible reasons), and then there's the after-Astra model (maybe a model targeting Rubin, so if it's available before spring 2027 or so, this will likely only happen at exorbitant prices, something like at least 5x the Astra prices; but then the prices probably come down to 2x the current Astra prices or lower).
After there's enough redundant capability (above a borderline-capable model), this becomes actually practical, and it's plausible the Navier-Stokes after-Astra model will be there. But it'll probably take another year after a model like this (with redundant capabilities) is available, until the prosaic RSI loop really starts going (and we start seeing major shifts in the labor market). So 2027 at the earliest, but probably 2028-2029, with more time after the first serious prototypes of 2027-2028 to turn into the actual broad automation of obscure skill whack-a-mole (or if the after-Astra model isn't sufficient to reach that redundant capability spot yet). But rapid pretraining scaling will continue until 2032 or so (at only a moderately reduced pace after 2027-2028), and the increases in scale-up system size that remain relevant for enabling bigger models will still be happening until 2031-2032 (until the m