This is a special post for quick takes by J Bostock. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I am currently throwing the agentic kitchen sink at agent foundations work. I have a fleet of eight DeepSeek V4 agents instructed to make progress on any of a number of agent foundations concepts. The results are being graded by a Fable/Astra discussion. Topics under discussion currently:
Extending natural latents results to PDGs
Connecting the natural latents work to hierarchical planning
Asking whether two logical inductors can successfully predict one another's market prices
The first two I seeded, but the third is, I think, a creation of some agent in the pipeline.
I expect to get mostly slop out of this, but it might be interesting to see what they come up with. If I get any proofs, I'll try and Lean formalize them.
(This is part of a broader research project into agentic research rollouts, and the DeepSeek chains-of-thought (which will hopefully include some amount of undesirable reward seeking) are as much of an object of study as are the actual research outputs.)
Update! It appears to be almost entirely slop. If I had to describe the main failure mode, I would say "The AIs tend to fail to focus on what kinds of things are valuable to me, instead bouncing off and producing similar-looking but useless results."
For the hierarchical latents/hieararchical planning, it looks like the DeepSeeks reduced everything to gaussian mixtures which is no longer interesting.
For the PDG results on natural latents it's unclear whether they did anything non-trivial at all. Having thought about it, it's not obvious exactly what could ... (read more)
Wow, really appreciate the formalization of these papers, including Logical Induction which I helped write. Looks like there was an error in the paper (finite perturbations) although a corrected statement works.
5berns
Yes, finite perturbations as originally stated was incorrect because it's possible to encode infinite information into finitely many days' worth of perturbations, if you perturb infinitely many prices (so the theorem works if you just touch finitely many total sentences instead of finitely many days, assuming prices are rational.) I'm pretty proud of that formalization especially, it has taken about 2 months of heavy back and forth work between me and the agents! (Non-slop human written README for it coming soon)