I disagree.  In practice diffusion models are autoregressive for generating non-trivial amounts of text.  A better way to think about diffusion models is that they are a generalization of multi-token prediction (similar to how DeepSeek does it) where the number of tokens you get to predict in 1 shot is controllable and steerable.  If you do use a diffusion model over a larger generation you will end up running it autoregressively, and in the limit you could make them work like a normal 1-token-at-a-time LLM or do up to 1-big-batch-of-N-tokens at a time.

That's a great link, thanks!

Though it doesn't really address the point I made, they do briefly mention it:

> Interestingly, diamond has the highest known oxidative chemical storage density because it has the highest atom number (and bond) density per unit volume. Organic materials store less energy per unit volume, from ~3 times less than diamond for cholesterol, to ~5 times less for vegetable protein, to ~10–12 times less for amino acids and wood ...
> 
> Since replibots must build energy-rich product structures (e.g. diamondoid) by consuming relatively energy-poor feedstock structures (e.g., biomass), it may not be possible for biosphere conversion to proceed entirely to completion (e.g., all carbon atoms incorporated into nanorobots) using chemical energy alone, even taking into account the possible energy value of the decarbonified sludge byproduct, though such unused carbon may enter the atmosphere as CO₂ and will still be lost to the biosphere.

Unfortunately they never bother to follow up on this with the rest of their calculations, and instead base their estimate for replication times on how long it takes the nanobots to eat up all the available atoms.  However, in my estimation the bottleneck on nanobot replication is not getting materials, but probably storing up enough joules to overcome the gibbs free energy of assembling another diamondoid nanobot from spare parts.  I would love to have a better picture for this estimate since it seems like the determining factor in whether this stuff can actually proceed exponentially or not.

It's good to hear from an actual expert on this subject. I've also been quite skeptical of the diamondoid nanobot apocalypse on feasibility grounds (though I am still generally worried about AI, this specific foom scenario seems very implausible to me).

Maybe you could also help answer some other questions I have about the scalability of nanomanufacturing.  Specifically, wouldn't the energy involved in assembling nanostructures be much much greater than snapping together ready made proteins/nucleic acids to build proteins/cells?  I am not convinced that run away nanobots can self assemble or be built in factories at planet scales due to simple thermodynamic limits.  For example if you are ripping apart atoms and sticking them together in some new diamondoid configuration, shouldn't the change in gibbs free energy be sufficiently high that energy becomes a limiting factor?  If this energy is greater than what could be obtained from nuclear or solar power in some reasonable amount of time, it would rule out most "grey goo" nano-apocalypse scenarios.

My back of the envelope calculation is that there's about 10^20 moles of CO2 in the atmosphere, and it takes about 390 kJ to turn one mole of CO2 into a diamond.  The earth receives about 10^17 watts of power from the sun.  If we use all of that energy to make diamond bots as fast as we can, then it'll take thousands of years before even 1% of the atmosphere is converted to nano machines.

Granted there's a lot of unknown variables here, and my modeling is probably quite stupid, but I feel like some one must have considered these situations and come up with some way to roughly estimate how much energy would be required to turn the world into a diamond nanobot swarm to check if its even feasible given the energy available on earth (via sunlight or whatever).

My current gut feeling is that its probably more efficient at the end of the day to hijack existing biological materials and processes to build self replicating machines than using covalent bonds to resynthesize everything from scratch, but I don't really know enough to estimate that precisely.

This is the same kind of thing as the Black-Scholes model for options pricing.  As a prediction with a finite time horizon approaches the probability of it updating to a known value converges.  In finance people use this to price derivatives like options contracts, but the same principle should apply to any information.

I think you can probably put some numbers on the ideas in this post using roughly the same sort of analysis.