This is a special post for quick takes by anaguma. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Comment Permalink
Showing 3 of 20 replies (Click to show all)
No I don't think that's right. I think it is a lot lower. I mean, I haven't trained looped transformers much, but my rough computation is like
- You scale up depth L times in a transformer. Usually you keep fixed aspect ratio, so you make the residual stream / mlp / attn all L times as wide. This makes all the matrices have L^2 more parameters. And since there's L times as many of them, we now have L^3 parameters. And when we scale transformers, we scale data as well, chinchilla optimally says proportionally, which would give like, every factor of x increase
1arielb1
Reading some papers, I could not actually find a reference that says that a looped model outperforms its “weight-untied” variant for any given amount of training tokens.
The advantage papers talk about is an increase in memory efficiency since you can use a weight multiple times, not an advantage in training data efficiency compared to an “untied” model. In the Vladimir Nesov evaluation strategy I don’t think it even reduces latency for frontier models unless you can keep a layer or two in fast memory, which I don’t think you can (or can you? I don’t know GPUs well enough for that).
4James Payor
Right so I had rushed to assume that they'd changed the latent information flow, otherwise I'd be surprised if it was just weight tying or adaptive loop count that spawned some leak about CoT becoming unmonitorable.
Imo Jakub has specifically not ruled out this kind of thing, late latents passed forward for earlier reads, that happens naturally for a bunch of the looped transformer variants, including the one Noosphere links.
I tried to clarify in this thread with Micah but didn't get a full answer yet I think: https://x.com/MicahCarroll/status/2095031872321647063
In worlds where it is on the benign side of looping (and CoT difficulties/changes are attributed to elsewhere) I hope they make it clear. Currently still stuck in my "this is Bayesian evidence for fuckery" interpretation.