CarolinaC
Message
Carolina, Researching digital minds @ Future Impact Group (grantee).
website: https://cleo.codes
9
3
Carolina, Researching digital minds @ Future Impact Group (grantee).
website: https://cleo.codes
Thanks for writing this up, I broadly agree. There's a framing from the AI measurement literature (Olteanu et al.'s position paper on rigor, NeurIPS '25) that I think validates and possibly strengthens the proposal: the proposed approach is more rigorous than the top-down one, if you stop equating rigor with methodological rigor.
The paper argues that the AI community's usual notion of rigor, roughly whether the methods are correctly chosen and applied, is only one of six facets. The others include epistemic rigor (is the background knowledge the work buil... (read more)
Two things I have observed - but keep in mind that I am an occasional reader, so I might be overestimating the prevalence of these issues:
Thanks for writing this up. I think some of the positions you argue against could be formulated more strongly, specifically:
2. “The model only generates assistant tokens, not user tokens.”
I think the more nuanced position you should argue against here is “The model is optimized to generate assistant tokens, not to predict user tokens, during post-training.”
You acknowledge user token masking and defer the response to §5,[1] but let me spend some time making the full point here. In assistant-only SFT, the loss on user tokens is masked, and during RL rewards ... (read more)