Cleo

Thanks for writing this up, I broadly agree. There's a framing from the AI measurement literature (Olteanu et al.'s position paper on rigor, NeurIPS '25) that I think validates and possibly strengthens the proposal: the proposed approach is more rigorous than the top-down one, if you stop equating rigor with methodological rigor.
The paper argues that the AI community's usual notion of rigor, roughly whether the methods are correctly chosen and applied, is only one of six facets. The others include epistemic rigor (is the background knowledge the work buil... (read more)

Two things I have observed - but keep in mind that I am an occasional reader, so I might be overestimating the prevalence of these issues:

• General tendency to prioritize documentation and busywork over doing actual work - this is very model-dependent, however.
• Pinning repeated mistakes on external factors and subsequently giving up on a task — e.g., assuming that a tool is broken because they are unable to use it correctly, or catastrophizing about data deletion when they repeatedly search for a file using the wrong name and can’t find it.