This is a special post for quick takes by Linch. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
A skill I might be unusually good at is a specific form of "close reading" where I try to infer and build partial models of the world based primarily on what a writer says but not necessarily what they want me to believe.
Eg if I read a paper I try to make inferences based on what the methods and figures say rather than what the author seems to want me to believe. For example it's very clear to me that the primary interesting thing about the Kruger and Dunning paper is the approximate functional form between within-distribution perceived competence and actual competence, not all the weird metacognition hypotheses.
Or in the OAI July blog post I quickly inferred that their model is/was in active circulation internally and likely used substantially based on what they actually said, even though the post itself wants me to believe that the use is quite limited.
I dunno what this skill is called. It's sort of the skill other people seem to apply in narrow domains, for example lawyers studying evidential filings, finance people on corporate reports, or fanfiction authors on canon texts.
The skill is not the same as the Straussian/esoteric reading, since I'm not working with the author. It's also not the same as the maximally adversarial reading since I'm trying to make inferences based on real people rather than cartoon or theoretical adversaries.
The generalization seems really obvious to me but I now suspect most (intelligent) people lack it.
The AIs are bad at it too. At least, this seems to be a predictable pattern in their failure to be as useful at reading papers as one might expect from an intelligent machine that read almost every text written and up online, at the level of reading comprehension that these machines usually have. I often have to correct them on the difference between what one can reasonably infer from the underlying reality of what the paper can tell us about, vs what the paper is trying to say. The AIs are often very good at the latter, even when the authors are very unclear by human standards)