Nice post! As someone who spends a lot of time in AI policy on strategic thought and talking to people who I think are amongst the best strategic thinkers on AI, I appreciated this piece and think you generally describe the skills pretty well.

However, you say "research" skill by default does not lead to strategic skill, which is very true, but this varies drastically depending on the type of research! Mechanistic interpretability, in fact, appears to me to be an example of a field which is so in the weeds empirical with good feedback loops, that it makes it much harder for researchers in this field to learn better strategic thinking. Other research fields with slower feedback loops are different—for example, societal impacts of AI research. More broadly, I think many fields of social science train strategic skill well, and some of the best political science thinkers clearly have significant strategic skill: Fukuyama, James C. Scott, Dominic Cummings, etc.

I made an attempt to brainstorm ways to evaluate strategic skill based on the abilities of the best thinkers I know, and came up with a list of characteristics I think it is correlated with:

• awareness of both the state of and fundamental principles behind politics and power. A recent example I was discussing with friends: if you read the transcript of the leaked Yemen signal group chat and could immediately write a 2 page doc with likely guesses of the power dynamics between the participants implied by their communication and roles, you probably have some of this awareness.
• scenario-based thinking: the ability to cleanly split the future into distinct scenarios, reason about each independently, effectively simulating the different actors involved (which makes me wonder how much one could accelerate this with LLMs, c.f. simulator theory), before comparing different scenarios based on a small number of key drivers.
• zooming in and zooming out: an ability to not get lost in the details, and put their focus where it is useful, whilst still being aware of the details or having the skill to dig in if necessary.
• generalist knowledge: their skills are usually T shaped, rather than I shaped. They may have read widely across law/econ/political theory/philosophy/sociology/international relations/evo psych or may have background in different worlds (finance, startup ecosystem, defense contracting/intelligence community, etc). This generalism helps in many ways, but one of the biggest is simply that because much of the thinking in any close intellectual community is highly correlated, since everyone reads the same things, you need some source of fresh signal to come up with significant new insights.
• an aptitude for the skill of translation, meaning translation of information and beliefs between frames/perspectives/ways of thinking—something that heavily overlaps with an ability to shape and tell believable narratives. This is the core skill you need to explain, for example, why compute is a useful lever for governing AI to politicians who don't know what a GPU is, but is also very useful to consider and discard lenses & framings on strategic problems until you can find the one which most cleanly carves up the situation.
• an ability, similar to startup founders, to tolerate everyone thinking they are wrong (correlated with disagreeableness). Often, an exceptional strategic insight backchains to actions which look like they have a weak theory of change to everyone else, which means that good thinkers are often in the position of having to explain themselves.
• context: having both the ability (through reading widely) and position (i.e. access to the thinking of significant figures in the field) to be high context on the big strategic problems. Context is that which is scarce (Cowen). I say context instead of seniority, because they are not quite the same, although strongly correlated—I occasionally meet a number of junior people who clearly are unusually high context for their experience level and have good strategic takes, which is one of the ways I spot promising talent.

Finally, I do notice a lot of those I think have the best strategic thought often use lenses and framings inspired by systems thinking, social evolution/selection processes, memetics, biology, and other similar ways of viewing society and human behavior.

Curated. I think this is a pretty important point. I appreciate Neel's willigness to use himself as an example. 

I do think this leaves us with the important followup questions of "okay, but, how actually DO we evaluate strategic takes?". A lot of people who are in a position to have demonstrated some kind of strategic awareness are people who are also some kind of "player" on the gameboard with an agenda, which means you can't necessarily take their statements at face value as an epistemic claim. 

Also, if you're asking a panel of people, even those skilled at strategic thinking will still be useless unless they've thought deeply about the particular question or adjacent ones. And skilled strategic thinkers can get outdated quickly if they haven't thought seriously about the problem in awhile.

Hey Neel, I've heard you make similar remarks informally at talks or during Q&A sessions in past in-person panels and events, and it's great that you've written them up so that they're available in a nuanced format to a broader audience. I agree with the points you've made, but have a slightly different perspective on how it connects to the example of people asking for your strategic takes specifically, which I'll share below (without presumption). 

TL;DR: "Good strategic takes are hard to measure; but status is easy to recognize".

I. Executive Summary

People aren't necessarily confusing research prowess with strategic insight. Rather, they recognize you as having achieved elite social standing within the field of AI more broadly and want:

1. Access to perceived insider knowledge
2. Connection to high-status individuals
3. The latest thinking from those "in the room"

II. Always Use The Best Introduction

Before reading this post, I believed that the median person asking these questions was motivated by your impressive academic performance during your undergraduate studies, something that can be (over)simplified to "wow, this guy studied pure math at Cambridge and ranked top of his class, he's one of the smartest people in the world, and smart people are correct about lots of things, he might have a correct answer to this question I have!". I'm quite embarrassed to admit that this is pretty much what was going through my head when I attended a session you were holding during EAG last year, and I wouldn't be surprised if others there were thinking that too.

Similarly along those lines, I recall reaching out to one of your former mentees for a 1:1 thinking, "wow, this guy studied computer science at Cambridge and ranked top of his class, he's one of the smartest people in the world, and smart people are correct about lots of things!". I also took the time to read his dissertation, and found it interesting, but that first impression mattered a lot more than it should have. An analogy is that when people are selecting the model to use for a task, they want to use the best model for that task. But if a model takes the top spot on the leaderboard where test scores are easy to measure, then that tends to mess with human psychology which irrationally pattern matches and assumes generalization across every possible task.

III. My Key Takeaway:

My key takeaway was that although that this winner-take-all dynamic may have played one factor, your model assigns more weightage towards the work you've done after graduating and pioneering the field of mechinterp.

IV. Credentials vs. Accomplishments

To be clear, founding mechinterp is a greater accomplishment than any formal credential. But even though teams of researchers at frontier labs are working on this agenda, it's not mainstream yet (just take a look at mechinterp.com), whereas the handle of "math/cs genius" is generic enough as a concept to be legible to the average person. The arguments in your post about research being an empirical science requiring skills not especially relevant to strategy are locally valid, but these points are the furthest thing from the mind of those waiting in line at conferences to ask what your p(doom) is. 

V. The Tyranny of the Marginal Spice Jar

Often the demands placed upon us by our environment play an instrumental role in shaping our skillset, because we adapt against the pressures placed upon us. I'm thankfully not in a leadership position where the role calls for executive project management decisions which require a solid understanding of the broader field and industry. I'm also grateful that I'm not a public figure with a reputation to maintain whose every move is open to scrutiny and close examination. I also understand that blog posts aren't meant to be epistemically bulletproof.

I think that it's true that when the people you speak with the most (e.g work colleagues or MATS scholars) ask you about your thoughts, their respect is based on the merits of the technical research you've published. And in general, when anyone publishes great AI research, then that does inspire interest in that person's AI takes. 

VI. Unnecessarily Skippable Digression Into Social Bubbles and Selection Effects

Your social circle is heavily filtered by a competitive application process which strongly selects for predicted ability to do quality research. This can distort intuitions around the prevalence of certain traits which are not as well represented in the common population. For example, authoring code or research papers requires to some extent that your brain is adapted for processing text content, the implications of which I haven't seen discussed in depth anywhere on lesswrong. If someone expresses a strong preference for reading above watching a video when both options are available, it's almost like a secret handshake, because so many cracked engineers have told me this that it's become a green flag. In this world, entertainment culture and information transfer happens from books, web novels, articles, etc.

There's an entirely separate world occupied by someone with the opposite preference, i.e wanting to watching a video above reading text when both options are available, an example secret handshake for that is when my Uber driver tells me that they're cutting down on instagram. I admit this is a shallow heuristic but it's become a red flag I watch out for indicating a potential vulnerability to predatory social media dark patterns or television binge-watching. It's not an issue of self-control, people in the first group need to apply cognitive effort to pick things up from videos, but might have difficulties setting aside an engaging fantasy web serial. Most treatment of this topic I've seen addresses the second group, which feels alienating to me, as if there's this ongoing dimorphism between producers and users of consumer software.

I'm typically skeptical of "high IQ bubble" typed arguments since they tend to prove too much, so I'll make a more specific point. I agree with you that within these groups, conflation between perceived research skills and strategic skill does occur. My (minor) contention is that I don't think that this particular mistake is the one being made by the average person asking a speaker about their strategic takes at the end of a talk.

VII. Main Argument: Research Takes?! What Research Takes?!!

Like, these sort of questions aren't just being fielded by researchers in the field, you know. Why do people ask random celebrities and movie stars about their takes on geopolitics? Are they genuinely conflating acting skill with strategic skill? What about pro athletes? Is physical skill being conflated with strategic skill too? Do you believe that if a rich heiress with no research background was giving a talk about AI risk, that no one in the audience would be interested in her big picture takes? It makes no sense. Other comments have pointed this out already, so I'm sorry about adding another rant to the pile, but there exists a simpler explanation which does a better job of tracking reality!

The missing ingredient here is clout.

Various essays go into the relationship between competence and power, but what you're describing as "research skill" can be renamed expertise. These folks aren't mistaking you for someone high in "strategic skill", instead they are making the correct inference that you are an elite. They want in on the latest gossip behind the waitlist at the exclusive private social where frontier lab employees are joking around about what name they'll use for tomorrow's new model. They're holding their breath waiting for invention and hyperstition and self-fulfilling prophecy. They want to know the story of how Elon Musk will save the U.S AISI and call it xAISI.

IX. Concluding Apologetic Remarks

I'm not sure if this was an aim for the above post, but it's an understandable impulse to want to distance oneself from scenes where it's easier to find elites (good strategic takes) than experts (good research takes), because there can be a certain culture attached which often fails to act in a way that consistently upholds virtuous truth-seeking.

Overall, I think that taking a public stance can warp the landscape being describing in ways that are hard to predict, and appreciate your approach here compared to the influencer extreme of "my strategic takes are all great, the best, and bigly" versus the corporate extreme of "oh there are so many great takes, how could I pick one, great takes, thanks all". The position of "yeah I've got takes but chill they're mid" is a reasonable midpoint, and it would be nice to have people defer more intelligently in general. 

Thanks for writing this post. I agree with the sentiment but feel it important to highlight that it is inevitable that people assume you have good strategy takes.

In Monty Python's "Life of Brian" there is a scene in which the titular character finds himself surrounded by a mob of people declaring him the Mesiah. Brian rejects this label and flees into the desert, only to find himself standing in a shallow hole, surrounded by adherents. They declare that his reluctance to accept the title is further evidence that he really is the Mesiah. 

To my knowledge nobody thinks that you are the literal Messiah but plenty of people going into AI Safety are heavily influenced by your research agenda. You work at Deepmind and have mentored a sizeable number of new researchers through MATS. 80,000 Hours lists you as example of someone with a successful career in Technical Alignment research. 

To some, the fact that you request people not to blindly trust your strategic judgement is evidence that you are humble, grounded and pragmatic, all good reasons to trust your strategic judgement.

It is inevitable that people will view your views on the Theory of Change for Interpretability as aithoritative. You could literally repeat this post verbatim at the end of every single AI safety/interpretability talk you give, and some portion of junior researchers will still leave the talk defering to your strategic judgement.

I completely agree on the importance of strategic thinking. Personally, I like to hear what early AI pioneers had to say about modeling AI. For example, Minsky's society of mind. I believe the trend of AI must be informed by the development of epistemology, and I’ve basically bet my research on the idea that epistemological progress will shape AGI