I think that getting clear feedback on how well one is doing, and how much one is progressing, tends to be somewhat hard in general, but especially when it comes to:
And I’ve primarily been doing big-picture/abstract research aimed at improving the world, compared to the counterfactual, from a longtermist perspective. So, yeah, I’m a tad in the dark about how it’s all been going…[1]
I think some of the best metrics by which to judge research are whether people:
I think this data is most useful if these people have relevant expertise, are in positions to make especially relevant and important decisions, etc. But anyone can at least provide input on things like how well-written or well-reasoned some work seems to have been. And whoever the respondents are, whether the research influenced them probably provides at least weak evidence regarding whether the research influenced some other set of people (or whether it could, if that set of people were to read it).
This year, I’ve gathered a decent amount of data about the above-listed metrics. But more data would be useful. And the data I’ve gotten so far has usually been non-anonymous, and often resulted from people actively reaching out to me. Both of those factors likely bias the responses in a positive direction.
So I’ve created this survey in order to get additional - and hopefully less biased - data, as an input into my thinking about:
But there’s also another aim of this survey. The idea of doing this survey, and many of the questions, was inspired partly by Rethink Priorities’ impact survey. But I don’t recall seeing evidence that individual researchers/writers (or even other organisations) run such surveys.[2] And it seems plausible to me that they’d benefit from doing so.
So this is also an experiment to see how feasible and useful this is, to inform whether other people should run their own surveys of this kind. I plan to report back here in a couple weeks September with info like how many responses I got and how useful this seemed to be.
[1] I’m not necessarily saying that that type of research is harder to do than e.g. getting students’ test scores up. I’m just saying it’s harder to get clear feedback on how well one is doing.
[2] Though I have seen various EAs provide links to forms for general anonymous feedback. I think that’s also a good idea, and I’ve copied the idea in my own forum bio.
If you’ve read anything I’ve written on the EA Forum or LessWrong, I’d really appreciate you taking this brief, anonymous survey. Your feedback is useful whether your opinion of my work is positive, mixed, lukewarm, meh, or negative. And remember what mama always said: If you’ve got nothing nice to say, self-selecting out of the sample for that reason will just totally bias Michael’s impact survey.
I plan to use people’s responses as inputs - rather than definitive answers - in my ongoing efforts to plan my career and improve in various ways. And I’ll combine these inputs with a lot of other inputs.
Thus, you shouldn’t feel that this is uncomfortably high-stakes, nor that you should only take the survey if you’re really confident in what you’d say. You can just provide any tentative thoughts you have, and I can be responsible for working out how much weight I should give them, whether and how they should affect my decisions, etc.
(This is a good division of labour, as I know more about myself and the context of my work than you do, but you have the advantage of existing outside of my swirling vortex of alternating imposter syndrome and overconfidence.)
I’ve also added a comment below with more info on why I’m running this survey, but you’re very welcome to answer the survey without reading that comment.
Update: See also my reflections on the survey responses I've gotten so far, and Should surveys about the quality/impact of research outputs be more common?
The Douglas Adams reference, not the Coldplay song. ↩︎