(Aspiring) Existential risk researcher | President Effective Altruism Groningen


Being the (Pareto) Best in the World

This was very informative!

How would you translate this into a heuristic? And how much do I need to have a secondary skill, rather than finding a partner that has a great complementary skill?

A case for strategy research: what it is and why we need more of it

I am not sure why you believe good strategy research always has infohazards. That's a very strong claim. Strategy research is broader than 'how should we deal with other agents'. Do you think Drexler's Reframing Superintelligence: Comprehensive AI Systems or The Unilateralist's Curse were negative expected value? Because I would classify them as public, good strategy research with a positive expected value.

Are there any specific types of infohazards you're thinking of? (E.g. informing unaligned actors, getting media attention and negative public opinion)

A case for strategy research: what it is and why we need more of it

I agree with you that #3 seems the most valuable option, and you are correct that we aren't as plugged in - although I am much less plugged in (yet) than the other two authors. I hope to learn more in the future about

  • How much explicit strategy research is actually going on behind close doors, rather than just people talking and sharing implicit models.
  • How much of all potential strategy research should be private, and how much should be public. My current belief is that more strategy research should be public than private, but my understanding of info hazards is still quite limited, so this belief might change drastically in the future.

To respond to your other questions:

  1. Are there enough people and funding to sustain a parallel public strategy research effort and discussion?

I am not sure whether I get the question: I don't think there is currently enough people or funding being allocated to public strategy research, but I think there could be a sustained public strategy research field. I also think there is not a high threshold for a critical mass: just a few researchers communicating with an engaged audience seems enough to sustain the research field.

  1. Are there serious info hazards, and if so can we avoid them while still having a public discussion about the non-hazardous parts of strategy?

Yes, there are serious info hazards. And yes, I think the benefits of having a public discussion outweigh the (manageable) risk that comes with public discussion. If there is a clear place for info-hazardous content to be shared (which there is: the draft-sharing network) and when there is a clear understanding and policy for limiting info-hazards (which can be improved on a lot), public discussion will have at least the following advantages:

  • Exposure to wider array of feedback will, on expectation, improve the quality of ideas
  • Outsiders have more accessible knowledge to learn from to contribute later. There are probably also a lot of benefits to be gained from making other people more strategically savvy!
  • It makes it easier for non-affiliated/less-connected individuals to create and share knowledge
A case for strategy research: what it is and why we need more of it

I'm not sure I understand what Allan is suggesting, but it feels pretty similar to what you're saying. Can you perhaps explain your understanding of how his take differs from yours?

I believe he suggests that there is a large space that contains strategically important information. However, rather than first trying to structure that space and trying to find the questions with the most valuable answers, he suggests that researchers should just try their hand at finding anything of value. Probably for two reasons:

  1. By trying to find anything of value, you get much more rapid feedback on whether you are good at finding information than by taking a longer time to find high-value information.
  2. When there is a lot of information acquirable ('low-hanging fruit'), it doesn't matter as much where you start, as long as you start quickly.

In addition, he might believe that fewer people are good at strategy research than at tactics or informing research, and he might have wanted to give more generalizable advise.