I am a longtime LessWrong and SSC reader who finally got around to starting a blog. I would love to hear feedback from you! https://harsimony.wordpress.com/

Wiki Contributions


Why I'm Optimistic About Near-Term AI Risk

I mostly agree with points 1 and 2. Many interacting AI's are important to consider, and I think individual AI's will have an incentive to multiply. And I agree that people will continue to push the capability frontier beyond what is directly useful, potentially giving AI dangerous abilities.

I think we differ on the timeline of those changes, not whether those changes are possible or important. This is the question I'm trying to highlight.

Longer timelines for dangerous AI don't mean that we shouldn't prepare for things going wrong; these problems will still need to be solved eventually. Ideally, some researchers would act as if timelines are really short and "roll the dice" on alignment research that could pan out in a few years.

But I'm arguing that the bulk of the field should feel safe investing in infrastructure, movement building, and projects over a 10-20 year timeframe. People considering pursuing AI safety should assume they will have enough time to make an impact. Finding ways to extend this horizon further into the future is valuable because it gives the field more time to grow.

Why I'm Optimistic About Near-Term AI Risk

Great point. I agree and should have said something like that in the post.

To expand on this a bit more, studying these specialized models will be valuable for improving their robustness and performance. It is possible that this research will be useful for alignment in general, but it's not the most promising approach. That being said, I want to see alignment researchers working on diverse approaches.

Prize for Alignment Research Tasks

Task: Identify How to Divide Up a Research Task Among Different Experts

Context: Many important projects are multi-disciplinary. Research can be accelerated by dividing tasks between individuals who specialize in different areas. However, it can be difficult to divide a project cleanly into sub-components and decide who is best suited to each part.

Input type: A problem statement, a list of people on a research team, and their skills.

Output type: A the list of team members each with an associated sub-task relevant to the overall goal.

Instance 1


Problem: Research how policy advocacy can improve the welfare of egg-laying hens


  • Alice, a legislative assistant in congress
  • Bob, a historian knowledgeable about the history of civil rights movements
  • Charlie, a poultry veterinarian


Alice: Research previous animal welfare legislation to create a blueprint for legislation which can pass in congress.

Bob: Identify the basis of a successful grassroots advocacy for animal welfare and determine what legislation would be popular with voters.

Charlie: Determine how conditions for egg-laying hens could most be improved.

Instance 2


Problem: Create a scalable technology to reduce surface transmission of pathogens.


  • Dina, an epidemiologist
  • Evan, a chemist
  • Fran, a biologist


Dina: Identify which surfaces contribute most to surface transmission of diseases.

Evan: Produce a set of antimicrobial compounds which can be added to a surface

Fran: Test the set of antimicrobial compounds produced by Evan for safety and effectiveness.

Prize for Alignment Research Tasks

Task: Suggest how to prioritize different tasks given a research goal

Context: When considering an open-ended research topic, it can be hard to identify the most important areas. In many situations, some considerations are vastly more important than others. In order to use researcher time more efficiently, it is helpful to identify which parts of a research project are most important.

Input type: A description of a broad research topic and sub-questions being considered for that topic.

Output type: An ordering of research questions by priority with justification.

Instance 1

Input: A study of different existential risks. How dangerous are asteroid impacts? How dangerous are cosmic rays? How dangerous are hurricanes? How dangerous is AI?


  1. How dangerous is AI? This is likely to be a near-term problem and should be researched first.
  2. How dangerous are asteroids? Asteroid impacts are unlikely, but could pose an existential risk.
  3. How dangerous are cosmic rays? Large cosmic ray bursts are exceedingly unlikely, but could pose an existential risk.
  4. How dangerous are hurricanes? Hurricanes occur regularly, are well studied, and do not pose an existential risk.

Instance 2

Input: Major causes of obesity. Effect of environmental contaminants, changing music tastes, higher spending on food, changes in sugar consumption.


  1. Environmental contaminants may play a role in the obesity epidemic, and this possibility has not been explored much in the literature.
  2. Higher food spending due to economic growth may explain rising obesity, and there is some research to suggest this is the case.
  3. Sugar consumption has fallen in recent decades despite rising obesity, but may explain some of the trend.
  4. Changing music tastes are unlikely to be connected to obesity.
Prize for Alignment Research Tasks

Task: Expand on a poorly-defined research idea

Context: Often, people have vague intuitions about what to research, but find it hard to flesh out that idea or are unable to communicate it to others. By clarifying an under-specified thought, researchers can quickly identify fruitful directions for research.

Input type: One sentence that vaguely defines an idea.

Output type: 1 to 3 paragraphs providing a clearer, more detailed explanation of the idea and some guidance on where to research further.

Instance 1

Input: The proliferation of drones as weapons seems bad.

Output: In Ukraine, bombs dropped from small drones have been used to destroy tanks and target military personnel. Some experts are worried that drones could be used to carry out assassinations or terrorist attacks. Drones are difficult to defend against and the fact that missions can be completed autonomously makes it hard to retaliate against the perpetrator. It is important to develop technology that can identify, track, and disable many malicious drones simultaneously.

Instance 2

Input: Immigration policy is relevant for international relations.

Output: Immigration policy can be a useful tool to improve international relations. For one, increased immigration between two countries can improve diplomacy by fostering connections between citizens and by increasing cultural awareness. Additionally, easy migration allows people to move away from oppressive regimes, incentivizing governments to treat citizens well in order to maintain their power. Research on how exposure to other cultures increases cooperation across groups would be valuable. Additionally, policies to increase global mobility can weaken oppressive regimes.

AI Will Multiply

I agree. I think it is pretty hard to cleanly divide something into a single agent or multiple agents in general. Practically, this means that it can be useful to model something using different frames (e.g. a single agent or as a collection of agents).

A common misconception about the evolution of viruses

I wrote about the literature around the evolution of virulence previously which people might find useful for finding references on the topic.

In short, it is complicated:

  1. zoonotic diseases (like COVID) don't follow a consistent pattern early in their evolution because they are so far removed from being adapted to humans as a niche.

  2. As long as there is a correlation between deadly-ness and ability to reproduce, diseases will face evolutionary pressure to become more/less deadly depending on the circumstances. This is strongly influenced by transmission mode (but isn't the only factor):

    a. Disease transmitted by close person-to-person contact -> disease can't make host too sick, or host won't come into contact with others -> disease evolves to cause milder infection.

    Example: HIV has relatively few effects on hosts for years before culminating in AIDS.

    b. Disease transmission doesn't require person-to-person contact -> disease can optimize other features of transmission such as producing many copies and using host resources -> disease evolves to cause severe infection.

    Example: malaria is spread via mosquitos and doesn't require person-to-person contact in order to spread, which might explain why it is such a debilitating disease.

Nice! Bryan Caplan did a "Theory and Practice of Oligarchical Collectivism Book Club" which I enjoyed as well:


[$10k bounty] Read and compile Robin Hanson’s best posts

Cool project! I was thinking along similar lines when I recently went through a bunch of his posts to find interesting ideas and collected them:


Whoever decides to work on this project up might be able to use this as a starting point.


I generally agree. It seems unlikely to me that Cryosleep will be developed or in use for very long before brain emulations or AI become dominant. But like you point out, a lot of the benefits listed here would apply to brain emulations too.

Even if it won't be useful for long, cryonics research seems like an important precursor to Em's. Developing tools to preserve/image the brain, determining which brain structures are important to preserve, and finding ways to upload organic minds will all be important.

Load More