MichaelA

I’m Michael Aird, a Staff Researcher at Rethink Priorities, Research Scholar at the Future of Humanity Institute, and guest manager at the Effective Altruism Infrastructure Fund. Opinions expressed are my own. You can give me anonymous feedback at this link.

With Rethink, I'm currently mostly working on nuclear risk research. I might in future work on topics related to what I'm calling "Politics, Policy, and Security from a Broad Longtermist Perspective".

Previously, I did longtermist macrostrategy research for Convergence Analysis and then for the Center on Long-Term Risk. More on my background here.

I mostly post to the EA Forum.

If you think you or I could benefit from us talking, feel free to message me or schedule a call. For people interested in doing effective-altruism-related research/writing, testing their fit for that, "getting up to speed" on EA/longtermist topics, or writing for the EA Forum/LessWrong, I also recommend this post.

Sequences

Information hazards and downside risks
Moral uncertainty

Comments

Announcing the Nuclear Risk Forecasting Tournament

Not sure what you mean by that being unverifiable? The question says:

This question resolves as the total number of nuclear weapons (fission or thermonuclear) reported to be possessed across all states on December 31, 2022. This includes deployed, reserve/ nondeployed, and retired (but still intact) warheads, and both strategic and nonstrategic weapons.

Resolution criteria will come from the Federation of American Scientists (FAS). If they cease publishing such numbers before resolution, resolution will come from the Arms Control Association or any other similar platform.

FAS update their estimates fairly regularly - here are their estimates as of May (that link is also provided earlier in the question text).

Though I do realise now that they're extremely unlikely to update their numbers on December 31 specifically, and maybe not even in December 2022 at all. I'll look into the best way to tweak the question in light of that. If that's what you meant, thanks for the feedback! 

(I do expect there'll be various minor issues like that, and we hope the community catches them quickly so we can tweak the questions to fix them. This was also one reason for showing some questions before they "open".)

[Part 1] Amplifying generalist research via forecasting – Models of impact and challenges

That makes sense to me.

But it seems like you're just saying the issue I'm gesturing at shouldn't cause mis-calibration or overconfidence, rather than that it won't reduce the resolution/accuracy or the practical usefulness of a system based on X predicting what Y will think? 

Introduction To Lesswrong Subculture

(Update: I just saw the post Welcome to LessWrong!, and I think that that serves my needs well.)
 

Introduction To Lesswrong Subculture

I think it's good that a page like this exists; I'd want to be able to use it as a go-to link when suggesting people engage with or post on LessWrong, e.g. in my post on Notes on EA-related research, writing, testing fit, learning, and the Forum.

Unfortunately, it seems to me that this page isn't well suited to that purpose. Here are some things that seem like key issues to me (maybe other people would disagree):

  • This introduction seems unnecessarily intimidating, non-welcoming, and actually (in my perception) somewhat arrogant. For example:
    • "If you have no familiarity with the cultural articles and other themes before you begin interacting, your social experiences are likely to be highly awkward. The rationalist way of thinking and subculture is extremely, extremely complex. To give you a gist of how complex it is and what kind of complexity you'll encounter:"
      • This feels to me like saying "We're very special and you need to do your homework to deeply understand us before interacting at all with us, or you're just wasting our time and we'll want you to go away."
      • I do agree that the rationalist culture can take some getting used to, but I don't think it's far more complex or unusual than the cultures in a wide range of other subcultures, and I think it's very often easiest to get up to speed with a culture partly just by interacting with it.
      • I do agree that reading parts of the Sequences is useful, and that it's probably good to gently encourage new users to do that. But I wouldn't want to make it sound like it's a hard requirement or like they have to read the whole thing. And this passage will probably cause some readers to infer that, even if it doesn't outright say it. (A lot of people lurk more than they should, have imposter syndrome, etc.)
        • I started interacting on LessWrong before having finished the Sequences (though I'd read some), and I think I both got and provided value from those interactions.
      • Part of this is just my visceral reaction to any group saying their way of thinking and subculture is "extremely, extremely complex", rather than me having explicit reasons to think that that's bad.
  • I wrote all of that before reading the next paragraphs, and the next paragraphs very much intensified my emotional feeling of "These folks seem really arrogant and obnoxious and I don't want to ever hang out with them"
    • This is despite the fact that I've actually engaged a lot on LessWrong, really value a lot about it, rank the Sequences and HPMOR as among my favourite books, etc.
  • Maybe part of this is that this is describing what rationalists aim to be as if all rationalists always hit that mark.
    • Rationalists and the rationalist community often do suffer from the same issues other people and communities do. This was in fact one of the really valuable things Eliezer's posts pointed out (e.g., being wary of trending towards cult-hood).

Again, these are just my perceptions. But FWIW, I do feel these things quite strongly. 

Here are a couple much less important issues:

  • I don't think I'd characterise the Sequences as "mostly like Kahneman, but more engaging, and I guess with a bit of AI etc." From memory, a quite substantial chunk of the sequences - and quite a substantial chunk of their value - had to do with things other than cognitive biases, e.g. what goals one should form, why, how to act on them, etc. Maybe this is partly a matter of instrumental rather than just epistemic rationality.
    • Relatedly, I think this page presents a misleading or overly narrow picture of what's distinctive (and good!) about rationalist approaches to forming beliefs and choosing decisions when it says "There are over a hundred cognitive biases that humans are affected by that rationalists aim to avoid. Imagine you added over one hundred improvements to your way of thinking."
  • "Kahneman is notoriously dry" feels like an odd thing to say. Maybe he is, but I've never actually heard anyone say this, and I've read one of his books and papers and watched one of his talks and found them all probably somewhat more engaging than similar things from the average scientist. (Though maybe this was more the ideas themselves, rather than the presentation.)

(I didn't read "Website Participation Intro or "Why am I being downvoted?"", because it was unfortunately already clear that I wouldn't want to link to this page when aiming to introduce people to LessWrong and encourage them to read, comment, and/or post there.)

Modernization and arms control don’t have to be enemies.

Authoritarian closed societies probably have an advantage at covert racing, at devoting a larger proportion of their economic pie to racing suddenly, and at artificially lowering prices to do so. Open societies have probably a greater advantage at discovery/the cutting edge and have a bigger pie in the first place (though better private sector opportunities compete up the cost of defense engineering talent).

These are interesting points which I hadn't considered - thanks!

(Your other point also seems interesting and plausible, but I feel I lack the relevant knowledge to immediately evaluate it well myself.)

Epistemic Warfare

Interesting post.

You or other readers might also find the idea of epistemic security interesting, as discussed in the report "Tackling threats to informed decisionmaking in democratic societies: Promoting epistemic security in a technologically-advanced world". The report is by researchers at CSER and some other institutions. I've only read the executive summary myself. 

There's also a BBC Futures article on the topic by some of the same authors.

Modernization and arms control don’t have to be enemies.

While I am not sure I agree fully with the panel, an implication to be drawn from their arguments is that from an equilibrium of treaty compliance, maintaining the ability to race can disincentivize the other side from treaty violation: it increases the cost to the other side of gaining advantage, and that can be especially decisive if your side has an economic advantage.

This is an idea/argument I hadn't encountered before, and seems plausible, so it seems valuable that you shared it.

But it seems to me that there's probably an effect pushing in the opposite direction: 

  • Even from an equilibrium of treaty compliance, if one state has the ability to race, that might incentivise the other side to develop the ability to race as well. That wouldn't necessarily require treaty violation. 
  • Either or especially both sides having the ability to race can increase risks if they could race covertly until they have gained an advantage, or race so quickly that they gain an advantage before the other side can get properly started, or if the states don't always act as rational cohesive entities (e.g., if leaders are more focused on preventing regime change than preventing millions of deaths in their own country), or probably under other conditions.
    • I think the term "arms race stability" captures the sort of thing I'm referring to, though I haven't yet looked into the relevant theoretical work much.
  • In contrast, if we could reach a situation where neither side currently had the ability to race, that might be fairly stable. This could be true if building up that ability would take some time and be detectable early enough to be responded to (by sanctions, a targeted strike, the other side building up their own ability, or whatever).

Does this seem accurate to you?

I guess an analogy could be to whether you'd rather be part of a pair of cowboys who both have guns but haven't drawn them (capability but not yet racing), or part of a pair who don't have guns but could go buy one. It seems like we'd have more opportunities for de-escalation, less risk from nerves and hair-triggers, etc. in the latter scenario than the former.

I think this overlaps with some of Schelling's points in The Strategy of Conflict (see also my notes on that), but I can't remember for sure.

The Future of Nuclear Arms Control?

Thanks for this thought-provoking post. I found the discussion of how political warfare may have influenced nuclear weapons activism particularly interesting.

Since large yield weapons can loft dust straight to the stratosphere, they don’t even have to produce firestorms to start contributing to nuclear winter: once you get particles that block sunlight to an altitude that heating by the sun can keep them lofted, you’ll block sunlight a very long time and start harming crop yields.

I think it's true that this could "contribute" to nuclear winter, but I don't think I've seen this mentioned as a substantial concern in the nuclear winter papers I've read. E.g., I don't think I've seen any papers suggest that nuclear winter could occur solely due to that effect, without there being any firestorms, or that that effect could make the climate impacts 20% worse than would occur with firestorms alone. Do you have any citations on hand for this claim? 

Notes on "Bioterror and Biowarfare" (2006)

Final thoughts on whether you should read this book

  • I found the book useful
    • The parts I found most useful were (a) the early chapters on the history of biowarfare and bioterrorism and (b) the later chapters on attempts to use international law to reduce risks from bioterror and biowarfare
  • I found parts of the book hard to pay attention to and remember information from
    • In particular, the middle chapters on various types and examples of pathogens
      • But this might just be a “me problem”. Ever since high school, I’ve continually noticed that I seem to have a harder time paying attention to and remembering information about biology than information from other disciplines. (I don’t understand what that would be the case, and I’m not certain it’s actually true, but it has definitely seemed true.)
  • I’m not sure how useful this book would be to someone who already knows a lot about bioterror, biowarfare, and/or chemical weapons
  • I’m not sure how useful this book would be to someone who doesn’t have much interest in the topics of bioterror, biowarfare, and/or chemical weapons
    • But I’m inclined to think most longtermists should read consume at least one book’s worth of content from experts on those topics
    • And I think the book could be somewhat useful for understanding WMDs, international relations, and international law more generally
  • There might be better books on the topic
    • In particular, it’s possible a more recent book would be better?
Notes on "Bioterror and Biowarfare" (2006)

My Anki cards

Note that:

  • It’s possible that some of these cards include mistakes, or will be confusing or misleading out of context.
  • I haven’t fact-checked Dando on any of these points.
  • Some of these cards are just my own interpretations - rather than definitely 100% parroting what the book is saying
  • The indented parts are the questions, the answers are in "spoiler blocks" (hover over them to reveal the text), and the parts in square brackets are my notes-to-self.

Dando says ___ used biological weapons in WW1, but seemingly only against ___.

the Germans and perhaps the French;

draft animals (e.g. horses), not humans

[This was part of sabotage operations, seemingly only/especially in the US, Romania, Norway, and Argentina. The US and Romania were neutral at the time; not sure whether Norway and Argentina were.]

Dando says the 1925 Geneva Protocol prohibits ___, but not ___, of chemical and biological weapons, and that many of the parties to the Protocol entered reservations to their agreement to make it clear that ___.

Use;

Development and stockpiling;

Although they would not use such weapons first, they were prepared to use them in retaliation if such weapons were used first against them

[And a number of offensive bio weapons programs were undertaken by major states in the interwar period. Only later in the 20th century were further arms control restrictions placed on chem and bio weapons.]

Japan's offensive biological warfare program was unique in that ___. The program probably caused ___.

It used human experimentation to test biological agents;

The deaths of thousands of Chinese people

[This program ran from 1931-1945]

Dando mentions 6 countries as having had "vigorous" offensive biological weapons programs during WW2:

Japan, The Soviet Union, France, the UK, the US, Canada

[He doesn't explicitly say these were the only countries with such programs, but does seem to imply that, or at least that no other countries had similarly large programs

He notes that Germany didn't have such a program.

France's program was interrupted by the German invasion in 1940, but was resumed after WW2.]

Dando suggests that the main or most thoroughly prepared type of British WW2 biological warfare weapon/plan was...

To drop millions of cattle cakes infected with anthrax spores onto German fields, to wipe out cattle and thus deal an economic blow to Germany's overstretched agricultural system

[The British did make 5 million of these cakes.]

Dando says that there are 7 countries which definitely had offensive biological weapons programs in the second half of the 20th century:

The US, the UK, the Soviet Union, Canada, France, South Africa, Iraq

[He also says there've been numerous accusations that other countries had such programs as well, but that there isn't definite information about them.]

Dando says that 3 countries continued to have offensive biological weapons after becoming the depository for, ratifying, and/or signing the BTWC:

Soviet Union, South Africa, and Iraq

[This was then illegal under international law. Prior to the BTWC, having such a program wasn't illegal - only the use of bioweapons was.

I think the other 4 states that had had such programs between WW2 and 1972 stopped at that point or before then.]

During WW2, the US offensive biological weapons program was developing anti-___, anti-___, and anti-___ weapons.

personnel; animal; plant

[And the US was considering using anti-plant weapons against Japanese rice production.]

What major change in high-level US policy regarding chemical and biological weapons does Dando suggest occurred around 1956?

What does he suggest this was partly a reaction to?

Changing from a retaliation-only policy for BW and CW to a policy stating that the US would be prepared to use BW or CW in a general war for the purposes of enhancing military effectiveness [and the decision would be reserved for the president];

Soviet statements in 1956 that chemical and biological weapons would be used in future wars for the purposes of mass destruction

[Dando notes that the retaliation-only policy was in line with the US's signature of the 1925 Geneva protocol, but also that the US didn't actually ratify the Geneva protocol till 1975; until then it was only a signatory.]

Dando says an army report says the origin of the US's shift (under Nixon) to renouncing biological and chemical weapons dates from...

Criticism of US application of chemical herbicides and riot control agent(s) in Vietnam starting in the 1960s

[I think this means criticism/opposition by the public.]

The UK's work on an offensive biological weapons capability had been abandoned by...

1957

[According to a report cited by Dando.

Though Dando later indicates the UK restarted some of this work in 1961, I think particularly/only to find a nonlethal incapacitating chemical weapon.]

Dando says that, at the end of WW2, the UK viewed biological weapons as...

On a par with nuclear weapons

["Only when the UK obtained its own nuclear systems did interest in biological weapons decline.”

I don't know precisely what Dando means by this.]

South Africa had an offensive biological weapons program during...

The later stages of the Apartheid regime

[But it was terminated before the regime change.]

What was the scale of South Africa's offensive biological weapons program? What does its main purpose seem to have been?

Relatively small (e.g. smaller than Iraq's program)

Finding means of assassinating the Apartheid regime's enemies

[Elsewhere, Dando suggests that original motivations for the program - or perhaps for some chemical weapons work? - also included the Angola war and a desire to find crowd control agents.]

What has Iraq stated about authority (as of ~1991) to launch its chemical and biological weapons?

Authority was pre-delegated to regional commanders if Baghdad was hit with nuclear weapons

[UNSCOM has noted that that doesn't exclude other forms of use, and doesn't constitute a proof of a retaliation-only policy.]

The approach to chemical weapons that Iraq pursued was ___, in contrast to a Western approach of ___.

Production and rapid use;

Production and stockpiling

[I'm guessing that this means that Iraq pursued the ability to produce chemical weapons shortly before they were needed, rather than having a pre-made, long-lasting stockpile of more stable versions.

Dando says a similar approach could've been taken towards biological weapons.]

Dando says that the main lesson from the Iraqi biological weapons program is that...

A medium-sized country without great scientific and technical resources was, within a few years, able to reach the stage of weaponising a range of deadly biological agents

What kind of vaccine does Dando say South Africa's biological weapons program tried to find? What does someone who had knowledge of the program say the vaccine might've been used for, if it had been found?

An anti-fertility vaccine;

Administering to black women without their knowledge

Dando lists 6 different types of biological agents that could be used for biological weapons:

Bacteria; Viruses; Toxins; Bioregulators; Protozoa; Fungi

[I'm not sure whether this was meant to be exhaustive, nor whether I'm right to say these are "different types of biological agents".

There's also a chance I forgot one of the types he mentioned.]

Dando says that vaccination during a plague epidemic would not be of much help, because...

Immunity takes a month to build up

[Note that I haven't fact-checked this, and that, for all I know, the situation may be different with other pathogens or newer vaccines.]

In the mid twentieth century, ___ tried to use plague-infected fleas to cause an outbreak among ___.

Japan; the Chinese

Dando notes at least 3 factors that could make the option of biowarfare or bioterrorism against animal agriculture attractive:

1. The animals are densely packed in confined areas

2. The animals reared are often from very limited genetic stock (so that a large percentage of them could succumb to a single strain of a pathogen)

3. Many/all pathogens that would be used don't infect humans (reducing risks to the people involved in producing and using the pathogens)

[Dando implies that that third point is more relevant to bioterrorism than biowarfare, but doesn't say why. I assume it's because terrorists will tend to have fewer skills and resources than military programs, making them more vulnerable to accidents.]

What proportion of state-level offensive biological weapons programs (of which we have knowledge) "carefully investigated anti-plant attacks"?

Nearly all

In the 1990s, the US OTA concluded that the cheapest overt production route for 1 nuclear bomb per year, with no international controls, would cost __.

They also concluded that a chemical weapons arsenal for substantial military capability would cost __.

They concluded that a large biological weapons arsenal may cost __.

~$200 million;

$10s of millions;

Less than $10 million

[I'm unsure precisely what this meant.

I assume the OTA thought a covert route for nuclear weapons, with international controls, would be more expensive than the overt route with no international controls.]

Efforts in the 1990s to strengthen the BWC through agreement of a verification protocol eventually failed in 2001 due to the opposition from which country?

The United States

The BTWC was opened for signature in __, and entered into force in __.

1972; 1975

Dando highlights two key deficiencies of the BTWC (at least as of it entering into force in 1975):

1. There was a lack of verification measures

2. No organisation had been put in place to take care of the convention, of its effective implementation, and of its development between review conferences

[Dando notes that, in contrast to 2, there was a large organisation associated with the Chemical Weapons Convention.

Wikipedia suggests that a (very small) Implementation Support Unit for the BTWC was finally created in 2006.]

Dando highlights a US-based stakeholder as being vocally opposed to the ideas that were proposed for verifying compliance with the BTWC:

The huge US pharmaceutical industry and its linked trade associations

[I think Dando might've been talking about opposition to inspections in particular

Dando implies that this contributed to US executive branch being lukewarm on or sort-of opposed to these verification ideas.]

Load More