MichaelA

I’m Michael Aird, an Associate Researcher with Rethink Priorities. In April, I'll also start part-time as a Research Scholar with the Future of Humanity Institute. Opinions expressed are my own. You can give me anonymous feedback at this link.

With Rethink, I'm currently mostly working on nuclear risk research. With FHI, I might work on these things.

Previously, I did longtermist macrostrategy research for Convergence Analysis and then for the Center on Long-Term Risk. More on my background here.

I mostly post to the EA Forum.

If you think you or I could benefit from us talking, feel free to message me or schedule a call.

Sequences

Information hazards and downside risks
Moral uncertainty

Comments

Notes on "Bioterror and Biowarfare" (2006)

Final thoughts on whether you should read this book

  • I found the book useful
    • The parts I found most useful were (a) the early chapters on the history of biowarfare and bioterrorism and (b) the later chapters on attempts to use international law to reduce risks from bioterror and biowarfare
  • I found parts of the book hard to pay attention to and remember information from
    • In particular, the middle chapters on various types and examples of pathogens
      • But this might just be a “me problem”. Ever since high school, I’ve continually noticed that I seem to have a harder time paying attention to and remembering information about biology than information from other disciplines. (I don’t understand what that would be the case, and I’m not certain it’s actually true, but it has definitely seemed true.)
  • I’m not sure how useful this book would be to someone who already knows a lot about bioterror, biowarfare, and/or chemical weapons
  • I’m not sure how useful this book would be to someone who doesn’t have much interest in the topics of bioterror, biowarfare, and/or chemical weapons
    • But I’m inclined to think most longtermists should read consume at least one book’s worth of content from experts on those topics
    • And I think the book could be somewhat useful for understanding WMDs, international relations, and international law more generally
  • There might be better books on the topic
    • In particular, it’s possible a more recent book would be better?
Notes on "Bioterror and Biowarfare" (2006)

My Anki cards

Note that:

  • It’s possible that some of these cards include mistakes, or will be confusing or misleading out of context.
  • I haven’t fact-checked Dando on any of these points.
  • Some of these cards are just my own interpretations - rather than definitely 100% parroting what the book is saying
  • The indented parts are the questions, the answers are in "spoiler blocks" (hover over them to reveal the text), and the parts in square brackets are my notes-to-self.

Dando says ___ used biological weapons in WW1, but seemingly only against ___.

the Germans and perhaps the French;

draft animals (e.g. horses), not humans

[This was part of sabotage operations, seemingly only/especially in the US, Romania, Norway, and Argentina. The US and Romania were neutral at the time; not sure whether Norway and Argentina were.]

Dando says the 1925 Geneva Protocol prohibits ___, but not ___, of chemical and biological weapons, and that many of the parties to the Protocol entered reservations to their agreement to make it clear that ___.

Use;

Development and stockpiling;

Although they would not use such weapons first, they were prepared to use them in retaliation if such weapons were used first against them

[And a number of offensive bio weapons programs were undertaken by major states in the interwar period. Only later in the 20th century were further arms control restrictions placed on chem and bio weapons.]

Japan's offensive biological warfare program was unique in that ___. The program probably caused ___.

It used human experimentation to test biological agents;

The deaths of thousands of Chinese people

[This program ran from 1931-1945]

Dando mentions 6 countries as having had "vigorous" offensive biological weapons programs during WW2:

Japan, The Soviet Union, France, the UK, the US, Canada

[He doesn't explicitly say these were the only countries with such programs, but does seem to imply that, or at least that no other countries had similarly large programs

He notes that Germany didn't have such a program.

France's program was interrupted by the German invasion in 1940, but was resumed after WW2.]

Dando suggests that the main or most thoroughly prepared type of British WW2 biological warfare weapon/plan was...

To drop millions of cattle cakes infected with anthrax spores onto German fields, to wipe out cattle and thus deal an economic blow to Germany's overstretched agricultural system

[The British did make 5 million of these cakes.]

Dando says that there are 7 countries which definitely had offensive biological weapons programs in the second half of the 20th century:

The US, the UK, the Soviet Union, Canada, France, South Africa, Iraq

[He also says there've been numerous accusations that other countries had such programs as well, but that there isn't definite information about them.]

Dando says that 3 countries continued to have offensive biological weapons after becoming the depository for, ratifying, and/or signing the BTWC:

Soviet Union, South Africa, and Iraq

[This was then illegal under international law. Prior to the BTWC, having such a program wasn't illegal - only the use of bioweapons was.

I think the other 4 states that had had such programs between WW2 and 1972 stopped at that point or before then.]

During WW2, the US offensive biological weapons program was developing anti-___, anti-___, and anti-___ weapons.

personnel; animal; plant

[And the US was considering using anti-plant weapons against Japanese rice production.]

What major change in high-level US policy regarding chemical and biological weapons does Dando suggest occurred around 1956?

What does he suggest this was partly a reaction to?

Changing from a retaliation-only policy for BW and CW to a policy stating that the US would be prepared to use BW or CW in a general war for the purposes of enhancing military effectiveness [and the decision would be reserved for the president];

Soviet statements in 1956 that chemical and biological weapons would be used in future wars for the purposes of mass destruction

[Dando notes that the retaliation-only policy was in line with the US's signature of the 1925 Geneva protocol, but also that the US didn't actually ratify the Geneva protocol till 1975; until then it was only a signatory.]

Dando says an army report says the origin of the US's shift (under Nixon) to renouncing biological and chemical weapons dates from...

Criticism of US application of chemical herbicides and riot control agent(s) in Vietnam starting in the 1960s

[I think this means criticism/opposition by the public.]

The UK's work on an offensive biological weapons capability had been abandoned by...

1957

[According to a report cited by Dando.

Though Dando later indicates the UK restarted some of this work in 1961, I think particularly/only to find a nonlethal incapacitating chemical weapon.]

Dando says that, at the end of WW2, the UK viewed biological weapons as...

On a par with nuclear weapons

["Only when the UK obtained its own nuclear systems did interest in biological weapons decline.”

I don't know precisely what Dando means by this.]

South Africa had an offensive biological weapons program during...

The later stages of the Apartheid regime

[But it was terminated before the regime change.]

What was the scale of South Africa's offensive biological weapons program? What does its main purpose seem to have been?

Relatively small (e.g. smaller than Iraq's program)

Finding means of assassinating the Apartheid regime's enemies

[Elsewhere, Dando suggests that original motivations for the program - or perhaps for some chemical weapons work? - also included the Angola war and a desire to find crowd control agents.]

What has Iraq stated about authority (as of ~1991) to launch its chemical and biological weapons?

Authority was pre-delegated to regional commanders if Baghdad was hit with nuclear weapons

[UNSCOM has noted that that doesn't exclude other forms of use, and doesn't constitute a proof of a retaliation-only policy.]

The approach to chemical weapons that Iraq pursued was ___, in contrast to a Western approach of ___.

Production and rapid use;

Production and stockpiling

[I'm guessing that this means that Iraq pursued the ability to produce chemical weapons shortly before they were needed, rather than having a pre-made, long-lasting stockpile of more stable versions.

Dando says a similar approach could've been taken towards biological weapons.]

Dando says that the main lesson from the Iraqi biological weapons program is that...

A medium-sized country without great scientific and technical resources was, within a few years, able to reach the stage of weaponising a range of deadly biological agents

What kind of vaccine does Dando say South Africa's biological weapons program tried to find? What does someone who had knowledge of the program say the vaccine might've been used for, if it had been found?

An anti-fertility vaccine;

Administering to black women without their knowledge

Dando lists 6 different types of biological agents that could be used for biological weapons:

Bacteria; Viruses; Toxins; Bioregulators; Protozoa; Fungi

[I'm not sure whether this was meant to be exhaustive, nor whether I'm right to say these are "different types of biological agents".

There's also a chance I forgot one of the types he mentioned.]

Dando says that vaccination during a plague epidemic would not be of much help, because...

Immunity takes a month to build up

[Note that I haven't fact-checked this, and that, for all I know, the situation may be different with other pathogens or newer vaccines.]

In the mid twentieth century, ___ tried to use plague-infected fleas to cause an outbreak among ___.

Japan; the Chinese

Dando notes at least 3 factors that could make the option of biowarfare or bioterrorism against animal agriculture attractive:

1. The animals are densely packed in confined areas

2. The animals reared are often from very limited genetic stock (so that a large percentage of them could succumb to a single strain of a pathogen)

3. Many/all pathogens that would be used don't infect humans (reducing risks to the people involved in producing and using the pathogens)

[Dando implies that that third point is more relevant to bioterrorism than biowarfare, but doesn't say why. I assume it's because terrorists will tend to have fewer skills and resources than military programs, making them more vulnerable to accidents.]

What proportion of state-level offensive biological weapons programs (of which we have knowledge) "carefully investigated anti-plant attacks"?

Nearly all

In the 1990s, the US OTA concluded that the cheapest overt production route for 1 nuclear bomb per year, with no international controls, would cost __.

They also concluded that a chemical weapons arsenal for substantial military capability would cost __.

They concluded that a large biological weapons arsenal may cost __.

~$200 million;

$10s of millions;

Less than $10 million

[I'm unsure precisely what this meant.

I assume the OTA thought a covert route for nuclear weapons, with international controls, would be more expensive than the overt route with no international controls.]

Efforts in the 1990s to strengthen the BWC through agreement of a verification protocol eventually failed in 2001 due to the opposition from which country?

The United States

The BTWC was opened for signature in __, and entered into force in __.

1972; 1975

Dando highlights two key deficiencies of the BTWC (at least as of it entering into force in 1975):

1. There was a lack of verification measures

2. No organisation had been put in place to take care of the convention, of its effective implementation, and of its development between review conferences

[Dando notes that, in contrast to 2, there was a large organisation associated with the Chemical Weapons Convention.

Wikipedia suggests that a (very small) Implementation Support Unit for the BTWC was finally created in 2006.]

Dando highlights a US-based stakeholder as being vocally opposed to the ideas that were proposed for verifying compliance with the BTWC:

The huge US pharmaceutical industry and its linked trade associations

[I think Dando might've been talking about opposition to inspections in particular

Dando implies that this contributed to US executive branch being lukewarm on or sort-of opposed to these verification ideas.]

[Part 1] Amplifying generalist research via forecasting – Models of impact and challenges

A final thought that came to mind, regarding the following passage:

It seems possible for person X to predict a fair number of a more epistemically competent person Y’s beliefs -- even before person X is as epistemically competent as Y. And in that case, doing so is evidence that person X is moving in the right direction.

I think that that's is a good and interesting point.

But I imagine there would also be many cases in which X develops an intuitive ability to predict Y's beliefs quite well in a given set of domains, but in which that ability doesn't transferring to new domains. It's possible that this would be because X's "black box" simulation of Y's beliefs is more epistemically competent than Y in this new domain. But it seems more likely that Y is somewhat similarly epistemically competent in this new domain as in the old domain, but has to draw on different reasoning processes, knowledge, theories, intuitions, etc., and X's intuitions aren't calibrated for how Y is now thinking. 

I think we could usefully think of this issue as a question of robustness to distributional shift.

I think the same issue could probably also occur even if X has a more explicit process for predicting Y's beliefs. E.g., even if X believes they understand what sort of sources of information Y considers and how Y evaluates it and X tries to replicate that (rather than just trying to more intuitively guess what Y will say), the process X uses may not be robust to distributional shift. 

But I'd guess that more explicit, less "black box" approaches for predicting what Y will say will tend to either be more robust to distributional shift or more able to fail gracefully, such as recognising that uncertainty is now much higher and there's a need to think more carefully.

(None of this means I disagree with the quoted passage; I'm just sharing some additional thoughts that came to mind when I read it, which seem relevant and maybe useful.)

[Part 1] Amplifying generalist research via forecasting – Models of impact and challenges

Here's a second thought that came to mind, which again doesn't seem especially critical to this post's aims...

You write:

Someone who can both predict my beliefs and disagrees with me is someone I should listen to carefully. They seem to both understand my model and still reject it, and this suggests they know something I don’t.

I think I understand the rationale for this statement (though I didn't read the linked Science article), and I think it will sometimes be true and important. But I think that those sentences might overstate the point. In particular, I think that those sentences implicitly presume that this other person is genuinely primarily trying to form accurate beliefs, and perhaps also that they're doing so in a way that's relatively free from bias.

But (almost?) everyone is at least sometimes primarily aiming (perhaps unconsciously) at something other than forming accurate beliefs, even when it superficially looks like they're aiming at forming accurate beliefs. For example, they may be engaging in "ideologically motivated cognition[, i.e.] a form of information processing that promotes individuals’ interests in forming and maintaining beliefs that signify their loyalty to important affinity groups". The linked study also notes that "subjects who scored highest in cognitive reflection were the most likely to display ideologically motivated cognition". 

So I think it might be common for people to be able to predict my beliefs and disagree with me, but with their disagreement not being based on knowing more or having better reasoning process but rather finding ways to continue to hold beliefs that they're (in some sense) "motivated" to hold for some other reason.

Additionally, some people may genuinely be trying to form accurate beliefs, but with unusually bad epistemics / unusually major bias. If so, they may be able to predict my beliefs and disagree with me, but with their disagreement not being based on knowing more or having better reasoning process but rather being a result of their bad epistemics / biases. 

Of course, we should be very careful with assuming that any of the above is why a person disagrees with us! See also this and this.

The claims I'd more confidently agree with are:

Someone who can both predict my beliefs and disagrees with me might be someone I should listen to carefully. They seem to both understand my model and still reject it, and this suggests they might know something I don’t (especially if they seem to be genuinely trying to form accurate beliefs and to do so via a reasonable process).

(Or maybe having that parenthetical at the end would be bad via making people feel licensed to dismiss people who disagree with them as just biased.)

[Part 1] Amplifying generalist research via forecasting – Models of impact and challenges

Thanks for this and its companion post; I found the two posts very interesting, and I think they'll usefully inform some future work for me.

A few thoughts came to mind as I read, some of which can sort-of be seen as pushing back against some claims, but in ways that I think aren't very important and that I expect you've already thought about. I'll split these into separate comments.

Firstly, as you note, what you're measuring is how well predictions match a proxy for the truth (the proxy being Elizabeth's judgement), rather than the truth itself. Something I think you don't explicitly mention is that:

  1. Elizabeth's judgement may be biased in some way (rather than just randomly erring), and
  2. The network-based forecasters' judgements may be biased in a similar way, and therefore
  3. This may "explain away" part of the apparent value of the network-based forecasters' predictions, along with part of their apparent superiority over the online crowdworkers' predictions.

E.g., perhaps EA-/rationality-adjacent people are biased towards disagreeing with "conventional wisdom" on certain topics, and this bias is somewhat shared between Elizabeth and the network-based forecasters. (I'm not saying this is actually the case; it's just an example.)

You make a somewhat similar point in the Part 2 post, when you say that the online crowdworkers:

were operating under a number of disadvantages relative to other participants, which means we should be careful when interpreting their performance. [For example, the online crowdworkers] did not know that Elizabeth was the researcher who created the claims and would resolve them, and so they had less information to model the person whose judgments would ultimately decide the questions.

But that is about participants' ability to successfully focus on predicting what Elizabeth will say, rather than their ability to accidentally be biased in the same way as Elizabeth when both are trying to make judgements about the ground truth.

In any case, I don't think this matters much. One reason is that this "shared bias" issue probably at most "explains away" a relatively small fraction of the apparent value of the network-adjacent forecasters' predictions, probably without tipping the balance of whether this sort of set-up is worthwhile. Another reason is that there may be ways to mitigate this "shared bias" issue.

Notes on Schelling's "Strategy of Conflict" (1960)

Good idea! I didn't know about that feature.

I've now edited the post to use spoiler-blocks (though a bit messily, as I wanted to do it quickly), and will use them for future lazy-Anki-card-notes-posts as well. 

Notes on Schelling's "Strategy of Conflict" (1960)

I didn't add that tag; some other reader did. 

And any reader can indeed downvote any tag, so if you feel that that tag shouldn't be there, you could just downvote it. 

Unless you feel that the tag shouldn't be there but aren't very confident about that, and thus wanted to just gently suggest that maybe the tag should be removed - like putting in a 0.5 vote rather than a full one. But that doesn't seem to match the tone of your comment.

That said, it actually does seem to me that this post fairly clearly does match the description for that tag; the exercise is using these Anki cards as Anki cards. People can find a link to download these cards in the Anki card file format here. (I've now added that link in the body of the post itself; I guess I should've earlier.)

---

As a meta comment: For what it's worth, I feel like your comment had an unnecessarily snarky tone, at least to my eye. I think you could've either just downvoted the tag, or said the same thing in a way that sounds less snarky. That said:

  • It's very possible (even probable?) that you didn't intend to be snarky, and tht this is just a case of tone getting misread on the internet
  • And in any case, this didn't personally bug me, partly because I've posted on LessWrong and the EA Forum a lot.
    • But I think if I was newer to the sites or to posting, this might leave a bad taste in my mouth and make me less inclined to post in future. (Again, I'm not at all trying to say this was your intent!)

(Edited to add: Btw, I wasn't the person who downvoted your comment, so that appears to be slightly more evidence that your comment was at least liable to be interpreted as unnecessary and snarky - although again I know that that may not have been your intention.)

Notes on Schelling's "Strategy of Conflict" (1960)

Yeah, I definitely agree that that's a good idea with any initialisations that won't already be known to the vast majority of one's readers (e.g., I wouldn't bother with US or UK, but would with APA). In this case, I just copied and pasted the post from the EA Forum, where I do think the vast majority of readers would know what "EA" means - but I should've used the expanded form "effective altruism" the first time in the LessWrong version. I've now edited that. 

Notes on Schelling's "Strategy of Conflict" (1960)

Here's a comment I wrote on the EA Forum version of this post, which I'm copying here as I'd be interested on people's thoughts on the equivalent questions in the context of LessWrong:

Meta: Does this sort of post seem useful? Should there be more posts like this?

I previously asked Should pretty much all content that's EA-relevant and/or created by EAs be (link)posted to the Forum? I found Aaron Gertler's response interesting and useful. Among other things, he said:

Eventually, we'd like it to be the case that almost all well-written EA content exists on the Forum somewhere.

[...]

I meant "quite EA-relevant and well-written". I don't especially care whether the content is written by community members, though I suppose that's slightly preferable (as community members are much more likely to respond to comments on their work).

[...]

A single crosspost with a bit of context from the author -- e.g. a few sentences each of summary/highlights, commentary, and action items/takeaways -- seems better to me than three or four crossposts with no context at all. In my view, the best Forum content tends to give busy people a quick way to decide whether to read further.

And I read a lot of stuff that I think it could be useful for at least some other EAs to read, and that isn't (link)posted to the Forum. So Aaron's comments, combined with my own thinking and some comments from other people, make me think it'd be good for me to make linkposts for lots of that stuff if there was a way to do it that took up very little of my time.

Unfortunately, writing proper book reviews, or even just notes that are geared for public consumption, for all of those things I read would probably take a while. 

But, starting about a month ago, I now make Anki cards for myself anyway during most of the reading I do. So maybe I should just make posts sort-of like this one for most particularly interesting things I read? And maybe other people could start doing that too?

A big uncertainty I have is how often the cards I make myself would be able to transmit useful ideas even to people who (a) aren't me and (b) didn't read the thing I read, and how often they'd do that with an efficiency comparable to people just finding and reading useful sources themselves directly. Another, related uncertainty is whether there'd be any demand for posts like this. 

So I'd be interested in people's thoughts on the above.

Load More