Edit: Just separating this for coherence's sake

Lab Safety Procedures/PPE/Sanitation: I think I have some ideas for where I could start on that? BSL is probably a good place to start.

I'd feel pretty weird posting about that on LessWrong, tbh? (I still might, though.)

I don't currently feel like writing this. But, I'll keep it in mind as a possibility.

Summary of orgs, positions, room-for-funding: I do not have the means, access, clearance, or credentials to do this. (I don't care about me lacking some of those credentials, but other people have made it clear that they do.)

I really would like this to exist! I get the sense that better people than me have tried, and were usually were only able to get part-way, but I haven't tracked it recently. This has led me to assume that this task is more difficult than you'd expect. I have seen a nice copy of a biosecurity-relevant-orgs spreadsheet circulating at one point, though (which I think could get partial-credit).

The closest thing I probably could output are some thoughts on what broad-projects or areas of research seem likely to be valuable and/or underfunded. But I would expect it to be lower-resolution, and less valuable to people.


Heh. Damn, did this post end up in the right Everett branch.


Here's a simplification of my current assessment heuristic...

  • What order-of-magnitude is the audience? (a multiplier)
    • Any relevant audience skews/filters?
    • What are the tails?
  • What's the trade-off for offense vs defense? (+/- direction, & size)
    • Is it + or - overall? How big?
    • Do any points swamp the others in importance?
  • What am I not easily factoring in? Are there any gotchas? (checklist + Murphyjutsu)
    • Future Advances
    • Idea Inoculation
    • Second-degree and unintended audiences
    • Murphyjutsu it
  • Sanity Check: Other
    • Roughly how much do I actually trust the judgement I reached?
      • Should I sleep on it? Withhold it?
    • Anyone I should run things by?

Ah! Thanks for the clarification.

I've actually had several people say they liked the Concrete Examples section, but that they wish I'd said more that would help them recreate the thought-process.

Unfortunately, these were old thoughts for me. The logic behind a lot of them feels... "self-evident" or "obvious" to me, or something? Which makes me a worse teacher, because I'm a little blind to what it is about them that isn't landing.

I'd need to understand what people were seeing or missing, to be able to offer helpful guidance around it. And... nobody commented.

(My rant on basic knowledge was a partial-attempt on my part, to crack open my logic for one of these.)

Edit: I added my core heuristic to the Concrete Examples thread

Perceptual Control Theory

Friston's Free Energy doesn't have its own page. But I think a lot of the PCT-relevant conversation on LW, ends up under that term. Unlike the "mostly just intro-posts" under the PCT term proper, FFE seems to have more-recent engagement, so I think FFE has more of a presence here than PCT.

In retrospect, not including those was a mistake.

Here's a (non-exhaustive) handful of the FFE posts:


"Basic facts" as "safe discussion topics": Ooh, I disagree! I think this heuristic doesn't always hold, especially for people writing on a large platform.

For basic information, it is sometimes a good idea to think twice if a fact might be very-skewed towards benefiting harmful actions over protective ones. If you have a big platform, it is especially important to do so.

(It might actually be more important for someone to do this for basic facts, than sophisticated ones? They're the ones a larger audience of amateurs can grasp.)

If something is already widely known, that does somewhat reduce the extent of your "fault" for disseminating it. That rule is more likely to hold for basic facts.

But if there is a net risk to a piece of information, and you are spreading it to people who wouldn't otherwise know? Then larger audiences are a risk-multiplier. So, sometimes spreading a low-risk basic thing widely could be more dangerous, overall, than spreading an high-risk but obscure and specialist thing.

It was easy for me to think of at least 2 cases where spreading an obvious, easy-to-grasp fact could disproportionately increase the hazard of bad actors relative to good ones, in at least some petty ways. Here's one.

Ex: A member of the Rajneeshee cult once deliberately gave a bunch of people food poisoning, then got arrested. This is a pretty basic fact. But I wouldn't want to press a button that would disseminate this fact to 10 million random people? People knowing about this isn't actually particularly protective against food poisoning, and I'd bet that there is least 1 nasty human in 10 million people. If I don't have an anticipated benefit to sharing, I would prefer not to risk inspiring that person.

On the other hand, passing around the fact that a particular virus needs mucus membranes to enter cells seems... net-helpful? It's easier for people to use that to advise their protective measures, and it's unlikely to help a rare bad actor who is sitting the razor's-edge case where they would have infected someone IF ONLY they had known to aim for the mucus membranes, AND where they only knew about that because you told them.

(And then you have complicated intermediate cases. Off the top of my head, WHO's somewhat-dishonest attempt to convince people that masks don't work, in a bid to save them for the medical professionals? I don't think I like what they did (they basically set institutional trust on fire), but the situation they were in does tug at some edge-cases around trying to influence actions vs beliefs. The fact that masks did work, but had a limited supply, meant that throwing information in any direction was going to benefit some and harm others. It also highlights that, paradoxically, it can be common for "basic" knowledge to be flat-out wrong, if your source is being untrustworthy and you aren't being careful...)


Thanks for the proposed edits! I'll look them over.

"Careful, clear, and dry" was basically the tone that I intended. I will try to incorporate the places where your wording was clearer than mine, and I have found several places where it was.

Perceptual Control Theory

In trying not to be vague, I veered towards writing too much. So if someone could take what I wrote and destroy half of it (or destroy all of it, and write a new and better thing) that would be lovely.

(In a slight inversion of PCT, I feel very sick of looking at my own writing.)

Perceptual Control Theory

I share the impression that there's been a bit more talk and thought on it.

But I tried out the obvious search terms on here, and for the life of me I can't find it. (Other than a few side-mentions, which it didn't seem worth tagging.)

Free Energy Principle gets mentioned a few more times, but I don't know that it's quite the same thing.

[Linkpost] AlphaFold: a solution to a 50-year-old grand challenge in biology

That is the dream. The reality is harder, and the combinatorics are not friendly.

In practice, trying to "catch 2 proteins hanging out together" has usually been easier.

The main way we actually check to see if 2 proteins are interacting is... well, this metaphor is fun.

We try to work out which proteins are a couple, by trying to catch the proteins holding hands at the school dance. Either by freezing them, or sticking glue on their hands.

Sometimes even dragging one of them out of the school dance, and then checking to see if the other one tagged along.

Or if you already have a pretty good guess, try just grounding one of them and see if the other one starts acting weird.

I guess this turns the simulation method into "computer-modeling which people are likely to end up in a relationship together" which... seems to capture some of the right intuitions for how hard it is, and how much knowing "they were present in the same place at the same time" matters (whether they had an opportunity to meet in a cell type & cell compartment; something protein-shape doesn't tell you). Watching for hand-holding has typically been easier.

Un-metaphoring: there's multiple variants of this broad class of technique, and there's even a variant of it for DNA-DNA, DNA-protein, or RNA-protein interactions.

Here's some slightly-de-metaphored executions:

  • Glue: A chimeric-protein with a sticky-end (and then isolating one of the proteins in a binding column, and checking what else tagged along).
  • Freeze: Chemicals that halt cellular processes and cause semi-random-binding (ideally reversible) of things that happen to be next to each other whenever you took the freeze-frame.
  • Grounding: Here that means either altering, removing, or silencing one protein, to see how it affects the behavior of another.

And of course, whenever you do this, you still have to do: isolating, sequencing, and identifying the batch of proteins you've nabbed.

Load More