Looks like you intended to include author names that got dropped?

Assuming that you do reach some kind of agreement or manage to explain it in the end, it's often possible to then look at/think about the dialog you had and condense it down to points of shared agreement or an explanation that would have communicated the thing to the other person faster if you'd just thought of giving it earlier.

Sometimes (if this was over text) you can also just copy-paste the most essential pieces of what you said in the conversation, adding some bridging sentences as context. My post on applying NVC was also stitched together from messages I wrote in dialog with someone. When it has this bit:

I think one of the most important parts of NVC is the idea about distinguishing observations and interpretations, where an "observation" is defined as something that you could objectively verify (e.g. by capturing it on camera) and an interpretation is something that blends in more stuff, such as generalizations or assumptions about intent the other person's intent.

For example, "You're always late" and "You don't care about my time" are interpretations, "On the last three times when we agreed to meet, you showed up 15 minutes after the agreed-upon time" is an observation.

If you can separate those, you can then go into a potentially charged conversation by transforming something like "You are always late, why don't you care about my time" to something like "On the last three times when we agreed to meet, you showed up 15 minutes after the agreed-upon time. I found that frustrating because I made sure to be on time and could have spent that extra fifteen minutes to do something else", which is often quite helpful.

This doesn't mean you'd need to keep detailed records to express things as observations. If you don't remember earlier specifics, you can just say something like "Hey you were fifteen minutes late today and I think that's happened before too". The main intents are to...

In the original conversation, this was two messages from me, with the second one being an answer to someone's question:

Me: I think the most important thing is the idea about distinguishing observations and interpretations, where an "observation" is defined as something that you could objectively verify (e.g. by capturing it on camera) and an interpretation is something that blends in more stuff, such as generalizations or assumptions about intent 

e.g. "you're always late" is an interpretation, "on the last three times when we agreed to meet, you showed up 15 minutes after the agreed-upon time" is an observation

if you can separate those, you can then go into a charged conversation by transforming something like "why are you always late" to something like "on the last three times when we agreed to meet, you showed up 15 minutes after the agreed-upon time. I found that frustrating because I made sure to be on time and could have spent that extra fifteen minutes to do something else", which is often quite helpful

Other person: One problem I personally have with this is it feels like it requires me to keep more detailed records of things than I naturally do, which in turn feels like I’m point-scoring

Like for me to be able to give a good observation about something more than a one-off, I’d have to write it down, which naturally puts me in an adversarial mindset

Me: I think it can be useful if you can point to previous cases in detail but I don't think it's actually necessary, like you could just be like "hey you were fifteen minutes late today and I think that's happened before too"

The main intent is...

I find that some of my effortposts are definitely appreciated on LessWrong, while others aren't. 

Two recent posts on LW that I put a lot of effort into: Four types of approaches for your emotional problems at 44 karma, and Creative writing with LLMs part II, at 2 karma. Going a little longer back, Genetic fitness is a measure of selection strength, not the selection target was something I worked on a lot and thought was quite important, but only got 57 karma. 

Some of this is I think a question of target audience. Genetic fitness definitely has that "narrow technical point relevant only to a few" quality that eukaryote talks about. I also learned from the comments that its central thesis had been a little unclear/muddled; I did clarify that in the comments, but people may have stopped reading before they ever got far enough to read the clarification. Four types of approaches got a more positive reception on my Substack and people messaging me about it in private. I'm not totally sure what happened with Creative writing, but I assume that it just wasn't something LW found particularly interesting and maybe even found a little cringe, whereas a couple of people who were more into LLM-driven creative writing have told me they found it useful.

While Don't ignore bad vibes you get from people was low-effort and is now at 163 karma. My most successful post of late, How anticipatory cover-ups go wrong is at 299 karma; I'd call that medium-effort.

But then I have definitely also had successful high-effort posts! Book summary: Unlocking the Emotional Brain is at 336 karma and took a lot of effort. So did Building up to an Internal Family Systems model (295 karma) and My attempt to explain Looking, insight meditation, and enlightenment in non-mysterious terms (241 karma).

Something that unites those three is that they were specifically written with LW as the target audience, with me asking myself something like "what is the LW-optimized way of expressing this idea that LW readers might find especially interesting". Of my low-karma effortposts, Genetic fitness did have that quality, but Four types of approaches was written for a broader audience and I could definitely have done more to express it in a more LW-adapted style. For Creative writing, I was somewhat thinking about the LW reception - in particular, I was a bit defensive about the previous post in the series apparently having given the impression I'd fallen for LLM sycophancy and thought of LLM outputs as better than they were, so a substantial chunk of the post was about critiquing and rewriting LLM outputs - but I did also explicitly have the thought of "well, this is something that I personally find interesting and I'll just put it out there and see if anyone else does, and if not too bad". So I guess a lot of that is explained by the extent to which I was tailoring it to my target audience. (Though Don't ignore bad vibes was not particularly LW-tailored.)

(My recent post about the importance of the target audience for your writing, a medium-effort one, is at 50 karma.)

Agree; I'd also like to emphasize this part:

Since 2016, I have been building HelixNano, a clinical stage biotech (and still my main gig), with Nikolai Eroshenko. Recently, HelixNano teamed up with OpenAI to push AI bio's limits. To our surprise, we saw models invent genuinely new wet lab methods (publication soon).

We got super excited. There was a path to superhuman drug designers. But we couldn't ignore the shadow of superhuman virus designers. A world with breakthrough AI drugs can't exist without new biological defenses. We spun out Red Queen Bio to build them. 

Based on this, they didn't need to set up a new company. They already had an existing biotech company that was focused on its own research, when they realized that "oh fuck, based on our current research things could get really bad unless someone does something"... and then they went Heroic Responsibility and spun out a whole new company to do something, rather than just pretending that no dangers existed or making vague noises and asking for government intervention or something.

It feels like being hostile toward them is a bit Copenhagen Ethics, in that if they hadn't tried to do the right thing, it's possible that nobody would have heard about this and things would have been much easier for them. But since they were thinking about their consequences of their research and decided to do something about it and said that in public, they're now getting piled on for not answering every question they're asked on X. (And if I were them, I might also have concluded that the other side is so hostile that every answer might be interpreted in the worst possible light and that it's better not to engage.)

That's certainly true. But at least for me it doesn't seem to be a very big factor, because when I reorient to explaining something to a person and then find it easier, it's very often also over text.

Man that's stressful. I hope you get to rest better soon, maybe just sleeping through one whole day like a hibernating bear. Making happy and content sleeping bear sounds. Which I guess bears don't make if they're hibernating. But I digress.

It's been cool to read many of the Inkhaven posts, so I'm happy that you've been organizing it!

Or is the opposite likely to happen - does the AI frequently fail to solve the customer's problem until the customer demands to speak to a human, and then you have to pay for the AI's and the human worker's time? And what's the chance that it gives wrong advice that the company is then held liable for? 

Even one case of that might be quite costly if the AI promised the customer something very expensive, and companies are likely to be nervous about such risks. Or in the case of electronic medical records, what's the chance of the voice-to-text hallucinating words and potentially getting a person killed due to misdiagnosis? (I'm sure that human workers mishear things too, but I also expect that a jury will be much harsher on "we deployed an experimental system with a known tendency for hallucinations in our hospital" than on "our receptionist misheard".)

I'm a little confused about what's going on since apparently the explicit goal of the company is to defend against biorisk and make sure that biodefense capabilities keep up with AI developments, and when I first saw this thread I was like "I'm not sure of what exactly they'll do, but better biodefense is definitely something we need so this sounds like good news and I'm glad that Hannu is working on this".

I do also feel that the risk of rogue AI makes it much more important to invest in biodefense! I'd very much like it if we had the degree of automated defenses that the "rogue AI creates a new pandemic" threat vector was eliminated entirely. Of course there's the risk of the AI taking over those labs but in the best case we'll also have deployed more narrow AI to identify and eliminate all cybersecurity vulnerabilities before that.

And I don't really see a way to defend against biothreats if we don't do something like this (which isn't to say one couldn't exist, I also haven't thought about this extensively so maybe there is something), like the human body wouldn't survive for very long if it didn't have an active immune system.

Today, we are launching Red Queen Bio (http://redqueen.bio), an AI biosecurity company, with a $15M seed led by OpenAI. Biorisk grows exponentially with AI capabilities. Our mission is to scale biological defenses at the same rate. 

Since 2016, I have been building HelixNano, a clinical stage biotech (and still my main gig), with Nikolai Eroshenko. Recently, HelixNano teamed up with OpenAI to push AI bio's limits. To our surprise, we saw models invent genuinely new wet lab methods (publication soon).

We got super excited. There was a path to superhuman drug designers. But we couldn't ignore the shadow of superhuman virus designers. A world with breakthrough AI drugs can't exist without new biological defenses. We spun out Red Queen Bio to build them. 

AI biosecurity is a different game from traditional biodefense, with relatively static threats and flat budgets. What do you do when the attack surface grows at the rate of AI progress, driven by trillions of dollars of compute?

Red Queen Bio's core thesis is **defensive co-scaling.** You have to couple defensive capabilities and funding to the same technological and financial forces that drive the AGI race, otherwise they can't keep up.

We work with frontier labs to map AI biothreats and pre-build medical countermeasures against them. For co-scaling to work, this needs to improve as models do, and scale with compute. So our pipeline is built upon the leading models themselves, lab automation and RL.

We also need *financial* co-scaling. Governments can't have exponentially scaling biodefense budgets. But they can create the right market incentives, as they have done for other safety-critical industries. We're engaging with policymakers on this both in the US and abroad. 

RQB's work is driven by a civilizational need. But the economic incentives are ultimately on our side too. The capital behind what may be the biggest industrial transformation in human history is not going to tolerate unpriced tail risk on the scale of COVID or bigger. 

We are committed to cracking the business model for AI biosecurity. We are borrowing from fields like catastrophic risk insurance, and working directly with the labs to figure out what scales. A successful solution can also serve as a blueprint for other AI risks beyond bio.

This is bigger than us. No company, AI lab or government is going to solve defensive co-scaling alone. Accordingly, we are committed to open collaboration with them all. Red Queen Bio is a Public Benefit Corporation, with governance to ensure mission takes precedence over any individual partnership. 

In case it's not obvious, Red Queen Bio and defensive co-scaling are very much inspired by VitalikButerin's d/acc philosophy. We find it inspiring, but differ in a couple of important ways. 

First, we are skeptical that the d/acc approach of building purely defensive capabilities first is possible: in our view, they have to piggyback on general capabilities.

In contrast to d/acc, we also believe it's hard to maintain defender advantage through de-centralization alone. For the sci-fi fans, writing DARKOME (a near-future biotech thriller) in part changed my mind on this! 

But we heartily agree with VitalikButerin on the brightness and centrality of human kindness and agency. 

In the face of fast AI timelines and the enormity of the stakes, it's easy to feel trapped in the AGI race dynamic. But the incentive structures driving it are not physical laws. They are no more real than others we can create. 

By launching Red Queen Bio, we are choosing a different race. One where defense keeps up with offense and economics spurs safety. 

The starting pistol has gone off. It's time to run together.

the current university system coddles

No doubt true in many cases, but I would assume this to depend on exactly which country, university, degree etc. we were talking about?

I don't play shooters so found this a fascinating read

This seems like one of those "I'll never enjoy playing this but I'll love to read stories about it" games