My ops background

7mo

This is a short story I wrote in mid-2022. Genre: cosmic horror as a metaphor for living with a high p-doom.

One

The last time I saw my mom, we met in a coffee shop, like strangers on a first date. I was twenty-one, and I hadn’t seen her since I was thirteen.

She was almost fifty. Her face didn’t show it, but the skin on the backs of her hands did.

“I don’t think we have long,” she said. “Maybe a year. Maybe five. Not ten.”

It says something about San Francisco, that you can casually talk about the end of the world and no one will bat an eye.

Maybe twenty, not fifty, was what she’d... (read 8691 more words →)

276

I do think it's fair to consider the work on GPT-3 a failure of judgement and a bad sign about Dario's commitment to alignment, even if at the time (also based on LinkedIn) it sounds like he was also still leading other teams focused on safety research.

(I've separately heard rumors that Dario and the others left because of disagreements with OpenAI leadership over how much to prioritize safety, and maybe partly related to how OpenAI handled the GPT-3 release, but this is definitely in the domain of hearsay and I don't think anything has been shared publicly about it.)

Edited first line, which hopefully clarifies this better.

It's deliberate that this post covers mostly specifics that I learned from Anthropic staff, and further speculation is going to be in a separate later post. I wanted to make a really clear distinction between "these are things that were said to me about Anthropic by people who have context" (which is, for the most part, people in favor of Anthropic's strategy), and my own personal interpretation and opinion on whether Anthropic's work is net positive, which is filtered through my worldview and which I think most people at Anthropic would disagree with.

Part two is more critical, which means I want to write about it with a lot of effort and care, so I expect I'll put it up in a week or two.

My sense is that it's been somewhere in between – on some occasions staff have brought up doubts, and the team did delay a decision until they were addressed, but it's hard to judge how much the end result was a different decision from what would have been made otherwise, versus just happening later.

The sense I've gotten of the culture is compatible with (current) Anthropic being a company that would change their entire strategic direction if staff started coming in with credible arguments that "what if we shouldn't be advancing capabilities?", but I think this hasn't yet been put to the test – people who choose work at Anthropic are going to be selected for agreeing on the premises behind the Anthropic strategy – and it's hard to know for sure how it would go.

Your summary seems fine!

Why do you need to do all of this on current models? I can see arguments for this, for instance, perhaps certain behaviors emerge in large models that aren’t present in smaller ones.

I think that Anthropic's current work on RL from AI Feedback (RLAIF) and Constitutional AI is based on large models exhibiting behaviors that don't work in smaller models? (But it'd be neat if someone more knowledgeable than me wanted to chime in on this!)

My current best understanding is that running state of the art models is expensive in terms of infrastructure and compute, the next generation models will get even more expensive to train and run, and... (read more)

This post is the first half of a series about my attempts understand Anthropic’s current strategy and lay out the facts to consider in terms of whether Anthropic’s work is likely to be net positive and whether, as a given individual, you should consider applying. (The impetus for looking into this was to answer the question of whether I should join Anthropic's ops team.) As part of my research, I read a number of Anthropic’s published papers, and spoke to people within and outside of Anthropic.

This post contains “observations” only, which I wanted to write up as a reference for anyone considering similar questions. I will make a separate post about the... (read 1157 more words →)

167

W h a t that's wild, wow, I would absolutely not have predicted DALL-E could do that! (I'm curious whether it replicates in other instances.)

Tragically DALL-E still cannot spell, but here you go:

"A group of happy people does Circling and Authentic Relating in a park"

Book review: The Checklist Manifesto

I got access to DALL-E 2 earlier this week, and have spent the last few days (probably adding up to dozens of hours) playing with it, with the goal of mapping out its performance in various areas – and, of course, ending up with some epic art.

Below, I've compiled a list of observations made about DALL-E, along with examples. If you want to request art of a particular scene, or to test see what a particular prompt does, feel free to comment with your requests.

DALL-E's strengths

Stock photography content

It's stunning at creating photorealistic content for anything that (this is my guess, at least) has a broad repertoire of online stock images – which... (read 2639 more words →)

304

353

Should doctors be neutral?

Atul Gawande’s The Checklist Manifesto was originally published in 2009. By the time I read it a few years ago, the hard-earned lessons explained in this book had already trickled into hospitals across North America. It’s easy to look at the core concept and think of it as trivial. For decades, though, it was anything but obvious.

Atul Gawande walks readers through his experience of how the modern medical system fails. The 20th century saw vast increases in medical knowledge, both through a richer understanding of the body, and from swathes of new drugs, tests, and surgical procedures. And yet, mistakes are still made; diagnoses are missed, critical tests aren’t run, standard treatments... (read 1305 more words →)

123

Curing insanity with malaria

Medical professionals providing humanitarian aid are divided over whether to hold to medical confidentiality and political neutrality or to speak out about the atrocities they witness.

[Note: this is not written as a typical Lesswrong post, and was part of a challenge to write something in the style of a journalist article.]

Along with all the many challenges of aid work, from lack of funding and supplies, to directly putting themselves in the line of violence, medical providers responding to humanitarian crises are often faced with an agonizing dilemma. On the one hand, they often witness terrible atrocities, where publicly speaking up is necessary to address the root causes and hold governments accountable for... (read 1147 more words →)

Sometimes the history of medicine is very, very surreal. For example, consider that in 1927, a physician named Julius Wagner-Jauregg received the Nobel Prize in medicine, for...deliberately infecting his patients with malaria. As a treatment for psychosis.

This often worked.

Well, it did kill around 15% of the patients, but it was nonetheless seen as a miracle cure.

General paralysis of the insane was first identified and described as a distinct disease in the early 19th century. It was initially thought to be caused by an ‘inherent weakness of character’. The initial symptoms were of mental deterioration and personality changes; patients suffered a loss of social inhibitions, gradual impairment of judgment, concentration and short-term memory.... (read 1028 more words →)

163

What is operations?

This the first in a sequence of posts about “operations”.

Acknowledgements to Malo Bourgon, Ray Arnold, Michelle Hutchinson, and Ruby for their feedback on this post.

My ops background

Several years ago, I decided to focus on operations work for my career. From 2017 to 2019 I was one of the operations staff at the Center for Effective Altruism, initially as the operations manager and later as the the Finance Lead. Prior to that, I was a volunteer logistics lead at approximately 10 CFAR workshops; I also ran ops for SPARC twice, and for a five day AI-safety retreat. I also attribute some of my ops skill to my previous work as an ICU... (read 2041 more words →)

115

Fiction writing ramble, #2: Worldbuilding.

This is an attempt to walk through the mental process I follow when writing fiction. [Goals: I'd like to better understand what my brain is doing, and put out ideas for other people who might be interested in writing fiction.]

Historically, worldbuilding (I'm talking mainly about fantasy settings here, but sci-fi as well; earthfic applies less) has been one of the planning steps that I most struggle to do alone; I've tended to do it via brainstorming with friends. Figuring out how magic systems and societal norms work is shiny, but apparently less shiny to me than just writing. This method has some obvious downsides; it's harder to... (read 400 more words →)

Fiction writing ramble, 1 of ??:

I've been trying to introspect lately on my fiction writing process. My brain is opaque about what it's doing, which I guess makes sense – I've been telling stories since literally before I knew how to write, and I've never had any formal instruction in it.

Current question: why is it so intensely a superstimulus for me? Writing fiction is pretty much the only thing I will eagerly and endorsedly do for 16 hours straight. My best and most enjoyable writing periods feel very much like "chasing the shiny" – what "the shiny" is, is hard to describe, but it feels more "substantial" than other second-to-second dopamine-hit-seeking behaviours... (read 467 more words →)

Swimmer963's Shortform

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

Reclaiming Eddie Willers

Micro feedback loops and learning

[Content: personal ramble, not trying to be rigorous]

When I read Atlas Shrugged a few years ago, it was one of the more intensely disturbing experiences I’ve had.

I remember that Eddie Willers was the only character I resonated or identified with much. He’s also, as far as I can tell, the only (even slightly positively portrayed) Hufflepuff character in the story. And the last we see is of him alone in the wilderness, as the last train breaks down – mistakenly loyal to the train company, an entity that isn’t capable of loyalty in return, and not agenty or cool enough to join the main protagonists in their escape from the... (read 435 more words →)