utilistrutil

Re going along with lies - Yeah, I think the coverage of data center water usage has been an example of that at its worst :/

Re journalists sitting on scoops - I'm curious if you're able to share any examples? I don't doubt that it happens.

Replying toWhy Talk to Journalists

I agree it's possible and it's worth thinking through considerations like this. But I still don't think this is a good model of journalists' incentives.

In practice, "probability of being seen as inaccurate" is the term that dominates, which means inaccuracies tend to show up at points in the news article that face the least scrutiny, eg the part of an AI article where the journalist rushes through what a transformer is. These are the parts that are often least important to readers, and least important to you as a source.

And then I would describe the motivation more as "career success" than "political benefit". As in getting a big scoop or writing a... (read more)

Replying toWhy Talk to Journalists

To make the analogy stronger, what if it only inverts it 1/10 times? Then I think the answer is non-obvious and depends on your principles.

If you're deontological about it, I think you could make a case that your hands are not dirty for making the best of a bad system.

If you're consequentialist about it, I'm saying the 9/10 accuracies could outweigh the 1/10 inaccuracies. And as Zack said, the 1/10 errors are rarely true inversions. That's why

even if you do get misquoted, it doesn't mean talking to the journalist was net-negative, even for that particular piece and even ex-post. As annoying as it is, it might be outweighed by the value of steering the article in positive ways.

Replying toWhy Talk to Journalists

How to Talk to Journalists

You are doing the lord's work fr

3mo

TLDR: Just pick up the phone. Agree on permissions for information before you share it (see Step 4).

I have worked as a professional journalist covering AI for over a year, and during that time, multiple people working in AI safety have asked me for advice on engaging with journalists. At this point, I've converged on some core lessons, so I figured I should share them more widely.

I've also reviewed some of the prior art about how to talk to journalists on LessWrong and found it unsatisfying. The answer to the question often seems to be "Don't."^[1]

Unsurprisingly then, I think many people feel like they are not prepared to talk to journalists. They... (read 3082 more words →)

Something Is Lost When AI Makes Art

3mo

Sources' motivations for talking to journalists are a bit of a puzzle. On the one hand, it's helpful for journalists to work out what those motivations are, to keep sources invested in the relationship. On the other hand, sources behave in perplexing ways, for instance sharing information against their own interests, so it's often best to treat their psychology as unknowable.

Reflecting on sources' willingness to share compromising information, one mystified AI journalist told me last weekend, "no reasonable person would do this."

But to the extent I can divine their motivations, here are some reasons I think people talk to me at work:

Bringing attention and legitimacy to themselves and their work
Trading tips and

... (read 843 more words →)

Replying toRead More News

utilistrutil11mo

Summary

This winter, MATS will be running our seventh program. In early-mid 2024, 46% of alumni from our first four programs (Winter 2021-22 to Summer 2023) completed a survey about their career progress since participating in MATS. This report presents key findings from the responses of these 72 alumni.

78% of respondents described their current work as "Working/interning on AI alignment/control" or "Conducting alignment research independently."
- 49% are "Working/interning on AI alignment/control."
- 29% are "Conducting alignment research independently."
- 1.4% are "Working/interning on AI capabilities."
Since MATS, 54% of respondents applied to a job and advanced past the first round of interviews.
- 64% of those who shared more details accepted a job offer.
- Alumni reported that MATS made it more likely that

... (read 3231 more words →)

MATS Winter 2023-24 Retrospective

That's right: a neartermist take! Cower before its sublime wrath!

The Resurrection of the Author

Here, hold my bee.
— Someone with beauty in their eye

There are many theories of aesthetics that seek to explain the value of art and the nature of beauty. On some of these theories, the artist's role is subordinate to the viewer's. Aesthetic value is located in a private experience between the viewer and the artistic object. The beauty of an intricate painting is not too different from the beauty of a sunset: the painter is irrelevant.

If you think of art in these terms, you might be excited by the prospect of a proliferation of cheap and beautiful AI-generated artwork.... (read 2998 more words →)

I would really like to see a post from someone in AI policy on "Grading Possible Comprehensive AI Legislation." The post would lay out what kind of safety stipulations would earn a bill an "A-" vs a "B+", for example.

I'm imagining a situation where, in the next couple years, a big omnibus AI bill gets passed that contains some safety-relevant components. I don't want to be left wondering "did the safety lobby get everything it asked for, or did it get shafted?" and trying to construct an answer ex-post.

File under 'noticing the start of an exponential': A.I. Helped to Find a Vast Source of the Copper That A.I. Needs to Thrive

Scott Alexander says:

Suppose I notice I am a human on Earth in America. I consider two hypotheses. One is that everything is as it seems. The other is that there is a vast conspiracy to hide the fact that America is much bigger than I think - it actually contains one trillion trillion people. It seems like SIA should prefer the conspiracy theory (if the conspiracy is too implausible, just increase the posited number of people until it cancels out).

I am often confused by the kind of reasoning at play in the text I bolded. Maybe someone can help sort me out. As I increase the number of people in the conspiracy... (read more)

How LLMs Work, in the Style of The Economist

utilistrutil, LauraVaughan, McKennaFitzgerald, Christian Smith, Juan Gil, Henry Sleight, Matthew Wearden, Ryan Kidd

Co-Authors: @Rocket, @LauraVaughan, @McKennaFitzgerald, @Christian Smith, @Juan Gil, @Henry Sleight, @Matthew Wearden, @Ryan Kidd

The ML Alignment & Theory Scholars program (MATS) is an education and research mentorship program for researchers entering the field of AI safety. This winter, we held the fifth iteration of the MATS program, in which 63 scholars received mentorship from 20 research mentors. In this post, we motivate and explain the elements of the program, evaluate our impact, and identify areas for improving future programs.

Summary

Key details about the Winter Program:

The four main changes we made after our Summer program were:
- Reducing our scholar stipend from $40/h to $30/h based on alumni feedback;
- Transitioning Scholar Support to Research Management;
- Using the full Lighthaven campus for

... (read 14584 more words →)

Analogy Bank for AI Safety

The Assignment:

(4 hours) Write an Economist-style explainer article on how LLMs work. You’ve just started as an AI reporter at The Economist, and your editor’s realised there’s no good Economist Explains style piece on how LLMs work. They’ve asked you to write one. It should be 500 words, and in the style of other Economist Explains pieces.
Examples: Economist explainer on biological weapons; Economist explainer on diffusion models; FT explainer on transformers.

Thank you to Shakeel Hashim for feedback! Shakeel previously worked at The Economist as an editor.

Since OpenAI released ChatGPT in November 2022, large language models (LLMs) have gained international attention. A language model is a piece of AI software designed for tasks... (read 595 more words →)

I just came across this word from John Koenig's Dictionary of Obscure Sorrows, that nicely capture the thesis of All Debates Are Bravery Debates.

redesis n. a feeling of queasiness while offering someone advice, knowing they might well face a totally different set of constraints and capabilities, any of which might propel them to a wildly different outcome—which makes you wonder if all of your hard-earned wisdom's fundamentally nonstraferable, like handing someone a gift card in your name that probably expired years ago.

MATS Summer 2023 Retrospective

“A great deal of thinking well is about knowing when to think at what levels of abstraction. On the one hand, abstractions drop detail, and the reliability of inferences using them varies in complex ways with context. On the other hand, reasoning abstractly can be much faster and quicker, and can help us transfer understanding from better to less known cases via analogy . . . my best one factor theory to explain the worlds I’ve liked best, such as “rationalists”, is that folks there have an unusually high taste for abstraction . . . Thus my strongest advice for my fellow-traveler worlds of non-academic conversation is: vet your abstractions more. For example,

... (read 2375 more words →)