AI Alignment Breakthroughs this week (10/08/23)

FWIW I don't think it's honest to title this "breakthroughs". It's almost the opposite, a list of incremental progress.

This. I think incremental progress is important to report on, and I do buy there have been breakthroughs in alignment, but most of these results are incremental progress at best, not breakthroughs, and I wish there was less hype in this newsletter.

[-]Logan Zoellner2y-1-5

I'm not interested in arguing about the definitions of words.

When I use the phase "alignment breakthrough" in my title I mean:

someone invented a new technique
Which allowed them to get a new result better than was previously possible
and this result is plausibly applicable to AI alignment

[-]Amalthea2y10

I don't think that essay does what you want. It seems to be about "you can't always capture the meaning of something by writing down defining a simple precise definition", while the complaint is that you're not using the word according to its widely agreed-upon meaning. If you don't want to keep explaining what you specifically mean by "breakthrough" in your title each time, you could simply change to a more descriptive word.

[-]Logan Zoellner2y2-2

A quick google search should show you that the way I am using the word "breakthrough" is in accordance with the common parlance.

[-][anonymous]2y40

How long do we normally have to wait to see if CryAnalyzer and Decoding Speech from Brain Recordings will replicate? Both of these have rather serious geopolitical implications.

Does human reasoning research automation like ThoughtCloning tend to get openly censored/refused or uncharacteristically weak responses on closed source systems like GPT4? That also has geopolitical implications I.e. indicates that providers at some point became wary of actors trying to use their systems for those kinds of applications (which further indicates that such actors already exist, rather than might exist in the future).

[-]amayhew2y30

The reviews on CryAnalyzer at least are pretty bad. I think we may still have a long way to go

[-]AprilSR2y31

This post introduced me to a bunch of neat things, thanks!

[-]Viliam2y20

New deepfake scam campaign claiming Mr Beast will give you an iPhone

I am just watching a YouTube ad where robotically speaking Stephen Colbert interviews robotically speaking Elon Musk about his latest invention, which is a quantum artificial intelligence investing on a financial market. You just have to click on the website, fill out a contact form, and an employee of the robotic Elon Musk will contact you later to tell you how you can most safely invest your money for a guaranteed profit.

I reported the ad. There is no "deepfake" category, but there is "impersonation", so I guess that will suffice.

[-]Adele Lopez2y20

What is the breakthrough: It was discovered that neural networks tend to produce “average” predictions on OOD inputs

This seems like somewhat good news for Goodhart's curse being easier to solve! (Mostly I'm thinking about how much worse it would be if this wasn't the case).

[-]TurnTrout2y40

Average predictions OOD is also evidence against goal-directed inner optimizers, in some sense; though it depends what counts as "OOD."

[This comment is no longer endorsed by its author]Reply

[-]Noosphere892y10

Yep, that's the big one. It suggests that NNs aren't doing anything very weird if we subject them to OOD inputs, which is evidence against the production of inner optimization, or at least not inner misaligned optimizers.

[This comment is no longer endorsed by its author]Reply

[-]TurnTrout2y40

I actually now think the OOD paper is pretty weak evidence about inner optimizers.

I think the OOD paper tells you what happens as the low-level and mid-level features stop reliably/coherently firing because you're adding in so much noise. Like, if my mental state got increasingly noised across all modalities, I think I'd probably adopt some constant policy too, because none of my coherent circuits/shards would be properly interfacing with the other parts of my brain. But I dont think that tells you much about alignment-relevant OOD.

[-]Maxime Riché2y30

I wonder if the result is dependent on the type of OOD.

If you are OOD by having less extractable information, then the results are intuitive.
If you are OOD by having extreme extractable information or misleading information, then the results are unexpected.

Oh, I just read their Appendix A: "Instances Where “Reversion to the OCS” Does Not Hold"
Outputting the average prediction is indeed not the only behavior OOD. It seems that there are different types of OOD regimes.

LESSWRONG
LW

LESSWRONG
LW

30

AI Alignment Breakthroughs this week (10/08/23)

30

30

AI Alignment Breakthroughs this week (10/08/23)

AI Lie Detector

Interpreting neurons in superposition

AI Alignment Breakthroughs this week

AI Agents

Mechanistic Interepretability

AI Explainability

Making AI Do what we Want

Uplifting

Mind Uploads

Simulation

AI Art

Things that are not AI Alignment