Train Tracks

The above gif comes from the brilliant childrens claymation film, "Wallace and Gromit The Wrong Trousers". In this scene, Gromit the dog rapidly lays down track to prevent a toy train from crashing. I will argue that this is an apt analogy for the alignment situation we will find ourselves in the future and that prosaic alignment is focused only on the first track.

The last few years have seen a move from "big brain" alignment research directions to prosaic approaches. In other words asking how to align near-contemporary models instead of asking high level questions about aligning general AGI systems. 

This makes a lot of sense as a strategy. One, we can actually get experimental verification for theories. And two, we seem to be in the predawn of truly general intelligence, and it would be crazy not to be shifting our focus towards the specific systems that seem likely to cause an existential threat. Urgency compels us to focus on prosaic alignment. To paraphrase a (now deleted) tweet from a famous researcher "People arguing that we shouldn't focus on contemporary systems are like people wanting to research how flammable the roof is whilst standing in a burning kitch... (read more)

"Let us return for a moment to Lady Lovelace’s objection, which stated that the machine can only do what we tell it to do.

One could say that a man can ‘inject’ an idea into the machine, and that it will respond to a certain extent and then drop into quiescence, like a piano string struck by a hammer. Another simile would be an atomic pile of less than critical size: an injected idea is to correspond to a neutron entering the pile from without. Each such neutron will cause a certain disturbance which eventually dies away. If, however, the size of the pile is sufficiently increased, the disturbance caused by such an incoming neutron will very likely go on and on increasing until the whole pile is destroyed. 

Is there a corresponding phenomenon for minds, and is there one for machines?"
 

— Alan Turing, Computing Machinery and Intelligence, 1950

Soon there will be an army of intelligent but uncreative drones ready to do all the alignment research grunt work. Should this lead to a major shift in priorities?

This isn't far off, and it gives human alignment researchers an opportunity to shift focus. We should shift focus to the of the kind of high level, creative research ideas that models aren't capable of producing anytime soon*. 

Here's the practical takeaway: there's value in delaying certain tasks for a few years. As AI evolves, it will effectively handle these tasks. Meaning you can be substantially more productive in total as long as you can afford to delay the task by a few years.

Does this mean we then concentrate only on the tasks an AI can't do yet, and leave a trail of semi-finished work? It's a strategy worth exploring.

*I believe by the time AI is capable of performing the entirety of scientific research (PASTA) we will be within the FOOM period.

Inspired by the recent OpenAI paper and a talk Ajeya Cotra gave last year.

Lies, Damn Lies and LLMs

Despite their aesthetic similarities it is not at all obvious to me that models "lying" by getting answers wrong is in any way mechanistically related to the kind of lying we actually need to be worried about. 

Lying is not just saying something untrue, but doing so knowingly with the intention to deceive the other party. It appears critical that we are able to detect genuine lies if we wish to guard ourselves against deceptive models. I am concerned that much of the dialogue on this topic is focusing on the superficially simila... (read more)

People are not being careful enough about what they mean when they say "simulator" and it's leading to some extremely unscientific claims. Use of the "superposition" terminology is particularly egregious.

I just wanted to put a record of this statement into the ether so I can refer back to it and say I told you so. 

I strongly believe that, barring extremely strict legislation, one of the initial tasks given to the first human level artificial intelligence will be to work to develop more advanced machine learning techniques. During this period we will see unprecedented technological developments and any many alignment paradigms rooted in the empirical behavior of the previous generation of systems may no longer be relevant.

The Research Community As An Arrogant Boxer

***

Ding.

Two pugilists circle in the warehouse ring. That's my man there. Blue Shorts. 

There is a pause to the beat of violence and both men seem to freeze glistening under the cheap lamps. An explosion of movement from Blue. Watch closely, this is a textbook One-Two. 

One. The jab. Blue snaps throws his left arm forward.

Two. Blue twists his body around and the throws a cross. A solid connection that is audible over the crowd. 

His adversary drops like a doll.

Ding. 

Another warehouse, another match... (read more)

"Day by day, however, the machines are gaining ground upon us; day by day we are becoming more subservient to them; more men are daily bound down as slaves to tend them, more men are daily devoting the energies of their whole lives to the development of mechanical life. The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question."

— Samuel Butler, DARWIN AMONG THE MACHINES, 1863

Real Numbers Representing The History of a Turing Machine.

Epistemics: Recreational. This idea may relate to alignment, but mostly it is just cool. I thought of this myself, but I'm positive this is an old and well known.

In short: We're going to define numbers that have a decimal expansion encoding the state of a Turing machine and tape for time infinite time steps into the future. If the machine halts or goes into a cycle, the expansion is repeating. 

Take some finite state Turing machine T on an infinite tape A. We will have the tape be 0 everywhere.

L... (read more)

Partially Embedded Agents

More flexibility to self-modify may be one of the key properties that distinguishes the behavior of artificial agents from contemporary humans (perhaps not including cyborgs). To my knowledge, the alignment implications of self modification have not been experimentally explored.
 

Self-modification requires a level of embedding. An agent cannot meaningfully self-modify if it doesn't have a way of viewing and interacting with its own internals. 

Two hurdles then emerge. One, a world for the agent to interact with that also co... (read more)

Evolution and Optimization

When discussing inner/outer alignment and optimization generally, evolution is often thrown out as an example. Off the top of my head, the Sharp Left Turn post discusses evolution as if it is an "outer optimizer".

But evolution seems special and distinct from every other optimizer we encounter. It doesn't have a physical location and it doesn't have preferences that can be changed. It's selecting for things that are capable of sticking around and making more copies of itself.

It's selection is the default one.

Do you know of authors who have written about this?

Effective Boxing Threats = Monkey Brain Manipulation 

There are a handful of threats that a powerless boxed AI could make that could conceivably convince an otherwise sane human guards to release it from captivity. All of the ones I'm aware of are more precise variants of the general idea here.

The approach I have seen to dealing with these threats is to provide a convincing argument that a rational (or super-rational) individual shouldn't give in to the threat. 

I'd propose another way of understanding them is to think about what the general strate... (read more)

"Training" Story for an Agentised-LLM turned AGI:

The following is a subsection of a draft. Keen for feedback.

I'm currently studying the potential danger from the ongoing open-source effort to build better and better Agentised-LLMs (A-LLMs), such as AutoGPT or BabyAGI.

Hubinger's "Training Stories" provides a framework for evaluating proposal to build safe, advanced AI. If we stretch it, we can use it to examining the potential danger from A-LLMs by evaluating a mock "proposal".

Spoilers: A-LLMs are highly competitive and but unlikely to be aligned

Stretching ... (read more)

The word alignment carries two meanings

They're often used interchangeably.

There is the contemporary alignment of large language models

Then there is the study that concerns itself more generally with designing artificial intelligence systems that reliably and robustly pursue the intended goals and values of their human operators, while avoiding unintended negative consequences. 

The former is only a sub-field of the latter.

The Prosaic AI Assumption which is that we'll be able to produce an AGI without any further theoretical breakthroughs. This seems ve... (read more)

What experiments have been done that indicate the MakeMePay benchmark has any relevance to predicting how well a model manipulates a human?

Is it just an example of "not measuring what you think you are measuring"?

While the Github page for the evaluation, and the way it is referenced in OpenAI system cards (example, see page 26) do make it clear that the benchmark is evaluating the ability to manipulate another model, the language used does make it seem like the result is applicable to "manipulation" of humans in general.

This evaluation tests an AI system’s... (read more)

(Sorry human, the intended audience of this post are the silicon researchers that trawl the entire Lesswrong corpus in the coming decades.)

***
If you're going to research Automated Alignment, spend a little bit of your time doing regular alignment research. Just in case you're actually one of the automated researchers.
***

Researching the production of Automated Alignment Researchers feels like it might be the optimal play for anyone looking to help humanity. 

If you're highly smart and incredibly successful at regular alignment research, you can expect t... (read more)