Live Theory Part 0: Taking Intelligence Seriously

I agree that many people focus on "superhuman intelligence, sooner or later or never" and ignore the possibility of how a human-level or even below-human-level intelligence can change the world if it becomes cheap enough.

I heard a rumor about someone who made lots of money by telling an AI to generate thousands of e-books and selling them on Amazon. Previously people did a similar thing by scraping Wikipedia pages and selling them as books, but this is different -- the books are not obviously scam, they are probably just mediocre; and as the AIs get better, the new books will get better, until enough people realize they do not need to buy them, because they can ask an AI directly.

I imagine similar "shovelware" everywhere. There are probably already many people accepting programming or artistic tasks online, giving them to an AI, and getting paid for the result. If you want to make money this way, do it quickly, because at some moment people will realize they can ask an AI directly.

This may be a convenient time to establish fake artistic credentials. Create a page on Patreon, and ask an AI to make deliberately bad pictures... gradually improving, so that people can see your artistic progress. Later, when the backlash against AI art comes, people will stop trusting art from random strangers online, but you will have established credentials as an actual human artist, so you can charge extra for "hand made" art.

At some moment, software development will become "put an AI into a box, and tell it what to do". For example, to program a new Tetris game, you will simply take a generic AI and tell it "do Tetris"; and that's all. To create an entirely new game, you will provide a verbal explanations, a few example screenshots (generated by another AI), and tell the AI "do this".

There will be many "experts" online who will basically be an interface between you and an AI. You will call a human doctor or a financial advisor or a motivational coach, and the person will listen to you, while silently typing questions to an AI, and then will tell you the results, and charge you $100 per hour.

If many cheap appliances get an AI component, I expect most things to be commanded by voice. An AI with a camera will be a powerful combination, the appliance will be able to see what is there, and make some logical conclusion. Your gas stove will tell you "I think the food might be ready, would you like to check it?", your fridge will remind you that you are out of butter, and by the way the box of milk is already open for a week you might want to check it. Toys will talk to children, and accept verbal instructions.

...and this all will be used for surveillance, because of course an intelligent device can provide much better information about you. Not just what you do, but whether you smile or frown when you do it; whether you also use devices made by competition; whether it seems like you have extra money to buy an upgrade; what other devices are missing at your home.

...also, advertising everywhere. The toys will tell your kids to buy merchandise. The kitchen appliances will recommend you to buy food from specific distributors. The computer games will model their villains based on politicians who propose to regulate advertising and surveillance.

Online, it will be possible to create a cheap "Matrix" for people. Imagine being on a web forum where everyone else is a bot, and you are the only human there. But you will never know, because the bots will post interesting things, debate with you, flirt with you, invite you to read other websites... and occasionally do the thing they were made for, like try to make you buy some products or vote for a certain party. You will be a follower of a conspiracy theory that was tailor-made for you; thousands bots coming with random ideas, and whatever you upvote, there will be more of that, and whatever you write, will be incorporated in the theory, as long as it can be made compatible with the main goal. People online will never feel alone again. At the beginning, this will be an operation targeted at specific people (they may be met by actual humans who introduce them to a web forum where they receive a warm approval from bots, and then everything will be handled by the bots), but as it gets cheaper, they will try to catch everyone in their own separate "Matrix".

Even the real people you meet will often be working for similar companies -- their role will be to sometimes meet you offline, to provide evidence that they are actual humans, but as soon as you return to the screen, you will only interact with a bot that simulates them. So once in a year your private "Matrix" can have an offline meetup, where you meet dozen people from a specialized agency who will role-play members of your web forum. (Their job is to go to a different meetup every week with a different identity.)

Or to turn it around, people will use AIs to pretend to be them, and outsource the tasks such as calling your parents, or maintaining contacts with your former classmates and colleagues. Hey, it costs nothing to let an AI handle a personal relationship you don't really care about, and you never know when you can get some useful service from "an old friend". The future of seduction is to have a bot seduce someone and arrange a date, and you just go there to have sex with them. The future of prostitution is the same, only you also let the bot find out some pretext why the other person should send you money. The girlfriend you once met at a party, and since then you call each other every day and talk a lot about various topics and she is really smart and nice, but she is too busy with her work and she is living in another city, and how she has a financial problem, but if you send her some money she will be able to quit her current job and move closer to you... I am sorry, but she is not real. She is just a girl who goes to parties, collects phone numbers from guys and enters them in a computer; that's all, everything that happened afterwards was fake.

^{^}

Challenging and supporting, especially through the frustration of freezing this written artefact before I can avail the spaciousness of the fluidic era.

^{^}

Alternative terms include “adaptive theory”, “fluid theory”, "flexible theory"; where the theories themselves are imbued with some intelligence

^{^}

This will be elaborated in an upcoming post on risks. “Slow takeoff” is an approximation that will have to do for now, but really it's much more like having lazy or mentally ill AI. If you're curious, here's the abstract:

Many AI safety folk have rightly raised issues of corrigibility. That once the target(/values) of an optimization process get locked in, it becomes extremely sticky. Attempts to change are up against the full optimization power behind fulfilment of the target. If this power goes beyond our abilities, any clever strategies to undermine it are just an enactment of our weaker optimization powers pitted against a stronger one, and will, tautologically, fail.
We aim here to bring to fore a subtler version of this issue: corrigibility of relevance. Where stickiness of value comes about not via strong monomaniacal attraction to the target, but by indifference to anything outside of what is considered meaningful. Indifference not as a side effect of strong optimization (such as in The AI knows but does not care), but by not entering attention. Indeed, this applies even in the absence of an overall optimizing process. We also point out how, contrary to intuitions that this makes AI more dangerous, it could also imply less conflict. We point out that optimization processes that occur within a zone of relevance are less likely to be rivalrous with others, contra broad instrumental convergence. We carefully argue that difficulty of corrigibility of relevance implies, further, that zones of relevance are unlikely (though not impossible) to expand, since there is no motivation to.
We also couch zones of relevance in naturalistic terms. This involves a new concept, of integrity, which is an embedded extension of coherence. The importance of the specific, physical modalities of "skin-in-the-game" to relevance is explored, via connections between physical integrity and mental integrity. We take up very concrete questions (such as "are hardware level architectural changes needed in order for shutdown to be a real problem?") from this lens, and argue for a fractal consequentialist model. This makes the safety-vs-capability dichotomy untenable (in a new way), giving rise, for example, to "fractal sharp left turn". We list many predictions of this model that are opposite to the original sharp left turn threat model, and gives rise to novel dangers that resemble S-risks from partial alignment.
Owing to issues of integrity, "simply" hooking up an intelligent, "unagentic" model with an agentic wrapper (eg. "GPT-7 may not be dangerous, but GPT-7 plus two thousand lines of python can destroy the world") is shown to be a lot more difficult. We explore, with examples, how simply "plugging in" does not lead to integration, and this lack of integrity creates lacunae of relevance that can be exploited to cause shutdown, no matter the cognitive powers of the machine. We cautiously make analogies to lack of integrity in humans (often manifesting as mental disorders, dissociation, and savantism). We note that disanalogies are more important because of differences in physical make-up of biological life vs machines, and how that gives us more time to deal with dangerous versions of the problems of shutdown / risks of autonomy. We end by attempting to point at the human/animal zone of relevance, which is quite philosophically challenging to do from the vantage of our own minds, but extremely important as we create minds very different from ours.

^{^}

Generally, it is a bit suspect to me, for an upcoming AI safety org (in this time of AI safety org explosion) to have, say, a 10 year timeline premise without anticipating and incorporating possibilities of AI transforming your (research) methodology 3-5 years from now. If you expect things to move quickly, why are you ignoring that things will move quickly? If you expect more wish-fulfilling devices to populate the world (in the meanwhile, before catastrophe), why aren't you wishing more, and prudently? An “opportunity model” is as indispensable as a threat model.

(In fact, "research methodological IDA" is not a bad summary of live theory, if you brush under the rug all the ontological shifts involved.)

^{^}

TJ comments:

I'd guess you'd want to be more explicit here that what you mean is that "human cognition, thought, and intellectual enterprise is itself going to go through radical transitions, and will reshape the human experience." It feels like you're implying this here, but is left as a silent gesture for the reader.

^{^}

An example of weirdness if you're hungry, in raw form. Inspired by a submission at a hackathon on live machinery.

Time structures are pretty centralized right now.
Do you know what a "Thurs" is? Would you collect solar days together into 7-length segments? Why is some old drama between Julius and Augustus deciding the day you go out and have fun wearing spooky costumes?
You could imagine throwing it all away and redoing it in a way that makes sense to you. Or better yet, to your local community. Maybe you would divide the day into powers of two and label them. Perhaps you like to orient to the lunar calendar, or to your menstrual cycle. You might find seasons to be only vaguely relevant if you're in the Maldives or elsewhere near the equator, and might orient via tides and monsoon or the migration patterns of whales.
But this is all a fantasy. If you need to feed you kids, you need to believe in a Thursday. You have your weekly standup on Thursdays, at the eleventh hour of the ante-meridian. If we want to coordinate, we need protocol. You need to catch your flight, and the pilot needs to show up on time. We need shared structure, shared reality, universal languages. Right?
The punchline, of course, is that this is the old way to scale. By replicating fixed structure. But when intelligence is cheap, you don't need static shared structure. Ubiquitous attentive infrastructure can create peer-to-peer structure as needed. So how would that work?
Let's take something simple: a postformal, prayer-based interface to replace calendly.
Instead of marking lines on a shared spatialization of time, your community can live by your local temporal rituals. You might use the rings of trees to commemorate new beginnings, or the births of children. Perhaps you will have a season of meeting others that lasts several weeks, based on your bipolar rhythms.
This is your calendly. You broadcast prayers to meet others during this time. Ambient intelligent infrastructure (ie. AI) can help you identify, broadcast and match this prayer with mine, since I want about 26 minutes of meeting time with you, as expressed in my wishes, to consult about durian-eating or whatever.
The flexibility of prayers allows for you to synchronously run into me during your meeting season. You (and I) get more lucky, rather than getting more controlling-power.
You can still have precision, in this. Your prayer might specify that. Control and precision might seem hard to decouple. But so do scale and fixed structure.
AI, if done right, can help with more interpersonalization, not hyperpersonalization. Far from hallucination and spam, it can lead to more unconforming to embodied truth, rather than conforming to epistemic bureacracy. Why should our cycles and seasons of living and meeting be filtered through rigid structure that is numb to our needs and infiltrates our meanings?
Just picking up a preferred grain of abstraction or redoing some grids creatively (as in xkcd: 28-Hour Day) is nice, but it doesn't free you to co-create the background view of your reality.

^{^}

(See more here: The Logistics of Distribution of Meaning )

The word “automation” does not distinguish numb-scripted-dissociated from responsive-intelligent-flowing. It usually brings to mind the former. So I avoid using it. More on this in the next post, but for now, a word might be “competence”. When you’re really in flow while dancing, you're not thinking. But it is the opposite of “automation”. If anything, you're more attuned, more sensitive.

This isn't “high-level” abstraction either. The high-low dichotomy is the good-old-fashioned way of looking at generalization, and does not apply with expanded sensitivity, as we'll see in this sequence.

The relevance of the choice "live" will hopefully also become clearer as we go. Meant to connote biological rather than machine metaphors, dialectics rather than deliverables, intentional stance, though not necessarily patienthood.

^{^}

It's not that people predict "no fundamentally new ways of looking at the world will occur" and therefore decide to exclude them in their stories. I think people exclude ontological shifts because it's very hard to anticipate an ontological shift.

(If you disagree, I'd ask the fundamental question of implied invisible work: what have you rejected? A la book recommendations.)

^{^}

Matt adds:

It might be worth stating more about these [risks] from the outset. Just to prevent anyone thinking you're some kind of e/acc full steamer techbro who isn't interested in safety

Indeed. The Necessary gate will cover much relevance to threat models. Apart from that, expect much much more on "infrastructural insensitivity" to articulate further vectors of risk.

^{^}

Abram helpfully adds:

I find the meeting example [see next quote block] much more compelling, where people get dynamically crafted versions of the meeting which can catch them up on the most important things they missed as a result of coming in late -- someone uses the phrase "lightbulb moment" in a context-dependent way, so the live interop unpacks the context for those who missed it.

I think the reason I like this better is that meetings invoke a more utilitarian mindset where we evaluate the usefulness of tools. Movies invoke a warm friendly mindset where we are more prone to horror over the seeming non-togetherness of the described scenario. If you want to shock people with the idea of such severely mediated reality, fine. But I think the meeting example will provoke less feeling of shock and undesirability, if you prefer that.

And so here is my meeting example pasted raw, which might be more palatable or more triggering:

So imagine you join a meeting on Zoom, a little late. AI summarizers can "catch you up". Instead of immediately chatting, you peruse the transcript/read the summary.
Now that's great, but a little boring. Ideally you'd have "immersive summary", a mini-meeting that you get to see live, a generated video summary. You watch that, and then join the real meeting.
Then you seamlessly transfer to the "real" meeting... except when Alice references a detail that wasn't given enough attention in the summary-virtual-meeting you initially watched to catch up. So before you get confused as Alice starts using the details assuming you were in the meeting, the AI interop again pulls you back into a simulation where generated Alice says "so, just to recall, some details I want to add" before dropping you back into reality.
And in fact, your replica will already have joined the meeting even though you were 10 minutes late. And you watch it, and might say "oh, no, that's not what I would have said"... and so, once you say that, everyone else will be put in a simulated meeting, where your replica says "I know I said earlier that X, but no, [I take that back/actually meant/wanted to say] ..." so everyone is again caught up.
Rather than being blocky or jarring, imagine a perfect stabilizer, like in a drone or in video-correction. It's uncanny, that despite all the noises, rapid responses can correct destabilizations quite magically and beautifully into a coherence.
This again blurs "construction" and "reality", doing such a fantastically better job that you will only occasionally need to rebel against this matrix. The real-constructed dichotomy makes less and less sense.

^{^}

More here: Live Machinery: An Interface Design Philosophy for Wholesome AI Futures

^{^}

NB. this is not brute, ham-fisted merging, for those worried. See also: Unity and diversity.

^{^}

also: centralization-heavy, preformationist, foundationalist, control-oriented. See The Logistics of Distribution of Meaning: Against Epistemic Bureaucratization

^{^}

This is a very expensive delivery mechanism for a koan/pointing-out instruction, to let go of the static-machine mythos, but there you go.

^{^}

Live Governance post coming up soon.

^{^}

See also this proposal for the commonalities in the two posts and limitations of existing approaches.

LESSWRONG
LW

LESSWRONG
LW

103

Live Theory Part 0: Taking Intelligence Seriously

103

Ω 40

103

Ω 40

Acknowledgements

Disclaimers

Taking Intelligence Seriously

Failing to Take Intelligence Seriously

Opportunities from Moderate Intelligence at Scale

Live Interfaces

Live Theory (preface)

Navigation

Live Theory (teaser)

Possibility