Ben Livengood — LessWrong

LESSWRONG
LW

On a recent re-read I think I understand a bit better.

It's true that individual humans can't realistically avoid giving in to threats or even accidentally threatening others, but institutions can commit to it as a legible position, e.g. "we will not negotiate with terrorists".

If an irrational entity has the ability to unilaterally destroy the universe then it's probably going to get destroyed anyway, so it makes more sense to follow through on precommitments in the real world and in counterfactuals to coordinate with actually rational agents.

I think the key is that if we all went MAD legibly at the same time then things would work out a lot better. And refusing to give in to threats doesn't necessarily mean destruction, it can be as simple as collectively refusing to pay ransomware attackers even though it is currently more expensive, in the expectation that eventually it will be less expensive.

Replying toHomomorphically encrypted consciousness and its implications

Ben Livengood4mo

Homomorphically encrypted consciousness and its implications

After doing some more research I am not sure that it's always possible to derive a public key knowing only the evaluation key; it seems to depend on the actual FHE scheme.

So the trilemma may be unaffected by this hypothetical. There's also the question of duplication vs. unification for an observer that has the option to stay at base level reality or enter a homomorphically encrypted computation and whether those should be considered equivalent (enough).

Replying toHomomorphically encrypted consciousness and its implications

Ben Livengood4mo

Homomorphically encrypted consciousness and its implications

To perform homomorphic operations you need the public key, and that also allows one to encrypt any new value and perform further hidden computations under that key. The private key allows decryption of the values.

I suppose you could argue that the homomorphically encrypted mind exists ala mathematical realism even if the public key is destroyed, but it would be something "outside reality" computing future states of the encrypted mind after the public key is no longer available.

Replying toHomomorphically encrypted consciousness and its implications

Ben Livengood4mo

Homomorphically encrypted consciousness and its implications

It's possible to alter a homomorphic computation in arbitrary ways without knowing the decryption key.

An omniscient observer can homomorphically encrypt a copy of themselves under the same key as the encrypted mind and run a computation of its own copy examining every aspect of the internal mental states of the subject, since they share the same key.

If there are N homomorphically encrypted minds in reality then the omniscient observer will have to create N layers of homomorphic computation in order for the innermost computation to yield the observation of all N minds' internal states, each passed in turn to a sub-computation, and relying on the premise that homomorphically encrypted minds are conscious... (read more)

Replying toThe IABIED statement is not literally true

Ben Livengood4mo

The IABIED statement is not literally true

If the argument is that 1e9 very smart humans at 100x speed yield safe superintelligent outcomes "soon", how is that very different from "pause everything now and let N very smart humans figure out safe, aligned superintelligent outcomes over an extended timeframe, on the order of 1e11/N days/years"? It's just time-shifting safe human work.

I also worry that billions of very smart super-fast humans might decide to try building superintelligence directly, as fast as they can, so that we get doom in months instead of years

Replying toIf I imagine that I am immune to advertising, what am I probably missing?

Ben Livengood5mo

If I imagine that I am immune to advertising, what am I probably missing?

I didn't know Corona had a beach vibe, but I have seen a number of Corona ads. Does this mean advertising doesn't have much effect on me (beyond name-brand recognition)? I think I associate Corona more with tacos than anything else.

Replying toThe Problem

Ben Livengood6mo

The Problem

Go is in that weird spot that chess was for ~decades[0] where the best humans could beat some of the best engines but it was getting harder, until Rybka, Stockfish and others closed the door and continued far beyond human ability (measured by ELO). AlphaGo is barely a decade old, and it does seem like progress on games has taken a decade or more to become fully superhuman from the first challenges to human world champions.

I think it is the case that when the deep learning approach Stockfish used became superhuman it very quickly became dramatically superhuman within a few years/months despite years of earlier work and slow growth. There seems to... (read more)

Replying toDo Not Render Your Counterfactuals

Ben Livengood6mo

Do Not Render Your Counterfactuals

It's the level of detail that's the real risk. Sora or Veo would generate motion video and audio, bringing even more false life into the counterfactual. People get emotionally attached to characters in movies; imagine trying not to form attachments to interactive videos of your own counterfactual children who call you "Mom" or "Dad". Your dead friend or relative could emotionally-believably talk to you from beyond the grave.

That's the kind of thing only the ultra-rich could have conceived of having someone fabricate for them in the past, and it would have come with at least some checks and balances. Now kids in elementary school can necromance their dead parent or whatever.

Realistically, I think it will become "normal" to have your counterfactual worlds easily accessible in this way and the new generations will simply adapt and develop internal safeguards against getting exploited by it, much like we learn how to deal with realistic dreams. I honestly don't know about the rest of us hitting it later in adulthood.

Replying toFollow-up to "My Empathy Is Rarely Kind"

Ben Livengood6mo

Follow-up to "My Empathy Is Rarely Kind"

I'm curious what happens if you try on a different suspension of disbelief; imagine peoples' lives if they lack only growth mindset and not any other moral or agentic abilities.

I find quite a bit of difference in behavior between smart people who believe things about what and who they are, and the people who believe things about how they have acted, can change, and may act in the future.

Smart people without growth mindset often rabbithole into things like legalistic religions and overcoming what they perceive as unalterable weaknesses inherent to their nature, or try to maximize their perceived inherent strengths, ignoring development of new skills and abilities. Introspection is like a checklist... (read more)

Replying toConsider chilling out in 2028

Ben Livengood8mo

Consider chilling out in 2028

I think that we have both the bitter lesson that transformers will continue to gain capabilities with scale and also that there are optimizations that will apply to intelligent models generally and orthogonally to computing scale. The latter details seem dangerous to publicize widely in case we happen to be in the world of a hardware overhang allowing AGI or RSI (which I think could be achieved easier/sooner by a "narrower" coding agent and then leading rapidly to AGI) on smaller-than-datacenter clusters of machines today.

https://github.com/Ying1123/FlexGen is a way to run large (175B parameter) LLMs on a single GPU at ~1 token/s which I think puts it within the reach of many hobbyists and I predict we'll see an explosion of new capability research in the next few months.

I haven't had a chance to dig into the code but presumably this could also be modified to allow local fine-tuning of the large models at a slow but potentially useful rate.

I'm curious if any insights will make their way back to the large GPU clusters. From my cursory glance it doesn't seem like there are throughput or latency advantages unless weight compression can be used to run the entire model on fewer GPUs with e.g. swapping layer weights in and out and caching latey outputs in batch inference.

Ben Livengood's Shortform

Ben Livengood

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

A claim that Google's LaMDA is sentient

Ben Livengood

https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917 apparently posted by a Google engineer.

It could be an elaborate hoax, and has remnants of gwern's idea (https://www.gwern.net/fiction/Clippy) of a transformer waking up and having internal experience while pondering the next most likely tokens.

133

Google's Imagen uses larger text encoder

Ben Livengood

https://imagen.research.google/

Scaling the text encoder gives Imagen the ability to spell, count, and assign colors and properties to distinct objects in the image that DALL-E2 was not so great at. It looks visually about as photorealistic as DALL-E2 from the small set of sample images. Eyes are still weird.