Aleksi Liimatainen — LessWrong

LESSWRONG
LW

Replying toCritical review of Christiano's disagreements with Yudkowsky

Critical review of Christiano's disagreements with Yudkowsky

I feel like this "back off and augment" is downstream of an implicit theory of intelligence that is specifically unsuited to dealing with how existing examples of intelligence seem to work. Epistemic status: the idea used to make sense to me and apparently no longer does, in a way that seems related to the ways i've updated my theories of cognition over the past years.

Very roughly, networking cognitive agents stacks up to cognitive agency at the next level up easier than expected and life has evolved to exploit this dynamic from very early on across scales. It's a gestalt observation and apparently very difficult to articulate into a rational argument. I could... (read more)

Replying toTrust your intuition - Kahneman's book misses the forest for the trees

Aleksi Liimatainen2y

Trust your intuition - Kahneman's book misses the forest for the trees

I for one found this post insightful though I wouldn't necessarily call it a book review.

Going against the local consensus tends to go over better when it's well-researched and carefully argued. This one unfortunately reads as little more than an expression of opinion, and an unpopular one at that.

Replying toRed Pill vs Blue Pill, Bayes style

Aleksi Liimatainen2y

Red Pill vs Blue Pill, Bayes style

Yeah, this seems close to the crux of the disagreement. The other side sees a relation and is absolutely puzzled why others wouldn't, to the point where that particular disconnect may not even be in the hypothesis space.

When a true cause of disagreement is outside the hypothesis space the disagreement often ends up attributed to something that is in the hypothesis space, such as value differences. I suspect this kind of attribution error is behind most of the drama I've seen around the topic.

Replying toRed Pill vs Blue Pill, Bayes style

Aleksi Liimatainen2y

Red Pill vs Blue Pill, Bayes style

Nathaniel is offering scenarios where the problem with the course of action is aesthetic in a sense he finds equivalent. Your question indicates you don't see the equivalence (or how someone else could see it for that matter).

Trying to operate on cold logic alone would be disastrous in reality for map-territory reasons and there seems to be a split in perspectives where some intuitively import non-logic considerations into thought experiments and others don't. I don't currently know how to bridge the gap given how I've seen previous bridging efforts fail; I assume some deep cognitive prior is in play.

Replying toWhy no Roman Industrial Revolution?

Aleksi Liimatainen3y

Why no Roman Industrial Revolution?

My suspicion is that it has to do with cultural-cognitive developments generally filed under "religion". As it's little more than a hunch and runs somewhat counter to my impression of LW mores, I hesitate to discuss it in more depth here.

Replying toConditional on living in a AI safety/alignment by default universe, what are the implications of this assumption being true?

Aleksi LiimatainenJul 18, 2023

Conditional on living in a AI safety/alignment by default universe, what are the implications of this assumption being true?

Conditional on living in an alignment-by-default Universe, the true explanations for individual and societal human failings must be consistent with alignment-by-default. Have we been deviated from the default by some accident of history or does alignment just look like a horrid mess somehow?

Replying toConditional on living in a AI safety/alignment by default universe, what are the implications of this assumption being true?

Aleksi Liimatainen3y

Conditional on living in a AI safety/alignment by default universe, what are the implications of this assumption being true?

You're describing an alignment failure scenario, not a success scenario. In this case the AI has been successfully instructed to paperclip-maximize a planned utopia (however you'd do that while still failing at alignment). Successful alignment would entail the AI being able and willing to notice and correct for an unwise wish.

Replying toDo we automatically accept propositions?

Aleksi LiimatainenJul 12, 2023

Do we automatically accept propositions?

I don't think it's possible to evaluate a model without inhabiting it. Therefore we must routinely accept (and subsequently reject) propositions.

Replying toCollective Identity

Aleksi Liimatainen3y

Collective Identity

Michal Levin's paper The Computational Boundary of a "Self" seems quite relevant re identity fusion. The paper argues that larger selves emerge rather readily in living systems but it's not quite clear to me whether that would be an evolved feature of biology or somehow implicit in cognition-in-general. Disambiguating that seems like an important research topic.

Replying toWhat in your opinion is the biggest open problem in AI alignment?

Aleksi LiimatainenJul 04, 2023*

What in your opinion is the biggest open problem in AI alignment?

I think the biggest thing holding AI alignment back is a lack of general theory of alignment. How do extant living system align, and what to?

The Computational Boundary of a "Self" paper by Michael Levin seems to suggest one promising line of inquiry.

AI alignment is a wicked problem. It won't be solved by any approach that fails to grapple with how deeply it mirrors self-alignment, child alignment, institutional alignment and many others.

Is there a culture overhang?

Aleksi Liimatainen

Culture is adapted for learnability, transmissibility and recombinability for humans. To the extent AI and natural intelligence operate on similar principles, these adaptations should be expected to carry over when culture is used as a training dataset for AI. If so, an AI trained on culture would catch up to the cultural state of the art fast but have more trouble progressing beyond it.

If the above holds, one would expect rapid AI progress up to the frontiers of human culture and nearby low-hanging fruit, after which progress would slow. More speculatively, depending on our compatibility with advancing AI's outputs, humans could have an unexpectedly easy time keeping up with the AI's (relatively slower) acceleration.

Are we currently living in a culture overhang? If so, how does it affect the picture of AI timelines?

Upgrading a primate didn't make it strongly superintelligent relative to other primates. The upgrades made us capable of recursively improving our social networking; that was what made the difference.

If you raised a child as an ape, you'd get an ape. That we seem so different now is due to the network effects looping back and upgrading our software.

Edit: On reflection, in many situations insulation from financial pressures may be a good thing, all else being equal. That still leaves the question of how to keep networks in proper contact with reality. As our power increases, it becomes ever easier to insulate ourselves and spiral into self-referential loops.

If civilization really is powered by network learning on the organizational level, then we've been doing it exactly wrong. Top-down funding that was supposed to free institutions and companies to pursue their core competencies has the effect of removing reality-based external pressures from the organization's network structure. It certainly seems as if our institutions have become more detached from reality over time.

Have organizations been insulated from contact with reality in other ways?

Are there spaces for extremely short-form rationality content?

Aleksi Liimatainen

I'm experimenting with very short-form, compressed evocations of various mental adjustments. (Yes, I spend a lot of time on Twitter. Why do you ask?) For example, there's an adjunct to the idea of carving reality at the joints that could go something like this: "Reality is interconnected. When you carve it at the joints, what are you severing and how will you stitch it back together?"

Do you think there is a purpose for extreme shortform like this? If yes, were should it go?

Hebbian Learning Is More Common Than You Think

Aleksi Liimatainen

Epistemic status: locating the hypothesis. I have my private confidence but you shouldn't take my word for it.

I originally got the idea from this video interview of professor Richard A. Watson where he explains how learning networks could arise naturally and be an important factor in evolution. The meat starts around 15 minutes in.

First, an intuition pump: If you had a suspended network of non-ideal springs, loading the network would slightly change the resting lengths of the springs for the next iteration. In effect, even a spring network has (admittedly limited) learning potential.

Dr. Watson focuses on evolution and makes his strongest case in that domain. In short, ecological stressors impose evolutionary pressures... (read 185 more words →)

AI alignment might be an ontology problem. It seems to me that the referent of "human values" is messy and fractal exactly in the way that our default modes of inquiry can't get a grip on. If you stick to those modes of inquiry, it's hard to even notice.

Is civilizational alignment on the table?

Aleksi Liimatainen

It seems to have become apparent that existing social systems are functionally unaligned: organizational and market dynamics are oft cited as an important factor exacerbating AI danger. It seems to me that progress in civilizational alignment would be instrumental for increasing the chances of succesfully navigating the AI alignment challenge and has significant theoretical overlap as both are in the class of agent alignment.

Are there people or groups actively looking into civilizational alignment and is there cross-pollination with AI alignment work?

Any goal specification implies a cluster of target states that in turn can be taken as a measure. Given that, it seems that any goal specification short of a complete extrapolation of the goal-setter's volition is subject to Goodhart's Law. If so, we should expect everything short of direct action by the goal-setter to be goodharted all the time.

From this perspective, it seems like a miracle that any large-scale action works at all. How do we do it?

Aleksi Liimatainen's Shortform

Aleksi Liimatainen

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.