Adele Lopez

Wiki Contributions


Biology-Inspired AGI Timelines: The Trick That Never Works

You're missing the point!

Your arguments apply mostly toward arguing that brains are optimized for energy efficiency, but the important quantity in question is computational efficiency! You even admit that neurons are "optimizing hard for energy efficiency at the expense of speed", but don't seem to have noticed that this fact makes almost everything else you said completely irrelevant!

Biology-Inspired AGI Timelines: The Trick That Never Works

Going to try answering this one:

Humbali: I feel surprised that I should have to explain this to somebody who supposedly knows probability theory. If you put higher probabilities on AGI arriving in the years before 2050, then, on average, you're concentrating more probability into each year that AGI might possibly arrive, than OpenPhil does. Your probability distribution has lower entropy. We can literally just calculate out that part, if you don't believe me. So to the extent that you're wrong, it should shift your probability distributions in the direction of maximum entropy.

[Is Humbali right that generic uncertainty about maybe being wrong, without other extra premises, should increase the entropy of one's probability distribution over AGI, thereby moving out its median further away in time?]

The uncertainty must already be "priced in" your probability distribution. So your distribution and hence your median shouldn't shift at all, unless you actually observe new relevant evidence of course.

Visible Thoughts Project and Bounty Announcement

This plausibly looks like an existing collection of works which seem to be annotated in a similar way:

Why Study Physics?

Physics also seems to help with clear philosophical thinking, and has lots of unintuitive stuff that trains the skill of looking past your models and into the Real Thing. Of Deep Fundamental Principles, I also think physics has some of the easier ones to see, like the conservation principles (once you get why a physicist can be so confident of the non-existence of a perpetual motion machine, you can start to imagine how there could be other Deep Principles which could justify seemingly excessive confidence about other complicated domains).

On the other hand, mediocre physicists tend to be too arrogant, and the current generation of physicists seems to have lost the way in some important (but hard to pin down) sense.

Christiano, Cotra, and Yudkowsky on AI progress

That seems a bit uncharitable to me. I doubt he rejects those heuristics wholesale. I'd guess that he thinks that e.g. recursive self improvement is one of those things where these heuristics don't apply, and that this is foreseeable because of e.g. the nature of recursion. I'd love to hear more about what sort of knowledge about "operating these heuristics" you think he's missing!

Anyway, it seems like he expects things to seem more-or-less gradual up until FOOM, so I think my original point still applies: I think his model would not be "shaken out" of his fast-takeoff view due to successful future predictions (until it's too late).

Christiano, Cotra, and Yudkowsky on AI progress

It seems like Eliezer is mostly just more uncertain about the near future than you are, so it doesn't seem like you'll be able to find (ii) by looking at predictions for the near future.

Matthew Barnett's Shortform

I lean toward the foom side, and I think I agree with the first statement. The intuition for me is that it's kinda like p-hacking (there are very many possible graphs, and some percentage of those will be gradual), or using a log-log plot (which makes everything look like a nice straight line, but are actually very broad predictions when properly accounting for uncertainty). Not sure if I agree with the addendum or not yet, and I'm not sure how much of a crux this is for me yet.

Yudkowsky and Christiano discuss "Takeoff Speeds"

Spending money on R&D is essentially the expenditure of resources in order to explore and optimize over a promising design space, right? That seems like a good description of what natural selection did in the case of hominids. I imagine this still sounds silly to you, but I'm not sure why. My guess is that you think natural selection isn't relevantly similar because it didn't deliberately plan to allocate resources as part of a long bet that it would pay off big.

Adele Lopez's Shortform

Re: Yudkowsky-Christiano-Ngo debate

Trying to reach toward a key point of disagreement.

Eliezer seems to have an intuition that intelligence will, by default, converge to becoming a coherent intelligence (i.e. one with a utility function and a sensible decision theory). He also seems to think that conditioned on a pivotal act being made, it's very likely that it was done by a coherent intelligence, and thus that it's worth spending most of our effort assuming it must be coherent.

Paul and Richard seem to have an intuition that since humans are pretty intelligent without being particularly coherent, it should be possible to make a superintelligence that is not trying to be very coherent, which could be guided toward performing a pivotal act.

Eliezer might respond that to the extent that any intelligence is capable of accomplishing anything, it's because it is (approximately) coherent over an important subdomain of the problem. I'll call this the "domain of coherence". Eliezer might say that a pivotal act requires having a domain of coherence over pretty much everything: encompassing dangerous domains such as people, self, and power structures. Corrigibility seems to interfere with coherence, which makes it very difficult to design anything corrigible over this domain without neutering it.

From the inside, it's easy to imagine having my intelligence vastly increased, but still being able and willing to incoherently follow deontological rules, such as Actually Stopping what I'm doing if a button is pressed. But I think I might be treating "intelligence" as a bit of a black box, like I could still feel pretty much the same. However, to the extent where I feel pretty much the same, I'm not actually thinking with the strategic depth necessary to perform a pivotal act. To properly imagine thinking with that much strategic depth, I need to imagine being able to see clearly through people and power structures. What feels like my willingness to respond to a shutdown button would elide into an attitude of "okay, well I just won't do anything that would make them need to stop me" and then into "oh, I see exactly under what conditions they would push the button, and I can easily adapt my actions to avoid making them push it", to the extent where I'm no longer being being constrained by it meaningfully. From the outside view, this very much looks like me becoming coherent w.r.t the shutdown button, even if I'm still very much committed to responding incoherently in the (now extremely unlikely) event it is pushed.

And I think that Eliezer foresees pretty much any assumption of incoherence that we could bake in becoming suspiciously irrelevant in much the same way, for any general intelligence which could perform a pivotal act. So it's not safe to rely on any incoherence on part of the AGI.

Sorry if I misconstrued anyone's views here!

Ngo and Yudkowsky on AI capability gains

There's more than just differential topology going on, but it's the thing that unifies it all. You can think of differential topology as being about spaces you can divide into cells, and the boundaries of those cells. Conservation laws are naturally expressed here as constraints that the net flow across the boundary must be zero. This makes conserved quantities into resources, for which the use of is convergently minimized. Minimal structures with certain constraints are thus led to forming the same network-like shapes, obeying the same sorts of laws. (See chapter 3 of Grady's Discrete Calculus for details of how this works in the electric circuit case.)

Load More