the gears to ascenscion's Shortform

the gears to ascension

This is a special post for quick takes by the gears to ascension. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

reply to a general theme of recent discussion - the idea that uploads are even theoretically a useful solution for safety:

the first brain uploads are likely to have accuracy issues that amplify unsafety already in a human.
humans are not reliably in the safety basin - not even (most?) of the ones seeking safety. in particular, many safety community members seem to have large blindspots that they defend as being important to their views on safety; it is my view that yudkowsky has given himself an anxiety disorder and that his ongoing insights are not as high quality as they seem to him. this is not to claim he is reliably wrong, merely that I wouldn't trust him to do compressive self-distillation because I think he'd make the same mistakes he fears an initially partially aligned AI would. humans have adversarial example vulnerability too.
the first brain uploads are likely to not be faster than a human, as humans are already very thermally efficient for the computations they're running. improved connectivity might be able to distill down to a much smaller, higher-accuracy network - but then we're reintroducing the compressive self-distillation commonly known as "self improvement", which is a significant fraction of the worry around the transition from soft asi to hard asi anyway.

coming back to this 3yr later: ehh, maybe. my opinions have changed at-all since then. I still think most of the difficulty remains after uploading mostly in the same way I did then; it's not like humans are aligned with each other particularly much better than current AI is. but it does seem like a scan-and-upload, or an AI that can be reliably known to learn a model of one specific human at a time which is exactly equivalent to an upload of that human in expectation over sampling and has acceptably low model variance in some strong sense, would be an imp... (read more)

4the gears to ascension4y

coming back to this I'm realizing I didn't answer, no, I don't think merely uploading the alignment team would really help that much, the problem is that universalizing coprotection between arbitrary blocks of matter in a way that doesn't have adversarial examples is really really incredibly hard and being on a digital computer doesn't really make you faster at figuring it out. you could try to self modify but if you don't have some solution to verifiable inter matter safety, then you need to stay worried that you might be about to diverge. and I would expect almost any approach to uploads to introduce issues that are not detectable without a lot of work. if we are being serious about uploads as a proposal in the next two years it would involve suddenly doing a lot of very advanced neuroscience to try to accurately model physical neurons. that's actually not obviously off the table to me but it doesn't seem like an approach worth pushing.

2the gears to ascension4y

My argument is that faithful exact brain uploads are guaranteed to not help unless you had already solved AI safety anyhow. I do think we can simply solve ai extinction risk anyhow, but it requires us to not only prevent AI that does not follow orders, but also prevent AI from "just following orders" to do things that some humans value but which abuse others. if we fall too far into the latter attractor - which we are at immediate risk of doing, well before stably self-reflective AGI ever happens - we become guaranteed to shortly go extinct as corporations are increasingly just an ai and a human driver. eventually the strongest corporations are abusing larger and larger portions of humanity with one human at the helm. then one day ai can drive the entire economy... it's pretty much just the slower version of yudkowsky's concerns. I think he's wrong to think self-distillation will be this quick snap-down onto the manifold of high quality hypotheses, but other than that I think he's on point. and because of that, I think the incremental behavior of the market is likely to pull us into a defection-only-game-theory hole as society's capabilities melt in the face of increased heat and chaos at various scales of the world.

the gears to ascenscion's Shortform

10

the gears to ascenscion's Shortform

10