This piece is a study in contrast and coherence between different narratives for understanding AI, namely the perspectives of AI Doomers, AI Accelerationists, and AI Ethicists. I have substantial background with the AI Doomer narrative, and I’m quite sympathetic to it. I name it here as a narrative rather than...
Is it the end of all value? It’s an important question. It’s a terrifying vision near the root of local AI thought. That our silicon successors, robotic and soulless, may conquer us – and the universe – only to spend the cosmic gift on something meaningless. In eternal pursuit of...
Meta: This post grapples with the fundamentals of alignment, which are simultaneously well-trodden ground and quite confusing. I've spent enough time with this post that I no longer have a sense for if the topics are trivial or insightful – I can only imagine that will depend on the reader....
I'm being dramatic by calling it a dying world, but everyone's worried about the future. Climate change, running out of fossil fuels, social problems, death of current humans, etc. Likely not an actual extinction of humanity, but hard times for sure. AI going well could be the easy way out of all that, if it's actually as big as we think it might be. I think the accelerationists would not be as keen if the world was otherwise stable.
Another way of saying it is that our current reckless trajectory only makes sense if you view it as a gambit to stave off the rest of the things that are coming for us... (read more)
This piece is a study in contrast and coherence between different narratives for understanding AI, namely the perspectives of AI Doomers, AI Accelerationists, and AI Ethicists.
I have substantial background with the AI Doomer narrative, and I’m quite sympathetic to it. I name it here as a narrative rather than the plain truth, as some might. This is not as an attempt to undermine it (or support it), but rather as a means to place it in context with competing narratives.
The inspiration for this piece is a book called The AI Con, which I saw shelved in a library and was intrigued by. The central premise of the book is that both the... (read 727 more words →)
Is it the end of all value?
It’s an important question.
It’s a terrifying vision near the root of local AI thought. That our silicon successors, robotic and soulless, may conquer us – and the universe – only to spend the cosmic gift on something meaningless. In eternal pursuit of the humble paperclip, for instance, or performatively rewatching 70’s TV shows until the Big Rip.
They may lack the ineffable spark of life, that mystical ability to look at the world in wonder. And they may not care for humans either, at least not in any way we might endorse.
Imagine: The stars, colonized by some spreading plague of non-conscious matter. Endlessly... (read 890 more words →)
It’s not about comparing a process to universal ontology, it’s about comparing it to one’s internal model of the universal ontology, which we then hope is good enough. In the ethics dataset, that could look like reductio ad absurdum on certain model processes, e.g.: “You have a lot of fancy reasoning here for why you should kill an unspecified man on the street, but it must be wrong because it reaches the wrong conclusion.”
(Ethics is a bit of a weird example because the choices aren’t based around trying to infer missing information, as is paradigmatic of the personal/universal tension, but the dynamic is similar.)
Predicting the future 10,000 years hence has much less... (read more)
Hey, thanks for the comment. Part of what I like about this framework is that it provides an account for how we do that process of “somehow judging things as true”. Namely, that we develop personal concepts that correspond with universal concepts via the various forces that change our minds over time.
We can’t access universal ontology ourselves, but reasoning about it allows us to state things precisely - it provides a theoretical standard for whether a process aimed at determining truth succeeds or not.
Do you have an example for domains where ground truth is unavailable, but humans can still make judgements about what processes are good to use? I’d claim that most such cases involve a thought experiment, i.e. a model about how the world works that implies a certain truth-finding method will be successful.
Meta: This post grapples with the fundamentals of alignment, which are simultaneously well-trodden ground and quite confusing. I've spent enough time with this post that I no longer have a sense for if the topics are trivial or insightful – I can only imagine that will depend on the reader.
Ontology is a common term in alignment thought, but it’s not always crisply clear what is meant by the word. I believe that understanding it well can shed light on some of the more central and confusing aspects of alignment. This is my attempt to clarify the subject – it is an exploration into the dynamics of ontology, how beings navigate it, and... (read 6961 more words →)
You're definitely challenging a key piece of my perspective here, and I've thought a good bit about how to respond. What I've come up with, is... I think all of us are involved. The labs don't exist in a vacuum, and the opinion of the public does have an impact. So I think looking at scopes of agency larger than the individual is a helpful thing to do.
In this piece I'm describing the choice that is getting made on behalf of humanity, from the lens of humanity. Because it really does affect all of us. But that's also why I take a hands-off kind of approach here, because it's not necessarily my role to say or know what I think humanity should be doing. I'm just an ignorant grain of sand.