One particular chart in this post would be much easier to look at if it didn't have an uncanny abomination wrapped in human skin stapled to it.
In futures where we survive but our plans all get derailed, why do you expect a utopia? Or, equivalently: In futures where we survive and get a utopia, why do you expect our plans to all get derailed?
What this means in practice is that the "entry-level" positions are practically impossible for "entry-level" people to enter.
I recently applied to an ML/AI upskilling program. My rejection letter pointed out my rather sparse GitHub (fair!) and suggested that I fill it with AI safety-relevant projects... politely skimming over the fact that I can't complete those projects until I learn ML/AI engineering. The cynic in me raises an eyebrow and asks what, other than credentials, the program actually has to offer, if they only accept people who already understand the material. (The gentler side of me says that maybe it's okay if they only offer credentials.)
For what it's worth, I'm glad to see that people more qualified than me are applying to these programs and jobs.
but even then there's handwaving around why we'll suddenly start producing stuff that nobody wants.
"Stuff that nobody wants"? Like what? If you're referring to AI itself... Well, a lot of people want AI to solve medicine. All of it. Quickly. Usually, this involves a cure for aging. Maybe that could be done by an AI that poses no threat... but there are also people who want a superintelligence to take over the world and micromanage it into a utopia, or who are at least okay with that outcome. So "stuff that nobody wants" doesn't refer to takeover-capable AI.
If you're referring to goods and services that AIs could provide for us... Is there an upper limit to the amount of stuff people would want, if it were cheap? If there is one, it's probably very high.
Yep, that all sounds right. In fact, a directed graph can be called transitive if... well, take a guess. And k-uniform hypergraphs (edit: not k-regular, that's different) correspond to k-ary relations.
Here's another thought for you: Adjacency matrices. There's a one-to-one correspondence between matrices and edge-weighted directed graphs. So large chunks of graph theory could, in principle, be described using matrices alone. We only choose not to do that out of pragmatism.
(I've also heard of something even more general called matroid theory. Sadly, I never took the time to learn about it.)
such as the Continuum Hypothesis, which is conjectured to be independent of ZFC.
It's in fact known to be independent of ZFC. Sources: Devlin, The Joy of Sets; Folland, Real Analysis; Wikipedia.
Not the way I'd use those words, nope. The first is a low bar; the second is extremely high, and includes a specific emotional reaction in it. I haven't seen any plausible vision of 2040 that I'd enthusiastically endorse, whether it's business-as-usual or dismantling stars, but it's not hard to come up with futures that are preferable to the end of love in the universe.
If you can’t enthusiastically endorse that outcome, were it to happen, then you should be yelling at us to stop.
I don't think it's that simple. I'm not enthusiastic about transhumanism, so I can't enthusiastically endorse that outcome, but I can't bring myself to say, "Don't build AI because it'll make transhumanism possible sooner." If anything, I expect that having a friendly-to-everyone ASI would make it a lot easier to transition into a world where some people are Jupiter-brained.
I am quite willing to say, "Don't build AI until you can make sure it's friendly-to-everyone," of course.
So, LLM-powered systems can do research now. That includes basic safety research. And it looks like they can have good research taste after a bit of fine-tuning. And they haven't tried to take over the world yet, as far as I know. At some point in the past, I expected that AIs that could do any scientific work at all would have to be smart enough to be takeover-capable. Evidently not!
Making a machine god benevolent is probably harder than coming up with a new jailbreaking technique and writing a half-lucid report about it. It could still be that there's no such thing as an alignment MVP. Let's assume that there is such... (read 280 more words →)
Why do you say this?