Intuitions about solving hard problems

One thing I'm interested in but don't know where to start looking for it, is seeing people who are working instead on the reverse direction - mathematical approaches which show aligned AI is *not* possible or likely. By this I mean formal work that suggests something like "almost all AGIs are unsafe", in the same way that the chances of picking a rational number at random from is zero because almost all real numbers are irrational.

I don't say this to be a downer! I mean it in the sense of a mathematician who spent 7 years attempting to prove X exists, ... (read more)

94mo

I have been working on an argument from that angle.
I've been developing it independently from my own background in autonomous
safety-critical hardware/software systems, but I discovered recently that it's
very similar to Drexler's CAIS [
https://www.lesswrong.com/posts/x3fNwSe5aWZb5yXEG/reframing-superintelligence-comprehensive-ai-services-as]
from 2019, except with more focus on low-level evidence or rationale for why
certain claims are justified.
It isn't so much a pure mathematical approach as it is a systems engineering or
systems safety perspective on all of the problems[1] [#fnkk3f1cry34r][2]
[#fnysnrym6gkfe]that would remain even if someone showed up tomorrow and dropped
a formally verified algorithm describing an "aligned AGI" onto my desk, and what
ramification that has for the development of AGI at all. The only complicated
math in it so far is about computational complexity classes and relatively
simple if, then logic for analyzing risk vectors.
I guess if I had to pick the "key" insight that I claim I can contribute, and
share it now, it would be this:
If you've read CAIS, you might recognize the above argument, where it was worded
as:
When this idea was proposed in 2019, it seems to me like it was criticized
because people didn't see how task-focused AI/ML systems could keep improving
and eventually surpass human performance without somehow developing "general
intelligence" along the way, plus a general skepticism that there would be
rational reasons to not "just" staple every single hypothetical task together
inside a system and call it AGI. I really think it's worth looking at this again
in light of the last 3 years and asking if that criticism was justified.
1. ^ [#fnrefkk3f1cry34r]In systems safety, we're concerned with the safety of a
larger system than the usual "product-focused" mindset. It is not enough for
there to be a proof that a hypothetical product as-designed is safe. We also
need to look at the likelihood of:

Why pessimism sounds smart

No known solutions can solve our hardest problems—that’s why they’re the hardest ones.

I like the energy, but I have to register a note of dissent here.

Quite a few of our hardest problems do have *known* solutions - it's just that those known solutions are, *or appear*, too hard to implement.

- Brute force algorithms exist for almost everything we care about, up to and including AGI.
- Overweight individuals know that if they eat less, they will eventually lose weight; it's just often frustratingly beyond them, for one reason or another. Same with alcoholics an

94mo

It's not really a solution if it can't be implemented: if it doesn't work, or is
unaffordable, or otherwise isn't practical.

I think your trust-o-meter is looking for people who have an unusually low level of self-deception. The energy is

Great if you share my axioms or moral judgments, but for Pete's sake, at least be consistent with your own.What suggests this to me is the Breaking Bad example, because Walter White really does move on a slow gradient from more to less self-decieved throughout the show in my read of his character - it just so happens that the less self-decieved he is, the more at home he becomes with perpetuating monstrous acts as a result of the previous histo... (read more)