Donald Hobson

MMath Cambridge. Currently studying postgrad at Edinburgh.

Sequences

Neural Networks, More than you wanted to Show
Logical Counterfactuals and Proposition graphs
Assorted Maths

Wiki Contributions

Comments

There is a view of "pesimistic solutionism", and "optimistic solutionism".

In optimistic solutionism, you think that yes it's possible to make mistakes, but only if your really trying to screw up. Basically any attempt at safety is largely going to work. Safety is easy. The consequences of a few mistakes are mild so we can find out by trial and error. 

In pesimistic solutionism you think doing it safely is theoretically possible, but really hard. The whole field is littered with subtle booby traps. The first 10 things you think of to try to make things safe don't work for complicated reasons. There is no simple safe way to test whether something is safe. The consequences of one mistake anywhere can doom all humanity. 

With optimistic solutionism, it's a case of "go right ahead, oh and keep an eye on safety". 

What about pesimistic solutionism? What should you do where safe AI is in theory possible to make, but really really hard. Perhaps try to halt AI progress until we have figured out how to do it safely? Take things slow. Organize one institution that will go as slowly and carefully as possible, while banning anyone faster and more reckless. 

I think this is the world we are in. Safe AI isn't impossible, but it is really hard.

Solutionism isn't opposed to optimism and pessimism. It's a separate axis. 

An accurate model of the future should include solutions we haven't invented yet, and also problems we haven't discovered yet. Both of these can be predictable, or can be hard to predict.

The correct large scale societal reaction is to try to solve the problem. But sometimes you personally have nowhere near the skills/resources/comparative advantage in solving it, so you leave it to someone else. 

In the example given of solutionism, the difficulty of fixing atmospheric nitrogen depended on the details of chemistry. If it had been easier, then it would have been a case of "we have basically almost got this working, keep a few people finishing off the details and it will be fine."

If nitrogen fixation had turned out to be harder, then a period of tightening belts and sharply rationing nitrogen, as well as a massive well funded program to find that solution ASAP would be required. 

Reality is not required to send you problems within your capability to solve. Reality also sends you problems that are possible to solve, but not without a lot of difficulties and tradeoffs. 

Then you have screwed up your alignment. Any time your woried that the AI will make the wrong decision, rather than that the AI will make the right decision, but you want to have made that decision, then your worried about an alignment failure. Now maybe it's easier to make a hands off AI than a perfectly aligned AI. An AI that asks us it's moral dilemmas because we can't figure out how to make an AI choose correctly on it's own. 

But this is a "best we could do given imperfect alignment". It isn't the highest form of AI to aim for. 

Firstly happiness is a combination of many things. A fair bit of that is pleasant sensations. Not in pain. Nice food. Comfortable. Pretty surroundings ...

It's an average, not an and gate, so you can still be happy with some things not great, if the rest are good. 

Secondly, a lot of people don't want responsibility. Ie prospective parents who aren't sure if they want the responsibility. People often want to escape their responsibilities. 

Thirdly, I am not responsible for global steel production. Do I have any reason to want some other humans to have that responsibility over no one having it? No, not really.  Who holds responsibility in todays world. A few rich and powerful people. But mostly responsibility is held by paperwork monstrosities, where each person just follows arcane bureaucratic rules.  

In this hypothetical, you have a superhuman AI actively trying to make the world nice. Therefore, whatever happens will be very nice. 

Maybe the AI leaves some things to humans, you know, like art or something. 

Isn't the point of improving the world to make the world better. If you are working on a cancer cure, and hear the news that someone else succeeded, do you go "How dare they. I wanted to cure cancer. Me Me Me." or do you go "yay"? 

Why should it be any different with AI's. 

More to the point. People still play chess, even when the AI is totally better than them. People will find something to do. People do that. People don't go around moping at the "hollow purposelessness of their existance". 

I estimate such letters chance of doing anything much are between 0 and 0 percent. (Rounding to nearest percent) 

There is a small but non-negligable chance that reasoning based on tradeing with aliens turns out to be correct. 

There is basically no chance that this reasoning is correct, and the AI smart enough to wipe out humanity would fail to work this out on it's own, yet recognize its factualness based on some letter like this. 

I think that benevolent aliens are more likely as it is Schelling point is goals' space. 

I think this strongly indicates you don't understand what a Schelling point is. If you mean that benevolent aliens are more likely because evolution often produces benevolence and some non-trivial fraction of aliens succeed at alignment. Sure. Of course, other less nice motivations are likely similarly common. 

I also hope that preserving initial creators will be Nash equilibrium between many paperclippers, as it would show each other that they are not defecting and worth partnership.

This isn't how this works. This isn't how any of this works. The behaviours "be nice to humans, even when the humans give you nothing of value" and "cooperate with super-intelligent trading partners" may feel close on your map, but the inference that an AI that does one must do the other isn't true in the space of general AI's, so it isn't a good signal. 

Also, the AI's want the least costly signal possible. If there is some fancy crypto thing where they can formally prove they cooperate for the cost of fractions of a joule in compute, why would they bother "preserving their initial creators".

Writing an argument for why an ASI should be friendly in the hopes you convince the AI is just not the sort of strategy that does anything much, other than confuse and distract some humans.

Ok. Trade with aliens. The "preserve it, aliens might want it" could be applied to anything. It depends entirely on the AI's beliefs about what aliens might want, and of course what the AI can do really cheaply.

Do we have any strong reason to be optimistic? Are benevolent aliens more likely than sadistic aliens? What fraction of aliens (or alien created paperclip maximizers) will care about us?

Also, such trade only works if the AI can't cheat. The human created AI runs into a civilization of octopods who managed to make friendly AI. The octopods are generally pretty close to humans, sure tentacles and a sense of fnord instead of humor, but basically the nicest aliens we could hope to meet. The AI ignores it's actual past, and makes up an entirely imaginary race of tentacled beings who are more appealing to the octopods. It then creates a bunch of these newly imagined beings using advanced bionanotech as part of its efforts to trade. 

To a superintelligence with sufficiently advanced technology. 

The cost of the minimal amount of resources needed to sustain human life is greater than the maximum value we could possibly provide to them. Whatever goals the AI has, if it is choosing over all possible arrangements of atoms looking to maximize productivity, it won't pick humans. There are machines that would be faster, more reliable and more efficient. 

Leaving other optimizers alive has a risk. They might decide to attack you. The fastest and most flawless victory is one where all the humans drop dead instantly. The AI doesn't particularly want to get into a prolonged war that is costly to both sides. Ideally it wants all humans to drop dead at no cost to itself. 

But suppose that wasn't an option. The nanobots don't work or something. The AI certainly doesn't want to deal with it's human problem forever. So it goes a slower way. Gently chipping away at whatever it is that makes fighting humans costly. Maybe nukes could destroy half the AI's infrastructure, so it builds missile defense systems, encourages disarmament or drugs some technician into wiring them up to never explode. 

And then, when we have been subtly declawed and least expect it, the AI strikes. 

If the collision is able to negotiate bets internally, then for any instance of the coalition getting Dutch booked, they can just agree to run that bet without the bookie, and split the bookies cut. 

Suppose human brains turned out to be hypercomputers. (They really really aren't) But imagine the world where they were.

We describe what is going on with equations. It's some complicated thing involving counterpolarized pseudoquarks or something. We can recreate this phenomena in the lab. Recent experiments with sheets of graphine in liquid xenon produce better hypercomputers. 

A halting oracle doesn't seem to contain magic "free will" stuff. It's behaviour is formally specified and mathematically determined. A quantum coin toss can already make random data, something no turing machine can do. (Yet the universe is computable, because a reality goes both ways) Is a quantum coin conscious? No, it's just a source of randomness. 

As soon as the magic mysterious essence of consciousness that Penrose hopes for is actually discovered, it will stop being magical and mysterious, and will be demoted to the dull catalogue of common things. 

I have a phone, but it's nearly 8 years old and I can't get new apps installed on it. 

I do play games. I have a sudoku and a few other puzzles to use while waiting for a bus. (Just a sudoku, none of this high score, leaderboard, micropayment bonus point nonsense (sgt-puzzles))

I have repeatedly had my mum not able to call me for several days because I left my phone somewhere and forgot to charge it. I have an excessive fear of emails and avoid looking at them. 

I think I have a different socio-tech problem from most other people. While others socialize on social media, I don't really. I think part of my problem is the internet contains an endless source of informative interesting info that isn't the information I need or am looking for. The random wikipedia page or arxiv paper or blog post or video on a topic that is somewhat informative and correct, but did I really need to read all about 18'th century steel making techniques? That video on complex numbers was good, but I already knew all that. Why am I watching this again?

Load More