You can't fetch the coffee if you're dead. That's obvious. Yet the statement neatly sums up a vexing problem for researchers and developers of Artificial Intelligence systems (AI). In his book Human Compatible: AI and the Problem of Control, Stuart Russell explains why programming an intelligent machine is tricky - even if its only job is to get the coffee.

You see, an intelligent machine understands that it can't complete its objective if switched off for some reason. So, the goal of serving coffee has an implicit subgoal: avoid being switched off. And the surest way to avoid being switched off is, of course, to disable the off-switch. If a machine is intelligent, it can figure out how.

But at that point, the machine isn't under human control anymore - which is a problem. This scenario isn't unique to coffee robots but applies to any intelligent system with a specific objective. Self-preservation is critical because, well, there's not much you can do once you're dead.

There are other issues, too. An intelligent machine might go about things differently than its human programmers. For example, an AI instructed to cure cancer might induce different kinds of tumours in every living person, allowing it to conduct a range of medical trials all at the same time. That would be the fastest way to find a solution - but it would also be wrong.

Think long enough about any single objective, and you'll notice problems. It's like the genie in a bottle granting three wishes. More often than not, the third wish is to undo the first two because unintended side effects happened when the wisher forgot to mention something they cared about. In Greek mythology, King Midas wished that whatever he touched turned into gold. That backfired when he discovered food and water weren't excluded. He died of starvation.

SINGLE PURPOSE

In all these examples, we optimise for a single objective and measure success only as achieving that goal. Current AI systems work like that, too. It hasn't caused significant problems because, so far, we've only built 'narrow AI'. That is, AI that does precisely one job. For example, a chess robot that follows the rules of chess to win a game it is told to play.

On the other hand, when instructed to 'win a game of chess, following the rules of chess, ' an intelligent machine might already cause trouble trying to interfere with others playing the game. And fo...

Posts

Wikitag Contributions

Comments