It seems interesting that people are just barely competent enough to drive. Maybe it's just that they drive as fast as they can. If we were more competent, we'd drive fast enough that we'd crash if we were drunk. If we were less competent, we'd drive slow enough that we wouldn't crash unless we were drunk.
Here's an interesting contrast: When I first moved from a small town to a big city I was fascinated by the fact that people cannot perform the simple task of walking down the street. Their attention is constantly being drawn to other things, they apparently have no awareness of or concern for other people, etc. They're constantly stopping dead in front of you, even though they're certainly aware they're on a busy street. They talk on their phones, text, play games, they even walk along reading novels. If they meet someone they know, they'll stop and have a conversation without moving out of the way. When somebody approaches a bus stop, they'll simply stop dead and won't move to the side, even if they're blocking the only way through. To be sure, people can navigate around other people, but as soon as they do something else (stop, answer their phone, meet someone they know, etc), the fact that they're on a busy street apparently disappears from their consciousness. There's a complete absence of vigilance (and courtesy).
If people drove cars the same way they walk on a busy street there'd be dozens of accidents per mile. I guess the lesson is that human beings are capable of being careful when they need to be but most of the time they don't need to be.
Most things are easier than they look, but writing software that's free of bugs seems to be an exception: people are terrible at it. So I don't share your hope.
Car crash is the first death cause among some demographics (15-25 years old). It's almost half of total accidental death. It's almost 2% of total death. And many more are badly wounded, often crippled for life.
So I would hardly say that humans can drive cars. At least, they can't do it safely.
I got about the opposite change; when I was a 12 years old, driving looked normal to me. Something everyone does. And then I tried to do it, and realized how a split second of inattention can wreak your life or someone's else. And I looked at statistics, of cars crash being a very significant cause of death or crippling injuries. Of people having more chance to die in a car accident between home and airport than flying in the plane. And I realized humans can't drive safely - it's just something we pretend because car crash being a daily occurrence, they don't make it to the news.
Humans can walk. Pretty crazy huh? Constantly balancing tens of kilograms on top of constantly shifting upside-down pendulums, one (or okay, a few) misstep(s) from death (depending on where you are walking). We can even learn to do so on stilts, or ride bikes, or pilot aircraft, or ride a surfboard.
We are very complex adaptive learning systems that can internalize and automate a huge range of activities that would result in disaster if the feedbacks fall out of range. it's a generalizeable ability.
I've always felt similarly, and found myself thinking about how plastic we are with our own body sense - we seem to be very capable of remapping our motor functions into completely new devices, cars, video game characters, etc, and gaining a sense of body with them. This seem to be supported by how tied driving is to which part of your body performs the control - for me, going from a hand clutch on a motorcycle to a foot clutch completely failed to translate the skill. I have no idea if this is neurologically correct.
On the Cruelty of Really Teaching Computer Science by prof. dr. Edsger W. Dijkstra:
http://www.cs.utexas.edu/~EWD/transcriptions/EWD10xx/EWD1036.html
I think the issue is that driving is a process of tiny course corrections, where if you're slightly off course you don't die. But, programs are fragile. One bit wrong and you die.
I just learned to ride a bicycle two months ago, and some part of my brain still refuses to believe it's physically possible at all.
Here is my understanding of the issue.
Traffic works well because driving is one of the activities that is relatively easy to internalize and perform as a matter of habit, like walking or riding a bicycle. System 1 thinking is usually fast, predictable and reliable. You don't need to "focus intently". It is true that you cannot take your eyes off the road (or, more accurately, shift your passive attention (see http://www.cdl.org/resource-library/articles/attention2.php) away) for longer than a brief instant, otherwise the subconscious feedback loo...
Maybe creating a safe AGI isn't as impossible as it looks to me. Maybe it isn't beyond human capabilities. Maybe.
Or, maybe creating any AGI (which isn't an uploaded human mind) is beyond human capabilities.
Yes, humans can drive, and that fact is literally remarkable. You have remarked upon it, I am remarking now upon it.
Can we drive well? Compared to what would be the interesting question. By all reports, self-driving cars will be much safer than human driven cars. By contrast, self-walking machines do not generally outperform humans or animals, machines for image recognition or voice recognition do not outperform humans. Perhaps it is going to be harder for us to beat, or even equal, with machines what we have spent millions of years evolving, while ...
Somewhat off topic, but I feel the need to point out that traffic doesn't really work all that well when you consider the space of possible transportation systems we could realistically implement.
First Google result says that "by car insurance industry estimates, you will file a claim for a collision about once every 17.9 years".
Poisoning, Accidental Falls, and Car Accidents each make up about a quarter of all accidental deaths. Breaking down by age, driving is also the leading cause of death for ages 18-24 (and the risk of driving does fall a b...
Yes, I have noticed this as well.
And also -- evolution built us to deal with speeds of, say, 10 km/h; and not 100 km/h.
Also, evolution built us with a kinesiological sense to know where every part of our human body is at any moment; and not to know where a huge hunk of metal is at any moment. Yet we can park centimeters away from another car, and even drive 100 km/h only decimeters away from another car.
Selection bias. We're using driving as an example because it turned out that humans are actually good enough at it. Lots of other things that humans aren't good enough at simply weren't done before automation and computers.
All this really tells us is that we're good at some things that weren't in our ancestral environment. Which we know already (we can do math!).
For things like building AGI, that no human has done, and that for which we don't yet have a coherent theory or roadmap (other than 'copy this hugely complicated black box'), we don't know how easy or difficult they really are. We can get an outside view by comparing with other tasks that we once didn't know how to do and then succeeded on some and failed on others despite a lot of effort. But I think there's a lot of variation between cases and prediction is hard.
We're using driving as an example because it turned out that humans are actually good enough at it.
Plus, our example is specifically driving at the skill level that humans are capable of.
It feels to me like we could drive safely while a little drunk, if we stuck to 20mph and wide roads with shallow turns, and if everyone else did the same. (I haven't driven in years, and never while drunk, so I might be wrong. Even if I'm right, other people do not do the same. Don't do this.) If that was the normal difficulty level to drive at, we might say that humans are pretty good at driving even while drunk. But the level we normally drive it is approximately the best we can do, so when we get drunk, we can no longer do it at that level.
If we were used to a world where cars were mostly driven by computers, would we really say humans were good at it? A human compared to a computer could easily be worse than a drunk human compared to a sober human.
A related observation I have made as a youth was:
How does it come that nobody (me included) seems to make any small missteps bringing certain death or injury?
I mean its just one small step to fall before a car, a train, down a bridge, out of a window, ... Its just one wrong grip and you gulp acid, poison, wrong medicine.
Sure it happens sometimes (when?). But it doesn't seem to happen for me or most persons.
And the answer: There are lots of safety measures and control feedback cycles in the human brain for reducing the chance of exactly this to almost zero.
Obviously selected massively for by evolution. But the exact meachnism is nonetheless somewhat elusive.
But what amazed me as a child was that people can drive cars.
Still amazes me. And I live up in the Pacific Northwest, where people drive in a sane fashion. But people are bat shit crazy on the NJ Turnpike, or LA freeways, on how closely they'll tailgate at 70mph+.
Funny that this has occurred to a lot of us. I wonder if that's another LW peculiarity, or if it's generally widespread.
I'm occasionally still amazed that traffic works as well as it does. I must say I'm hesitant at using this example to claim that people are more capable than you might think. Driving is just something humans happen to be competent at. There are plenty of things roughly as complicated as driving a car that people aren't surprisingly good at.
This also reminded my of something people said at the latest meetup. At least two people told me they had deliberately tried to get more scared of driving, because they had noticed they had less fear in a car than on a plane despite planes being safer.
Driving is just something humans happen to be competent at.
I don't think it is pure chance, since it was designed in iterations around human capabilites.
Good post - this has struck me too. By the way, this is a good example showing that social life and human behaviour in general is much more "law-like" and indeed predictable than many "anti-positivists" in the social sciences would have it. Car drivers' behaviour is remarkably regular and predictable, even though it is, as you say, in no way trivial to drive a car.
Still many mistakes which end up causing accidents are made, and thus I'm sure automatized or semi-automatized cars could decrease the number of accidents hugely.
People can drive cars - but only just barely. You can't do it safely even while only mildly inebriated.
How are our speed limits determined? Is it reasonable to assume that they're set such that it's just within our abilities to drive that fast (plus some margin, of course)?
I've thought about the same thing. Automobiles are relatively unsafe compared to other modes of tranportation, but it is amazing to me how commutes in major metro areas seem so free of major accidents on the majority of days.
However, though I'm pretty ignorant of programming, I'm not sure your analogy works.
Driving is hard. But there are lots of mechanisms in play that make it easy enough. Traffic signals and road markings, for instance. Plus, even if there is a crash, damage is limited in scope based on the nature of the situation.
From what I can gather i...
Maybe creating a safe AGI isn't as impossible as it looks to me. Maybe it isn't beyond human capabilities. Maybe.
Maybe humans are not safe AGI. Maybe both the idea of "safety" and the idea of "general intelligence" are ill-defined.
3. Well, this does not really check out. We do take our eyes off the road, for a second. Sometimes. There are somewhat safe moments to do this and less safe ones. When several hundred meters of highway ahead are empty, the road we will travel in those seconds + some safety-cushion, and it is straight, it is somewhat safe. You are just exxagerating a bit. On a long boring ride or drive, the mind is not really occupied by the task, the eyes can still safely look at the road, ready to react with exactly one single thing to anything unforeseen: appyling the brakes.
You seem to say that the difficult part of driving is staying in the lane. That's by far the easiest part of driving, both for humans and computers.
There's been a lot of fuss lately about Google's gadgets. Computers can drive cars - pretty amazing, eh? I guess. But what amazed me as a child was that people can drive cars. I'd sit in the back seat while an adult controlled a machine taking us at insane speeds through a cluttered, seemingly quite unsafe environment. I distinctly remember thinking that something about this just doesn't add up.
It looked to me like there was just no adequate mechanism to keep the car on the road. At the speeds cars travel, a tiny deviation from the correct course would take us flying off the road in just a couple of seconds. Yet the adults seemed pretty nonchalant about it - the adult in the driver's seat could have relaxed conversations with other people in the car. But I knew that people were pretty clumsy. I was an ungainly kid but I knew even the adults would bump into stuff, drop things and generally fumble from time to time. Why didn't that seem to happen in the car? I felt I was missing something. Maybe there were magnets in the road?
Now that I am a driving adult I could more or less explain this to a 12-year-old me:
1. Yes, the course needs to be controlled very exactly and you need to make constant tiny course corrections or you're off to a serious accident in no time.
2. Fortunately, the steering wheel is a really good instrument for making small course corrections. The design is somewhat clumsiness-resistant.
3. Nevertheless, you really are just one misstep away from death and you need to focus intently. You can't take your eyes off the road for even one second. Under good circumstances, you can have light conversations while driving but a big part of your mind is still tied up by the task.
4. People can drive cars - but only just barely. You can't do it safely even while only mildly inebriated. That's not just an arbitrary law - the hit to your reflexes substantially increases the risks. You can do pretty much all other normal tasks after a couple of drinks, but not this.
So my 12-year-old self was not completely mistaken but still ultimately wrong. There are no magnets in the road. The explanation for why driving works out is mostly that people are just somewhat more capable than I'd thought. In my more sunny moments I hope that I'm making similar errors when thinking about artificial intelligence. Maybe creating a safe AGI isn't as impossible as it looks to me. Maybe it isn't beyond human capabilities. Maybe.
Edit: I intended no real analogy between AGI design and driving or car design - just the general observation that people are sometimes more competent than I expect. I find it interesting that multiple commenters note that they have also been puzzled by the relative safety of traffic. I'm not sure what lesson to draw.