I realize that no analogy is perfect, but I don't think your sleeper cell hypothetical is analogous to AI. It would be a more accurate analogy if someone were to point out that, gee, a sleeper cell would be quite effective, and it's just a matter of time before the enemy realizes this and establishes one. There is a vast amount of Knightian uncertainty that exists in the case of AI, and does not exist in your hypothetical.
Well, it depends on what you mean, but I do think that almost any AGI we create will be unfriendly by default, so to the extent that we as a society are trying to create AGI, I don't think it's exaggerating to say that the sleeper cell "already exists". I'm willing to own up to the analogy to that extent.
As for Knightian uncertainty: either the AI will be an existential threat, or it won't. I already think that it will be (or could be), so I think I'm already being pretty conservative from a Knightian point of view, given the stakes at hand. Worst case is that we waste some research money on something that turns out to be not that important.
(Of course, I'm against wasting research money, so I pay attention to arguments for why AI won't be a threat. I just haven't been convinced yet)
Philosophers seem to treat their field as being almost inconsequential
Like the authors of LW comments, but to an even greater degree, philosophers usually try to signal intelligence rather than trying to find the right answer.
What would constitute a right answer is a philosophical question. That kind of circularity is what makes the subject difficult.
It is not my on radar because either I I don't understand intelligence properly or disagree about something fundamental about it. Basically my brain threw in the towel when Eliezer wrote that nonsentient superintelligence is possible. The what? How can you have intelligence without sentience? Intelligence, amongst humans, means to talk intelligently, to hold conversations about deep topics. It is social. (Optimization as such is not necessarily intelligence, Solomonoff induction shown you could just optimize things by trial and error.) How could anyone non-sentient talk at all? What would be their talk like? They were unable to talk about themselves, how they feel? Or they could talk about that, but because being nonsentient, that would be a manufactured lie?
But fine. Let's just taboo the word intelligence and just focus on optimization. Can you have a nonsentient optimizer? Sure, we already do, we use genetic algorithms to evolve solutions to the Travelling Salesman Problem - this is actually being used to route trucks to shops.
But to make the jump beyond that and optimize for abstract values like human health in general you need to have something that is through and through social, something that talks, something that understands a value in a conversation. Something that is indistinguishable from something that is sentient.
And this is why it matters when I say intelligence is not mere optimization but more of a social, communication ability like conversing about deep topics. Imagine the whole of humankind as super-brain with individual brains being neurons in it, and they communicate through words. Speech and writing. Science for example is the result of this collective brain, with its memories (libraries), replacing old neurons with new ones (education) and so on. In this model, intelligence is the ability of a neuron-brain in the collective brain to deal with complicated communications. When you measure someone's intelligence by their ability to understand a difficult idea of yours, you measure the ability of two neurons to communicate in the collective brain.
Cognition is not an individual thing. It is a highly collective. Nobody ever invented anything, it is the collective brain of humankind that invented anything, with one neuron-brain whom we credit with the invention simply being the last push in a long chain of pushes. And in this, intelligence is communication.
My point is, there is a huge gap in the genetic algorithm optimizing graphs type of optimization and the kind of optimization that can affect values described in words. For an AI to do this, it has to have sentience and that kind of communicative intelligence humans have. If it does, it becomes one neuron in the collective brain.
Imagine an AI trying to make MealSquares. It has to understand highly unspecific human concepts like optimal nutrition i.e. optimal body function, why we like states called health over states called illness etc. and so on. To understand all this it needs to have a form of communicative intelligence, a high level of empathy with human minds, to figure out we like to have protein because we like to have muscle mass because a bunch of reasons such as having fun at playing sports and why exactly is that fun and and... very, very few of its ideas will be his own. By the time it understands any value, it is programmed through and through with a huge set of human values, because there is no individual intelligence, just the ability to participate in the cognitive web of humankind.
TL;DR skeptical because not believing in cognitive individualism like the idea that heroic individuals can invent things.
It seems to me that pop philosophy is being compared to rigorous academic science. Philosophers make great effort to undertand each others' frameworks. Controversy and disagreement abound, but exercising the mind in predicting consequences using mental models is fundamental to both scientific progress AND everyday life. You and I may disagree on our metaphysical views, but that doesn't prevent us from exploring the consequences each viewpoint predicts. Eventually, we may be able to test these beliefs. Predicting these consequences in advance helps us use resources effectively (as opposed to testing EVERY possibility scientifically). (Human) philosophy is an important precursor to science.
I'm also glad to see in other comments that the AI case has greater uncertainty than the sleeper cell case.
Having made one counterpoint and mentioned another, let me add that this was a good read and a nice post.
You and I may disagree on our metaphysical views, but that doesn't prevent us from exploring the consequences each viewpoint predicts.
Right. Which leads to a complementary suggestion to the OP author's
the immediate next step should be to zoom in on your intuitions – to figure out the source and content of the intuition as much as possible.
In addition, we should zoom out, to find consequences of the intuitions (when combined with known or discoverable facts). That's where the interesting stuff happens.
Philosophers make great effort to undertand each others' frameworks.
This amused me, because I somewhat doubt the term "philosophy" would exist without Alexander the Great, and it appears to me that philosophers do not make great effort to understand relevant work they've classified as 'not philosophy'.
I recall the celebrated philosophy journal Noûs recommending an article, possibly this one, which talked a great deal about counterfactuals without once mentioning Judea Pearl, much less recognizing that he seemed to have solved the problems under direct discussion. (Logical uncertainty of course may still be open. I will be shocked if the solution involves talking about logically impossible "possible worlds" rather than algorithms.)
Now on second search, the situation doesn't seem quite as bad. Someone else mentioned Pearl in the pages of Noûs - before the previous article, yet oddly uncited therein. And I found a more recent work that at least admits probabilities (and Gaifman) exist. But I can see the references, and the list still doesn't include Pearl or even that 2005 article. Note that the abstract contains an explicit claim to address "other existing solutions to the problem."
Not to denigrate the towering contributions of Pearl, but potential outcomes (what Pearl calls counterfactuals) were invented by Jerzy Neyman in 1920s, and much of the stuff was worked out by Don Rubin and company in the 1970s (to be fair, they didn't talk to philosophers very much).
Of course Pearl contributed quite a bit himself, and is probably the best popularizer slash controversy magnet in the field, and has incredible scholarship and reach.
Still, if you are going to shit on academic philosophy for failing scholarship, at least do the scholarship homework yourself.
Also: did you read that paper? Are you sure Pearl et al. solved what they are talking about, or is even relevant? "Counterfactuals" is a big area.
"Counterfactuals" is a big area.
Yes, I just mentioned that. The article I remember reading, which was definitely about Sobel and reverse Sobel sequences, did not contain anything that looked remotely problematic in a Pearlian framework.
As to the rest, you're talking about giving credit for the answer and I'm talking about getting the right answer.
did not contain anything that looked remotely problematic in a Pearlian framework.
Maybe. Truth value being dependent on orderings for Sobel sequences is (a) weird (b) true according to our intuitions, apparently. So we need to figure out what's going on (sounds like a problem for logic, really). Maybe you can have a Pearlian account of it -- I am not sure myself, and I have not read such an account.
My point is this.
(a) If your complaint is about scholarship because you feel Pearl is relevant, which I base on you saying things like:
which talked a great deal about counterfactuals without once mentioning Judea Pearl
then that's fine, but please get your own scholarship right.
(b) If your complaint is of the form: "why are you people wasting time, Pearl's structural equations just immediately solve this," then can you explain why, or point to a paper? How do you know this? Structural equations are not a panacea for every problem in causation and counterfactual reasoning.
...So, I found it ludicrous to call the reverse Sobel sequence false. I still basically feel that way after reading the article again, though I should say that the author gets around to a better account of the situation on page 12 of 29 (and on close inspection, Moss doesn't really follow this by heading back to the same zombie-infested cul-de-sac. Not quite.) If the following statements come from different speakers, the second seems socially rude:
(3a) If Sophie had gone to the parade and been stuck behind a tall person, she would not have seen Pedro.
(3b) #But if Sophie had gone to the parade, she would have seen Pedro.
Since the author sometimes seems to treat these as made by one speaker, and the article is nominally about counterfactuals, I assume that both statements come from the same world-model or causal graph. I also assume that each comes from a human being, and not a logician or Omega, and thus we should take each as asserting a high probability. (The article only briefly considers someone saying, "Suppose that Sophie might not see Pedro, but that she will see Pedro," which I don't think we could make sense of without reading an explicitly conditional probability into the first half of the premise.) So with two different speakers, I see two credible ways to read (3b):
A. 'Shut up.'
B. 'You are altering the wrong node.' (Because, eg, 'Had Sophie gone to the parade, she would have looked for a good place to see it.')
If it's just B, then politeness would require doing more to rule out A and maybe explain why the speaker doesn't care about the first counterfactual. Note that reversing their order (that is to say, un-reversing it) would give us what sounds like an invitation to keep talking. As Moss says, the "reply may be a non sequitur, perhaps even a little annoying." But it doesn't read as contempt or dismissal.
In the unlikely event that (3) has only one speaker and she isn't disagreeing with anyone, then we have:
C. 'The answer you get depends almost entirely on the question you ask. The whole point of counterfactuals is to screen off the causes that would ordinarily determine the (probability of the) node you change. Suppose we compute different answers when we ask, "What would the founding fathers say if they were alive today?" and "What would the founding fathers say if they had lived in our time?" Then the person who chooses the question probably knows the answer they want. Peripherally, I assert that the chance of Sophie getting stuck behind tall people would be too small for me to bother qualifying my statement.'
The main point of C seems obviously true as soon as I think about doing surgery on a causal graph. We can assume both counterfactuals are also true. (3) still seems "infelicitous" because as a rule of thumb it seems like a terrible way to express C, but that depends on the audience. If we rule out C as a reading, then the speaker's statements could still be true, so I would probably call them "epistemically responsible", but her goals would make no sense. She would sound like a more philosophical Gollum.
Now if you're saying that C does not seem immediately clear to you when you think about doing counterfactual surgery, I will withdraw my original criticism. I would have many new criticisms, like the author assuming intuitions I may not share and analyzing common language while barely mentioning society or motive. There's also what reads to me as mock-formalism, adding nothing to Pearl while distracting from the issues at hand. But the issue would at least warrant discussion, and I'd have to consider whether checking for references was a good test for me apply to the article.
(having given this a bit of thought):
Some people feel that the truth value should reasonably be changed under reversal. I think this might be because humans expect a social convention where info is given in order of relevance. So if (a) comes first, this is info that maybe the parade is crowded-by-default (e.g. preemption is the norm). This makes (b) implausible. If (b) comes first, this is info that we should expect to see people, by default. But this can still be preempted by an exceptional situation.
I don't think this is even about counterfactuals, specifically, but about lack of commutativity of human utterances (most logics and mathematical languages allow you to commute propositions safely). I can think of all sorts of non-counterfactual versions of this where expectation-of-a-default/preemption create non-commutativity.
There exist cases where people gave "Pearlian accounts" for these types of things (see e.g. Halpern and Pearl's paper on actual cause). But I think it requires quite a bit more work to make it go through than just "surgeries and causal graphs." Their paper is quite long and involved.
It feels to me as though you are cherrypicking both evidence and topic. It may very well be that philosophers have a lot of work to do in the important AI field. This does not invalidate the process. Get rid of the term, talk about the process of refining human intelligence through means other than direct observation. The PROCESS, not the results (like the article you cite).
Speaking of that article from Noûs, it was published in 2010. Pearl did lots of work on counterfactuals and uncertainty dating back to 1980, but I would argue that, "The algorithmization of counterfactuals" contains the direct solution you reference. That paper was published in 2011. Unless, of course, you are referring to "Causes and Explanations - a sturctural model approach," which was published in 2005 in the British Journal for the PHILOSOPHY of Science.
Mind you, the 2015 article may have at least recognized (in a way that seems only mildly unhelpful) that we could ask more than one question.
That first one I mentioned is the article Noûs told me to read first at some time or other, the best face that the journal could put forward (in someone's judgment).
Also, did my links just not load for you? One of them is an article in Noûs in 2005 saying Pearl had the right idea - from what I can see of the article its idea seems incomplete, but anyone who wasn't committed to modal realism should have seen it as important to discuss. Yet not only the 2010 article, but even the one I linked from Jan 2015 that explicitly claimed to discuss alternate approaches, apparently failed to mention Pearl, or the other 2005 author, or anything that looks like an attempted response to one of them. Why do you think that is?
Because while I could well have been wrong about the reason, it looks to me like the authors are in no way trying to find the best solution. And while scientists no doubt have the same incentives to publish original work, they also have incentives to accept the right answer that appear wholly lacking here - at least (and I no longer know if this is charitable or uncharitable) when the right answer comes from AI theory.
I think you have hit upon the crux of the matter in your last paragraph: the authors are in no way trying to find the best solution. I can't speak for the authors you cite, but the questions asked by philosophers are different than, "what is the best answer?" They are more along the lines of, "How do we generate our answers anyways?" and "What might follow?" This may lead to an admittedly harmful lack of urgency in updating beliefs.
Because I enjoy making analogies: Science provides the map of the real world; philosophy is the cartography. An error on a map must be corrected immediately for accuracy's sake; an error in efficient map design theory may take a generation or two to become immediately apparent.
Finally, you use Pearl as the champion of AI theory, but he is equally a champion of philosophy. As misguided as your citations may have been (as philosophers), Pearl's work is equally well-guided in redeeming philosophers. I don't think you have sufficiently addressed the cherrypicking charge: if your cited articles are strong evidence that philosophers don't consider each other's viewpoints, then every article in which philosophers do sufficiently consider each other's viewpoints is weak evidence of the opposite.
Note that what seens to be a lsolution to a less philosophically aware person, might mot seem that way to a more philosophically aware one.
When you compare philosophy to science
Is that a fair comparison? Given that the two do not deal with the same questions, one is not forced to internet the slower progress of philosophy as due to incompetence.
we need vastly improved discussion norms when it comes to philosophy, and we need far greater effort on the part of philosophers when it comes to explaining philosophy, and we need these things right now. Regardless of what you think about AI, the 21st century will clearly be fraught with difficult philosophical problems – from genetic engineering to the ethical treatment of animals to the problem of what to do with global poverty,
Why is it so urgent? Are there things other than AI which are both existential threatening and philosophically relevant? Is the problem of poverty a problem of philosophy, or of political will?
While there are certain brute facts of existence, many important questions can't be answered by an examination of external reality (sense data?) - or at all, really. You get to choose your position on these issues in terms of what appeals to you.
For example, the existence of other minds: the reason to believe in the existence of other minds is that life is more enjoyable if other people are real. Is that a problem?
A sleeper cell is likely to do something dangerous on a rather short time scale, such as weeks, months, or perhaps a year or two. This is imminent in a much stronger sense than AI, which will take at least decades. Scott Aaronson thinks it more likely to take centuries, and this may well be true, given e.g. the present state of neuroscience, which consists mainly in saying things like "part of the brain A is involved in performing function B", but without giving any idea at all exactly how A is involved, and exactly how function B is performed at all.
I wonder if hard AI would prefer to be loved than feared, if it cannot be both.
I am of the opinion that you're probably right. That AI will likely be the end of humanity. I am glad to see others pondering this risk.
However, I would like to mention that there are 2 possible/likely modes of that end coming about.
First is the "terminator" future, and second is the "Wall-e" future. The risk that AI war machines will destroy humanity is a legitimate concern, given "autonomous drones", and other developmental projects. The other side has a LOT more projects and progress. Siri, Emospark, automated factories, advanced realdolls, AI, even basic things like predictive text algorithms lead toward the future where people relate to the digital world, rather than the real world, and, instead of being killed off, simply fail to breed. Fail to learn, fail to advance.
However, here's the more philosophical question, and point.
Organisms exist to bring their children to maturity. Species exist to evolve.
What if AI is simply the next evolution of humanity? If AI is the "children" of humanity?
If humanity fails to get out of this solar system, then everything we are, and everything we ever were is for nothing. It was all, from Gilgamesh to hawking, a zero sum game, nothing gained, all for nothing. But if we can make it out to the stars, then maybe it was all for something, Our glory and failings need not be lost.
So while I agree that it's likely that AI will end humanity, it's my opinion that A) it will probably be by "coddling" us to death, and B) either way, that's okay.
I'm assuming you haven't read this;
http://lesswrong.com/lw/k9/the_logical_fallacy_of_generalization_from/
I had not. And I will avoid that in the future. However, that has very little bearing on my overall post. Please ignore the single sentence that references works of fiction.
I'm not quite sure how to put this, but there are many other posts on the site which you seem unaware of.
True enough. I hadn't read that one either, and, having joined a few days ago, there is very little of the content here that I have read. This seemed like a light standalone topic in which to jump in.
This second article however really does address the weaknesses of my thought process, and clarify the philosophical difficulty that the op is concerned with.
Also all of the framing that are implied by those works? And the dichotomy that you propose?
You shouldn't just read it, think about how it has warped your perspective on AI risks - that's the point.
That first one I mentioned is the article Noûs told me to read first at some time or other, the best face that the journal could put forward (in someone's judgment).
Also, did my links just not load for you? One of them is an article in Noûs in 2005 saying Pearl had the right idea - from what I can see of the article its idea seems incomplete, but anyone who wasn't committed to modal realism should have seen it as important to discuss. Yet not only the 2010 article, but even the one I linked from Jan 2015 that explicitly claimed to discuss alternate approaches, apparently failed to mention Pearl, or the other 2005 author, or anything that looks like an attempted response to one of them. Why do you think that is?
Because while I could well have been wrong about the reason, it looks to me like the authors are in no way trying to find the best solution. And while scientists no doubt have the same incentives to publish original work, they also have incentives to accept the right answer that appear wholly lacking here - at least (and I no longer know if this is charitable or uncharitable) when the right answer comes from AI theory.
I think you have hit upon the crux of the matter in your last paragraph: the authors are in no way trying to find the best solution. I can't speak for the authors you cite, but the questions asked by philosophers are different than, "what is the best answer?" They are more along the lines of, "How do we generate our answers anyways?" and "What might follow?" This may lead to an admittedly harmful lack of urgency in updating beliefs.
Because I enjoy making analogies: Science provides the map of the real world; philosophy is the ...
[Many people have been complaining about the lack of new content on LessWrong lately, so I thought I'd cross-post my latest blog post here in discussion. Feel free to critique the content as much as you like, but please do keep in mind that I wrote this for my personal blog and not with LW in mind specifically, so some parts might not be up to LW standards, whereas others might be obvious to everyone here. In other words...well, be gentle]
---------------------------
You know what’s scarier than having enemy soldiers at your border?
Having sleeper agents within your borders.
Enemy soldiers are malevolent, but they are at least visibly malevolent. You can see what they’re doing; you can fight back against them or set up defenses to stop them. Sleeper agents on the other hand are malevolent and invisible. They are a threat and you don’t know that they’re a threat. So when a sleeper agent decides that it’s time to wake up and smell the gunpowder, not only will you be unable to stop them, but they’ll be in a position to do far more damage than a lone soldier ever could. A single well-placed sleeper agent can take down an entire power grid, or bring a key supply route to a grinding halt, or – in the worst case – kill thousands with an act of terrorism, all without the slightest warning.
Okay, so imagine that your country is in wartime, and that a small group of vigilant citizens has uncovered an enemy sleeper cell in your city. They’ve shown you convincing evidence for the existence of the cell, and demonstrated that the cell is actively planning to commit some large-scale act of violence – perhaps not imminently, but certainly in the near-to-mid-future. Worse, the cell seems to have even more nefarious plots in the offing, possibly involving nuclear or biological weapons.
Now imagine that when you go to investigate further, you find to your surprise and frustration that no one seems to be particularly concerned about any of this. Oh sure, they acknowledge that in theory a sleeper cell could do some damage, and that the whole matter is probably worthy of further study. But by and large they just hear you out and then shrug and go about their day. And when you, alarmed, point out that this is not just a theory – that you have proof that a real sleeper cell is actually operating and making plans right now – they still remain remarkably blase. You show them the evidence, but they either don’t find it convincing, or simply misunderstand it at a very basic level (“A wiretap? But sleeper agents use cellphones, and cellphones are wireless!”). Some people listen but dismiss the idea out of hand, claiming that sleeper cell attacks are “something that only happen in the movies”. Strangest of all, at least to your mind, are the people who acknowledge that the evidence is convincing, but say they still aren’t concerned because the cell isn’t planning to commit any acts of violence imminently, and therefore won’t be a threat for a while. In the end, all of your attempts to raise the alarm are to no avail, and you’re left feeling kind of doubly scared – scared first because you know the sleeper cell is out there, plotting some heinous act, and scared second because you know you won’t be able to convince anyone of that fact before it’s too late to do anything about it.
This is roughly how I feel about AI risk.
You see, I think artificial intelligence is probably the most significant existential threat facing humanity right now. This, to put it mildly, is something of a fringe position in most intellectual circles (although that’s becoming less and less true as time goes on), and I’ll grant that it sounds kind of absurd. But regardless of whether or not you think I’m right to be scared of AI, you can imagine how the fact that AI risk is really hard to explain would make me even more scared about it. Threats like nuclear war or an asteroid impact, while terrifying, at least have the virtue of being simple to understand – it’s not exactly hard to sell people on the notion that a 2km hunk of rock colliding with the planet might be a bad thing. As a result people are aware of these threats and take them (sort of) seriously, and various organizations are (sort of) taking steps to stop them.
AI is different, though. AI is more like the sleeper agents I described above – frighteningly invisible. The idea that AI could be a significant risk is not really on many people’s radar at the moment, and worse, it’s an idea that resists attempts to put it on more people’s radar, because it’s so bloody confusing a topic even at the best of times. Our civilization is effectively blind to this threat, and meanwhile AI research is making progress all the time. We’re on the Titanic steaming through the North Atlantic, unaware that there’s an iceberg out there with our name on it – and the captain is ordering full-speed ahead.
(That’s right, not one but two ominous metaphors. Can you see that I’m serious?)
But I’m getting ahead of myself. I should probably back up a bit and explain where I’m coming from.
Artificial intelligence has been in the news lately. In particular, various big names like Elon Musk, Bill Gates, and Stephen Hawking have all been sounding the alarm in regards to AI, describing it as the greatest threat that our species faces in the 21st century. They (and others) think it could spell the end of humanity – Musk said, “If I had to guess what our biggest existential threat is, it’s probably [AI]”, and Gates said, “I…don’t understand why some people are not concerned [about AI]”.
Of course, others are not so convinced – machine learning expert Andrew Ng said that “I don’t work on not turning AI evil today for the same reason I don’t worry about the problem of overpopulation on the planet Mars”.
In this case I happen to agree with the Musks and Gates of the world – I think AI is a tremendous threat that we need to focus much of our attention on it in the future. In fact I’ve thought this for several years, and I’m kind of glad that the big-name intellectuals are finally catching up.
Why do I think this? Well, that’s a complicated subject. It’s a topic I could probably spend a dozen blog posts on and still not get to the bottom of. And maybe I should spend those dozen-or-so blog posts on it at some point – it could be worth it. But for now I’m kind of left with this big inferential gap that I can’t easily cross. It would take a lot of explaining to explain my position in detail. So instead of talking about AI risk per se in this post, I thought I’d go off in a more meta-direction – as I so often do – and talk about philosophical differences in general. I figured if I couldn’t make the case for AI being a threat, I could at least make the case for making the case for AI being a threat.
(If you’re still confused, and still wondering what the whole deal is with this AI risk thing, you can read a not-too-terrible popular introduction to the subject here, or check out Nick Bostrom’s TED Talk on the topic. Bostrom also has a bestselling book out called Superintelligence. The one sentence summary of the problem would be: how do we get a superintelligent entity to want what we want it to want?)
(Trust me, this is much much harder than it sounds)
So: why then am I so meta-concerned about AI risk? After all, based on the previous couple paragraphs it seems like the topic actually has pretty decent awareness: there are popular internet articles and TED talks and celebrity intellectual endorsements and even bestselling books! And it’s true, there’s no doubt that a ton of progress has been made lately. But we still have a very long way to go. If you had seen the same number of online discussions about AI that I’ve seen, you might share my despair. Such discussions are filled with replies that betray a fundamental misunderstanding of the problem at a very basic level. I constantly see people saying things like “Won’t the AI just figure out what we want?”, or “If the AI gets dangerous why can’t we just unplug it?”, or “The AI can’t have free will like humans, it just follows its programming”, or “lol so you’re scared of Skynet?”, or “Why not just program it to maximize happiness?”.
Having read a lot about AI, these misunderstandings are frustrating to me. This is not that unusual, of course – pretty much any complex topic is going to have people misunderstanding it, and misunderstandings often frustrate me. But there is something unique about the confusions that surround AI, and that’s the extent to which the confusions are philosophical in nature.
Why philosophical? Well, artificial intelligence and philosophy might seem very distinct at first glance, but look closer and you’ll see that they’re connected to one another at a very deep level. Take almost any topic of interest to philosophers – free will, consciousness, epistemology, decision theory, metaethics – and you’ll find an AI researcher looking into the same questions. In fact I would go further and say that those AI researchers are usually doing a better job of approaching the questions. Daniel Dennet said that “AI makes philosophy honest”, and I think there’s a lot of truth to that idea. You can’t write fuzzy, ill-defined concepts into computer code. Thinking in terms of having to program something that actually works takes your head out of the philosophical clouds, and puts you in a mindset of actually answering questions.
All of which is well and good. But the problem with looking at philosophy through the lens of AI is that it’s a two-way street – it means that when you try to introduce someone to the concepts of AI and AI risk, they’re going to be hauling all of their philosophical baggage along with them.
And make no mistake, there’s a lot of baggage. Philosophy is a discipline that’s notorious for many things, but probably first among them is a lack of consensus (I wouldn’t be surprised if there’s not even a consensus among philosophers about how much consensus there is among philosophers). And the result of this lack of consensus has been a kind of grab-bag approach to philosophy among the general public – people see that even the experts are divided, and think that that means they can just choose whatever philosophical position they want.
Want. That’s the key word here. People treat philosophical beliefs not as things that are either true or false, but as choices – things to be selected based on their personal preferences, like picking out a new set of curtains. They say “I prefer to believe in a soul”, or “I don’t like the idea that we’re all just atoms moving around”. And why shouldn’t they say things like that? There’s no one to contradict them, no philosopher out there who can say “actually, we settled this question a while ago and here’s the answer”, because philosophy doesn’t settle things. It’s just not set up to do that. Of course, to be fair people seem to treat a lot of their non-philosophical beliefs as choices as well (which frustrates me to no end) but the problem is particularly pronounced in philosophy. And the result is that people wind up running around with a lot of bad philosophy in their heads.
(Oh, and if that last sentence bothered you, if you’d rather I said something less judgmental like “philosophy I disagree with” or “philosophy I don’t personally happen to hold”, well – the notion that there’s no such thing as bad philosophy is exactly the kind of bad philosophy I’m talking about)
(he said, only 80% seriously)
Anyway, I find this whole situation pretty concerning. Because if you had said to me that in order to convince people of the significance of the AI threat, all we had to do was explain to them some science, I would say: no problem. We can do that. Our society has gotten pretty good at explaining science; so far the Great Didactic Project has been far more successful than it had any right to be. We may not have gotten explaining science down to a science, but we’re at least making progress. I myself have been known to explain scientific concepts to people every now and again, and fancy myself not half-bad at it.
Philosophy, though? Different story. Explaining philosophy is really, really hard. It’s hard enough that when I encounter someone who has philosophical views I consider to be utterly wrong or deeply confused, I usually don’t even bother trying to explain myself – even if it’s someone I otherwise have a great deal of respect for! Instead I just disengage from the conversation. The times I’ve done otherwise, with a few notable exceptions, have only ended in frustration – there’s just too much of a gap to cross in one conversation. And up until now that hasn’t really bothered me. After all, if we’re being honest, most philosophical views that people hold aren’t that important in grand scheme of things. People don’t really use their philosophical views to inform their actions – in fact, probably the main thing that people use philosophy for is to sound impressive at parties.
AI risk, though, has impressed upon me an urgency in regards to philosophy that I’ve never felt before. All of a sudden it’s important that everyone have sensible notions of free will or consciousness; all of a sudden I can’t let people get away with being utterly confused about metaethics.
All of a sudden, in other words, philosophy matters.
I’m not sure what to do about this. I mean, I guess I could just quit complaining, buckle down, and do the hard work of getting better at explaining philosophy. It’s difficult, sure, but it’s not infinitely difficult. I could write blogs posts and talk to people at parties, and see what works and what doesn’t, and maybe gradually start changing a few people’s minds. But this would be a long and difficult process, and in the end I’d probably only be able to affect – what, a few dozen people? A hundred?
And it would be frustrating. Arguments about philosophy are so hard precisely because the questions being debated are foundational. Philosophical beliefs form the bedrock upon which all other beliefs are built; they are the premises from which all arguments start. As such it’s hard enough to even notice that they’re there, let alone begin to question them. And when you do notice them, they often seem too self-evident to be worth stating.
Take math, for example – do you think the number 5 exists, as a number?
Yes? Okay, how about 700? 3 billion? Do you think it’s obvious that numbers just keep existing, even when they get really big?
Well, guess what – some philosophers debate this!
It’s actually surprisingly hard to find an uncontroversial position in philosophy. Pretty much everything is debated. And of course this usually doesn’t matter – you don’t need philosophy to fill out a tax return or drive the kids to school, after all. But when you hold some foundational beliefs that seem self-evident, and you’re in a discussion with someone else who holds different foundational beliefs, which they also think are self-evident, problems start to arise. Philosophical debates usually consist of little more than two people talking past one another, with each wondering how the other could be so stupid as to not understand the sheer obviousness of what they’re saying. And the annoying this is, both participants are correct – in their own framework, their positions probably are obvious. The problem is, we don’t all share the same framework, and in a setting like that frustration is the default, not the exception.
This is not to say that all efforts to discuss philosophy are doomed, of course. People do sometimes have productive philosophical discussions, and the odd person even manages to change their mind, occasionally. But to do this takes a lot of effort. And when I say a lot of effort, I mean a lot of effort. To make progress philosophically you have to be willing to adopt a kind of extreme epistemic humility, where your intuitions count for very little. In fact, far from treating your intuitions as unquestionable givens, as most people do, you need to be treating them as things to be carefully examined and scrutinized with acute skepticism and even wariness. Your reaction to someone having a differing intuition from you should not be “I’m right and they’re wrong”, but rather “Huh, where does my intuition come from? Is it just a featureless feeling or can I break it down further and explain it to other people? Does it accord with my other intuitions? Why does person X have a different intuition, anyway?” And most importantly, you should be asking “Do I endorse or reject this intuition?”. In fact, you could probably say that the whole history of philosophy has been little more than an attempt by people to attain reflective equilibrium among their different intuitions – which of course can’t happen without the willingness to discard certain intuitions along the way when they conflict with others.
I guess what I’m trying to say is: when you’re discussing philosophy with someone and you have a disagreement, your foremost goal should be to try to find out exactly where your intuitions differ. And once you identify that, from there the immediate next step should be to zoom in on your intuitions – to figure out the source and content of the intuition as much as possible. Intuitions aren’t blank structureless feelings, as much as it might seem like they are. With enough introspection intuitions can be explicated and elucidated upon, and described in some detail. They can even be passed on to other people, assuming at least some kind of basic common epistemological framework, which I do think all humans share (yes, even objective-reality-denying postmodernists).
Anyway, this whole concept of zooming in on intuitions seems like an important one to me, and one that hasn’t been emphasized enough in the intellectual circles I travel in. When someone doesn’t agree with some basic foundational belief that you have, you can’t just throw up your hands in despair – you have to persevere and figure out why they don’t agree. And this takes effort, which most people aren’t willing to expend when they already see their debate opponent as someone who’s being willfully stupid anyway. But – needless to say – no one thinks of their positions as being a result of willful stupidity. Pretty much everyone holds beliefs that seem obvious within the framework of their own worldview. So if you want to change someone’s mind with respect to some philosophical question or another, you’re going to have to dig deep and engage with their worldview. And this is a difficult thing to do.
Hence, the philosophical quagmire that we find our society to be in.
It strikes me that improving our ability to explain and discuss philosophy amongst one another should be of paramount importance to most intellectually serious people. This applies to AI risk, of course, but also to many everyday topics that we all discuss: feminism, geopolitics, environmentalism, what have you – pretty much everything we talk about grounds out to philosophy eventually, if you go deep enough or meta enough. And to the extent that we can’t discuss philosophy productively right now, we can’t make progress on many of these important issues.
I think philosophers should – to some extent – be ashamed of the state of their field right now. When you compare philosophy to science it’s clear that science has made great strides in explaining the contents of its findings to the general public, whereas philosophy has not. Philosophers seem to treat their field as being almost inconsequential, as if whatever they conclude at some level won’t matter. But this clearly isn’t true – we need vastly improved discussion norms when it comes to philosophy, and we need far greater effort on the part of philosophers when it comes to explaining philosophy, and we need these things right now. Regardless of what you think about AI, the 21st century will clearly be fraught with difficult philosophical problems – from genetic engineering to the ethical treatment of animals to the problem of what to do with global poverty, it’s obvious that we will soon need philosophical answers, not just philosophical questions. Improvements in technology mean improvements in capability, and that means that things which were once merely thought experiments will be lifted into the realm of real experiments.
I think the problem that humanity faces in the 21st century is an unprecedented one. We’re faced with the task of actually solving philosophy, not just doing philosophy. And if I’m right about AI, then we have exactly one try to get it right. If we don’t, well..
Well, then the fate of humanity may literally hang in the balance.
