I fear this uses the word "responsible" in a few different ways, without fully defining what it means in that context. The categories of use tend to be "self-directed feelings of needing to act/control" and "judging others for their behavior", with subcategories as needed for specifics of expression of the judgement".
For the case of informal moral responsibility, it's mostly "feels the pain of bad outcomes". Ultimately, you, the parent, are responsible for helping your children, by finding the right doctors, AI, cultural lore ("an apple a day..."), and actual behaviors you take.
For the case of legal responsibility, it's more about "can be punished for bad behaviors", which leads doctors to follow standard-of-care guidelines rather than thinking deeply about individual cases, and which puts disclaimers in front of AI chats.
Right now, when humans make consequential decisions, someone is answerable for what happened.
Well, no. some decisions, for some outcomes, there's a process for identification and prosecution. For others, you can make some guesses and yell at them. But the vast majority of decisions, even consequential ones, don't have a legible/agreed/visible ownership nor accepted mechanism for enforcement.
Note also that causality is complicated, and most things have multiple causes. Many are overdetermined, and the causes add up to more than 100%. This makes any attempt at formal assessment fully impossible, even if you could measure counterfactuals (which you can't, so this impossibility itself is overdetermined).
Thank you! I used doctors and lawyers because they are people with clearly delineated professional obligations who can be held accountable. A parent is not responsible in the same way. I guess I need to think about separating out different meanings of the word "responsible." And your point that legal accountability can push doctors toward the standard of care, even when it fails a particular patient, is correct.
I may be missing how AI systems handle this, and would appreciate guidance on what to search for. My question is what it means for a professional to "approve" an AI's work when they can not really evaluate it themselves.
AI systems don't handle this, any more than any other non-human system does. How does your local library handle it?
What it means for a human to approve an AI's work depends on the context and what form the approval takes. A coder committing AI-written code has their name on the commit, and will probably suffer some consequences if it breaks horribly. Exactly the same as any other code. A lawyer who submits an AI-generated brief takes the same liability (and benefit) for the quality/correctness as if they'd written it by hand, or if they used one that a junior associate wrote at 2am.
All of these are common, and AI doesn't change very much about the responsibility side of the question.
note: as AI systems get more autonomous and complicated, and as they get productized for humans who are clueless about their limitations, the distinction between legal responsibility and informal cultural responsibility is going to get murkier.
Worse, it'll be weaponized, probably in favor of corporations against individuals. The "not our fault - AI did it and your user agreement accepted this" defense will be used, and it's dumb and meaningless, but will probably be effective in a lot of cases. When you give your AI assistant access to your logged-in bank website, intending to get budgeting advice, and it instead transfers all your money to a top-rated charity, you likely have no recourse.
I already see it a bit in large software projects - calling someone out for a bad design decision often hits the "AI did it" response, and the second half ("I didn't want to slow down to really understand it myself") is not mentioned until you dig. So far, even in cases I wouldn't have accepted 2 years ago, I end up agreeing - we caught it, it caused no harm, let's keep moving fast.
>All of these are common, and AI doesn't change very much about the responsibility side of the question.
Suppose a doctor can't tell if the AI is wrong, but follows its recommendation, because it has a much better track record. I understand that the doctor may still be liable, but what, specifically, should they have done differently? How much responsibility belongs to the hospital that decided this was an acceptable way to make decisions? Your software example sounds similar. AI at least affects some important aspects of this model.
At some point, the person clicking "approve" is becoming the whipping boy for the failed AI. A bad AI could easily use this against humans. A bad company could use this against the person whose job it is to click.
To respond to your other comment, that's exactly the sort of issue that concerns me. The ability to deploy AI and avoid responsibility. That's part of what made me write this post
I. The AI Urgent Care
Years ago, when I had 3 small children, they all somehow caught whooping cough. I took one of them to the ER after they turned blue, and was very distressed to learn that they would only observe children whose oxygen was under 96. Because my children were fully vaccinated, nobody tested them for whooping cough for the first six weeks. Their decision seemed to follow the correct algorithm. It was wrong.
A few years from now, someone in my position might instead reach out to the after-hours AI. The AI never gets tired or misses a phone call, and it has access to the medical record. It would have even more sources and data to base its decision on. Suppose it reviews the record, and says the child can go home.
But within a couple of hours, the child turns blue again. One of the signs of whooping cough, as opposed to other types of cough, is that the child seems fine in between episodes. Back then, had my child not turned out okay, there would have been people I could complain to and people who could investigate what happened.
What happens if AI made the decision?
II. We Can Keep a Human in the Loop
The obvious solution is to keep a responsible human in the loop, responsible for making the decision. AI can only make its recommendations, conditional on a doctor in the loop, who must approve it.
But suppose the AI is better at medicine than the doctor. And being AI, it's not just a little better than the doctor, but exponentially better. It has seen millions of cases, remembers every relevant paper, catches obscure drug interactions and is right substantially more often than the human doctor.
At first, presumably, the human doctor would check the AI very carefully. But if every time the doctor disagrees with it, the human turns out to be wrong, eventually the human is going to conclude that they should probably listen to the AI.
So I say my child turned blue, but the AI says my child can go home. The doctor clicks APPROVE.
Is the human really responsible? What exactly was the doctor supposed to do? If they don't understand the AI's reasoning well enough to independently reproduce it, saying that they "reviewed" the decision starts to sound like my child saying that they thought I gave them permission to do somwthing.
What responsibility can we reasonably or morally or fairly assign to someone who cannot evaluate the judgment they are approving? What is the human doing in the loop?
III. Lawyers Have the Same Problem
Doctors are an especially obvious example, because if something goes wrong somebody can die, but many other professions seem to have a similar function. Lawyers are needed not only to draft the contract, but to sign it.
Lawyers don't just sell knowledge of law. Depending on what they are doing and for whom, they can also owe clients fiduciary duties, including duties of loyalty and confidentiality, along with other professional obligations. Lawyers can be disciplined, as members of a bar. They can be sued, because they put their name on something and thus took personal responsibility for the answer.
Knowing the answer responsibly is a very different service from knowing the answer.
IV. Responsibility Is Something We Expect of Real Humans
I have six children, so I spend an unreasonable amount of time determining responsibility.
"Why did you hit him?"
"He hit me first!"
"He took my Lego Warden!"
"Because he traded it for a Wither!"
At some point I have to sit down and figure out who did what, who suggested what, who knew what, and how I am going to convince everyone to clean it up. The fact that another child suggested doing something stupid may be relevant, but you are still personally - humanly - responsible for doing the stupid thing.
Adults have a similar, more serious version of this system. We enforce things with contracts, which are enforceable by courts, whose rulings are enforceable, in some circumstances, by prison. But all of these systems assume that humans own things and can lose things. Humans own houses and care about careers and can be embarrassed. Humans can be held accountable when they don't keep their promises.
So what does it mean for an AI to take responsibility? Can we take away its medical license? Can it be shamed or upset? Does it have assets?
We can tell it that if it kills another patient, we will turn on the torture neurons. This seems like a bad idea. This seems related to the solution to the observability problem I wrote about previously.
Maybe this is actually what happens. AI does more and more of the intellectual work, while humans remain attached to important decisions because our legal and social systems require a human being who can be held responsible for them. Are humans to become the whipping boys for AI mistakes? It seems to me that the less the human actually understands and controls the decision, the harder it is to say that the human is responsible.
Right now, when humans make consequential decisions, someone is answerable for what happened. Even if that person didn't personally perform every part of the work, you can eventually point to a human or institution and say to them: you were responsible for making sure this was done properly.
An equivalent system might be built around AI. The responsibility could fall upon the developer, the deployer, the hospital, the insurer, the user, or some combination of them. Perhaps AI systems themselves will eventually be legal entities capable of holding assets and insurance and acquiring something resembling a professional license. But if the AI makes the decision, we need an answer to who takes responsibility for it.