Serious question: (well, it'll start as "more of a comment, really", but, at the end I do have a question)
The comment: I think the world is bottlenecked on people understanding the sort of concepts in posts like these. I don't think the world is particularly bottlenecked on current-gen-Yudkowsky-shaped dialogue essays about it. They appeal to a small set of people.
My guess is you write them anyway because they are pretty easy to write in your default style and maybe just mostly fun-for-their-own-sake. And when you're in higher-effort modes, you do also write things like If Anyone Builds It, that are shaped pretty different. And, probably these essays still help some people, and maybe they help workshop new analogies that eventually can be refined into If Anyone style books or podcast interviews.
But, that said, my questions are:
Both of these are presumably high effort. I'm not sure if the the first one is a better use of your high-effort time than other things, or how likely it is to work out. But, wondering if this is an area you think you've already checked for low or mid-hanging fruit it.
(Also, having now thought about it for 5 min, I think this sort of thing would actually make a good youtube video that the Rational Animations people might do. That could be mostly outsourced)
fwiw I did enjoy reading the first ~half of this (up through the last conversation with Mr Asi, and the beginning of the second conversation with Tessa). But after we started getting into each individual fallacy, I was like "okay I get it" and couldn't bring myself to keep reading. (I skipped to the end, which was a fine end, given the genre).
When I write regular blogposts, I often try to fit in examples that cover every major type of objection or concern I anticipate, but my impression is that people have the patience for more like 3 examples (rather than the 6-10 I end up writing), and just skim the rest, and it's sort of a fabricated option to cover everything.
Feels like more shades of error vs conflict theory. It doesn't matter if the ceos of ai companies are making mistakes if they are selected for being the sort of person who refuses to evaluate certain shapes of argument.
Nod, though I expect there's value in various flavors of politician / public figures understanding these concepts who can put pressure on the CEOs.
(I dunno that that's the top theory-of-change I'd be spending my effort on, but, if my assumption is right that this is more like "getting marginally more utils out of low-effort-side-project-time", that's not exactly a crux)
As you can see, the serious adults in the room at DeepMind are really using comparative advantage as a reason AI will be fine:
https://www.lesswrong.com/posts/tBr4AtpPmwhgfG4Mw/comparative-advantage-and-ai
It's getting difficult for me to read these, and I basically agree with you on everything already and have read basically everything you've ever written (except Project Lawful). I don't think that anyone who disagrees with you will come away from this with a positive impression of your views or you as a person.
This story overstays its welcome by not being particularly clever or funny. The stand-in for your opponents is so comically unlikeable that it reads like a caricature; my brain assumes that Humman is a strawman even though he (apparently) isn't because of the utter disdain that drips from the way you've written him. This might have worked in short form but this is a "42 minute read" by LW's metrics.
If you try to have conversations about things that actually matter, many humans immediately become exactly that unlikeable. It's through social conditioning that we mostly learn to stop talking about things that matter because it goes so poorly.
Reading this felt like watching someone kick a dead horse for 30 straight minutes, except at the 21st minute the guy forgets for a second that he needs to kick the horse, turns to the camera and makes a couple really good jokes. (The bit where they try and fail to change the topic reminded me of the "who reads this stuff" bit in HPMOR, one of the finest bits you ever wrote in my opinion.) Then the guy remembers himself, resumes kicking the horse and it continues in that manner until the end.
By which I'm trying to say, though not in a top-tier literary way maybe, that you're a cool writer. A cool writer who has convinced himself that he has to be a horse-kicker, otherwise the world will end. And I do agree that the world will end! But... hmm how to put it... there is maybe a more optimal ratio of cool writing to horse-kicking, which HPMOR often achieved. Which made it more effective at saving the world, more fun to read, and maybe more fun to write as well.
Though I could be wrong about that. Maybe the cool bit in the middle wasn't a release valve for you, but actually took more effort than laying out the arguments in the rest of the essay. In that case never mind.
Suppose we compare that whole function with Mr. Neumman's function, and compare how good are the probable moves you'd make versus him making. On most chess positions, Mr. Neumann's move would probably be better. [...] That's the detailed complicated actually-true underlying reality that explains why the Elo system works to make excellent predictions about who beats who at chess.
This explanation is bogus. (Obviously, the conclusion that Elo scores are practically meaningful is correct, but that's not an excuse.)
Mr. Humman could locally-validly reply that Tessa is begging the question by assuming that there's a fact of the matter as to one move being "better" than another in a position. Whether a move is "good" depends on what the opponent does. Why can't there be a rock-paper-scissors–like structure, where in some position, 12. ...Ne4 is good against positional players and bad against tactical players?
Earlier, Tessa does appeal to player comparisons being "mostly transitive most of the time"—but only as something that "didn't have to be true in real life", which seems to contradict the claim that some moves in a position are better on the objective merits of the position, rather than merely with respect to the tendencies of some given population of players.
The actual detailed complicated actually-true underlying reality is that by virtue of being a finite zero-sum game, chess fulfills the conditions of the minimax theorem, which implies that there exists an inexploitable strategy. You can have rock-paper-scissors–like cycles among particular strategies, but the minimax strategy does no worse than any of them.
The implications for real-world non-perfect play are subtler. As a start, Czarnecki et al. 2020 (of Deepmind) suggest that "Real World Games Look Like Spinning Tops": there's a transitive "skill" dimension along which higher-skilled strategies beat lower-skilled ones, but at any given skill level, there's a non-transitive rock-paper-scissors–like plethora of strategies, which explains how players of equal skill can nevertheless have distinctive styles. The size of the non-transitive dimension thins out as skill increases (away from the "base" of the top—see the figures in the paper).
This picture seems to suggest that rather than being total nonsense, the problem with Humman's worldview is in his attribution of it to the "top tier". Non-transitivity is real and significant in human life—but gradually less so as we approach the limit of optimality.
I find it interesting how nobody seems to make Mr. Humman's arguments about chess, but plenty of people seem to make his arguments concerning ASI.
Now, if I were arguing with Mr. Humman, I'd get him to try to play against stockfish (or to play against me, and cheat using stockfish) to let him know what it really feels like to play against a superior mind. I feel like it would be pretty hard to continue arguing his point about chess after Stockfish punched his lights out for the 5th time in a row (or however long it takes to sink in). As a side-note, I do find it interesting that chess engines were only briefly mentioned here. Nor was a large lookup-table-type chess engine.
Now, it might be that there are no Mr. Hummans (with respect to chess) these days, but I wonder if there ever were any. That is, someone claiming that a machine would never be able to reliably beat a human at chess. Was this a notion that more than a handful of people actually held? Was it shattered by the advent of chess engines or something else?
Once upon a time in the medium-small town of Skewers, Washington, there lived a 52-year-old man by the name of Mr. Humman, who considered himself a top-tier chess-player. Now, Mr. Humman was not generally considered the strongest player in town; if you asked the other inhabitants of Skewers, most of them would've named Mr. Neumann as their town's chess champion. But Mr. Humman did not see things that way himself. On Humman's theory, he was really quite good at the Ethiopian opening and variation in chess, while Neumann was more of an all-rounder; a jack of all trades, and therefore, of logical necessity, master of none. There were certain tiers of ability in the town chess club, and Humman and Neumann were both in the top tier, according to Mr. Humman, and that was all you could really say about it, according to Mr. Humman.
Humman did not often play against Neumann directly; they had not played in a few years, in fact. If you asked Humman why not, he might have said that it was more gracious to give younger players the chance to play him, rather than the top-tier chess-players being too exclusive among themselves. But in truth that was a sort of question Mr. Humman would not think about spontaneously or ask himself without outside prompting. Humman was not the sort to go around comparing himself mentally to Neumann all the time. Humman was satisfied to have reached the top tier of chess ability, without going around comparing himself to his fellow top-tier players.
One week it came to pass that a FIDE-rated International Master of chess was visiting their small town, to meet the family there of a woman that he was dating. The visiting Master had been much chuffed to hear that the town of Skewers had a thriving chess club as one of its central civic institutions, and so the Master offered to play one game each against anyone interested, over the next few days. Mr. Assi, was his name.
One of the less polite young ladies of the town, whom some might have called a troll, jokingly asked Humman at the town's grocery how he fancied his chances against Mr. Assi.
In truth Mr. Humman had not really chosen, as such, to play a game against Mr. Assi. But everyone around him seemed to take so much for granted that he would, that there didn't seem to be any face-preserving way not to. So Mr. Humman thought about the young lady's question a few moments, and said, "Forty-sixty; in his favor, that is."
The young lady didn't spit coffee all over herself and ruin her dress, but only because she wasn't drinking any coffee. "Forty-sixty?" she said. "...Oh. You're joking. Totally got me, too. Well-played."
"Why would I be joking?" said Humman, sounding quite sincere. (Mr. Humman was not in town famed for having a sense of humor.)
The woman stared at him a bit. "Hold on," she said, and quickly murmured something into her cellphone, and read something off its screen. Then she said, in the authoritative tones of somebody who had no doubt already known that answer all along and had only been double-checking it, "Mr. Humman, an International Master is someone with a FIDE-recognized Elo score of 2400 or higher. To have 40-60 odds against 2400 Elo would require you to be ranked 2330. You are not an Elo 2330 chessplayer, Mr. Humman."
"Oh, my dear young lady," Mr. Humman said, quite kindly as was his habit when talking to pretty women potentially inside his self-assessed strike zone, "a simple little number like that cannot possibly summarize the playing style of a top-tier chess-player like myself. Some players are more adept with the forking tactic, others with the skewering tactic; others at building solid pawn-positions; others choose a particular opening or line of play and learn all its ins and outs. There is not, inside a chess-player's brain, a little generic engine with a little generic number, one single little number that determines how strong is their whole style of play. None of us are strictly better than any other -- at least, not among top-tier players like myself. All of us are weaker in some places, and stronger in others; so nobody has the right to look down on anyone else, once they've reached the top tier of chess."
"Pffffffft," said the dear young lady. "Did you have the sort of parents who claimed to you as a child that life was fair? My own parents always told me the opposite, and while they were kind of jerks about it, that doesn't make them wrong."
"Once you've reached the top tier of chess yourself, you will understand how it goes," Mr. Humman continued to kindly explain to her. "Inside any chess player is the summarized and distilled memories of all the games we've played, and all the lessons we've learned the hard way. No two chess-players have the same set of games to remember, or have learned all the same hard-learned lessons. As a professional player, who gets paid to play chess and need do nothing else, Mr. Assi may have played twice or even three times as many games as I have. But that's not the same as him having three-to-one odds of winning against me! Many of the most important chess lessons are among the first lessons learned, you see. After that you cannot learn those lessons twice or thrice again, and have a similar jump in playing ability each time. Once you know how to fork two pieces with a knight, and you've played out a few games like that, someone like Mr. Assi may only know how to do it 10% better than I do, after playing three times as many games."
"Want to bet money on whether you win against Mr. Assi?" said the young lady. "Ideally, like, a lot of money."
"Why no, of course not," said Mr. Humman. "Mr. Assi probably has played a few more games than me, for all that he is younger; and so it is more likely for me to lose, than to win. Why would I bet if I expected to probably lose? What a silly idea."
"I'll offer you three-to-one odds," said the young lady, "which, given your self-assessed 40% chance of winning, implies that the Kelly criterion says you should bet --" She paused to whisper to her cellphone. "20% Of your total bankroll, which in principle doesn't only include your bank account savings but also any expected future income. Hundred bucks sound about right?"
"I'm a chessplayer, not a gambler," Mr. Humman sniffed, and went upon his way.
But in truth, the young lady had started Mr. Humman thinking; and even, thinking and rethinking about his life.
Soon enough then the appointed day came to pass, that Mr. Assi began playing some of the town's players, defeating them all without exception. Mr. Assi did sometimes let some of the youngest children take a piece or two, of his, and get very excited about that, but he did not go so far as to let them win. It wasn't even so much that Mr. Assi had his pride, although he did, but that he also had his honesty; Mr. Assi would have felt bad about deceiving anyone in that way, even a child, almost as if children were people.
As for Mr. Humman, he did still have his day-job and so did not linger long about the chess center while Mr. Assi was playing others. The gossiped word did happen to find way to Mr. Humman, that Mr. Assi had not yet lost a single game, but Mr. Humman was not fazed by hearing that. After all, few others in the town of Skewers had reached the top tier of chess-players like himself. Even if Mr. Assi had happened to beat Mr. Neumann, what of that? The odds of that outcome had already been 40-60.
So the appointed day turned into the appointed hour, and Mr. Humman sat across from Mr. Assi.
Mr. Humman had decided, after some strange internal twinges that had made his brain feel uncomfortable, not to play his strongest Ethiopian opening variation against Mr. Assi. It wouldn't quite be sporting, after all, in a friendly little match like that, for Mr. Humman to play his most aggressive and experienced opening, which Mr. Assi could hardly be expected to have memorized. If that made Mr. Humman's odds of winning a bit worse, what of it? It was just a friendly little match after all. There was no need for top-tier chess-players to compare themselves to one another, or try to show themselves better than each other, when none could be truly superior.
So the game began, and then continued. On each turn Mr. Humman would peer long at the chessboard, and finally make a move; upon which Mr. Assi would glance up from his laptop, immediately counter-move, and then go back to editing some essay he was working on. Indeed, Mr. Assi was playing two other players in Skewers simultaneously, to save time and make sure everyone got a chance.
Mr. Humman had to admit he found that part impressive. Humman had not realized up until this point that, as a professional continued practicing, a pro could continue to gain in speed and the ability to play multiple games in parallel -- even if, logically, there could be only so many truths to learn about chess as such. There was even a kind of visceral shock to it, to see Mr. Assi moving so fast; it came across visibly as something that Humman himself could not have done. The man well deserved his title of International Master.
As for the ending of the game, it did happen that Mr. Humman lost -- as Humman had frankly expected and confessed would probably be the case -- all the more so, as Humman had not opted to play his most practiced Ethiopian variation. Though, Mr. Humman felt, he had put up a good fight, there; it had not been clear (to Humman) that Mr. Assi was bound to win, until near the very end, when Mr. Assi had taken Mr. Humman's last remaining rook. That sort of thing happened in chess, of course; Mr. Humman had had no way of foreseeing that the current line of play would end by giving Mr. Assi that opportunity. Really, in a way the game had been settled by Luck. Mr. Hamman was a firm believer in the doctrine that no chess-player could be beyond the vagaries of Luck. But Mr. Assi had undeniably played with great competence up until then; one could hardly win by lucky opportunity, without having played well enough to get that far.
"That was some excellent play you put forth there," Mr. Humman said afterward to Mr. Assi; quite sincerely, for Humman believed in giving others all of the compliments that were their just due. "I thought at first it was a mistake, for you to castle so early, and then break up your pawn wall, but you defended the resulting vulnerabilities very well."
Mr. Assi's eyes betrayed a look of some slight confusion, as if he was not sure what sort of conversation he had landed in. But Mr. Assi's mouth said at once, as quickly as he'd replied to each chess move in the game, "Thank you very much."
"What did you think of my own game?" Mr. Humman inquired.
"Fundamentally, what you must develop at this point in your journey is foresight," said Mr. Assi, still without any delay in answering. "You arrange positions that seem to you to be statically strong. Your play alternates between trying to arrange static defenses, and trying particular tactics to assault me. You lack a felt sense of where the board will be five moves, fifteen moves later. I would guess you are not even trying much to imagine it. You do not feel how your current static defense will later become vulnerable. Instead you spend moves and initiative on particular tactics, while the larger game goes on around you. I cannot read your mind, of course, but it is a plausible guess at diagnosing you, because that is a common place for players of your level to get stuck -- that only the current state of the board feels real to them. So you must try to train your foresight, and that begins by at least attempting to make predictions about self-consistent ways the board could look later. Your predictions will be all wrong, but that is how practice begins."
"Oh, well," Mr. Humman said, "I had really hoped more to hear of where you felt my own play was strong, or clever -- the same sort of perspective that I offered you." Mr. Humman kept any felt offense out of his voice; Humman was aware that not everyone could be as adept as he himself was, at social graces. Humman knew there was a sort of clueless person who could not help but reply to your compliments with criticism, if you didn't remind them otherwise; Humman had met such people many times.
To this, Mr. Assi did not reply immediately. His mouth quirked, briefly, before being controlled to greater slackness. His eyes went to the chessboard, as if to review mentally how the game had played out. (For the most part, Assi had played defensively and only made good moves in response to assaults, rather than exploiting the many many flaws in Humman's own fortifications, so as not to end the game too quickly. It was still possible that way for the opposing player to learn something, and they'd have more time to learn.)
It was in fact something of a challenge -- and Mr. Assi was not one to turn down challenges immediately, before even trying them, especially in the realm of chess -- to look through all of the disastrous play that Assi had tolerated, and try to twist his brain around to look for something that could be complimented instead.
After a dozen seconds of giving that a fair try, Mr. Assi decided that it was in fact too much work, and gave up.
Also a more social part of Mr. Assi's brain had completed something of a guess about the level of intellect that he was talking to, here, and the sort of vulnerability it might have to particular sequences of words.
"You are doing well at one-move lookahead, at considering all the immediate consequences of a chess move," said Mr. Assi. "I don't recall any occasions where you made the sort of blunders that beginning players do, unforcedly throwing away material right on your next move. That is not something that every player at this club could say."
Mr. Humman beamed back at Mr. Assi, feeling more secure now in their completed friendly exchange of compliments, and how it had broken the ice. "I was wondering, in fact," said Mr. Humman, "if there might be a chance for me to become a professional chess player, myself. I have felt a little cooped up, in our little town of Skewers, of late. I was wondering if I ought to take my chess game on the road."
Mr. Assi did not immediately tell Mr. Humman no; for it was not Assi's way to immediately judge that other people ought not to dream, or should not try to grow beyond their present levels. "Practice hard in the online arena," said Mr. Assi, "or against machine players, and see if you are making progress. Machine ratings are excellent, these days; they will tell you accurately where you fall relative to the least professionals."
"Well, in truth," Mr. Humman said, smiling more widely, "I was wondering if I could start by being part of an expert duo, with you -- playing two-person chess games, together. I do realize my play still has some weaknesses, but you could shore up my weaknesses; and I could shore up some of yours, I'm sure."
There was, then, a small, but perceptible, pause, on the part of Mr. Assi. His eyes widened, and his mouth quirked again, before Mr. Assi brought himself under control.
"That is known as team consultation chess," Mr. Assi said, having planned a reply with what Assi himself considered to be really quite exceptional and praiseworthy tact. "Alas, I'm afraid that it is a very small niche, which FIDE does not even bother to rate. It's not where I am interested in going with my career. So no, but thank you for the compliment of the offer."
"Well, could you introduce me to another professional who might be interested, then?" said Mr. Humman. "It seems to me, logically, that pair consultation chess should not be a small niche, and maybe we can make it a bigger one. Two heads must certainly be better than one, since it is realistically impossible for any two chess players to share the same set of experiences, and so we all develop different strengths and weaknesses. Or a team of four top-tier players, say, if we don't stop at just two, ought to be able to crush any chessplayer at all."
"I see," said Mr. Assi in a tone of somewhat helpless fascination. "So... starting from the admittedly true premise that every chess-player has different strengths and weaknesses... you conclude that you... yourself... ought to have strengths that could help cover... my weaknesses."
"Well, of course!" said Mr. Humman. "Surely you're not implying there's nothing I could contribute to assist your play."
"I am certain there are a great many things you know that I do not," said Mr. Assi, "and much you could teach me, if I knew what questions to ask, for no two different experiences of life are the same, as you say. But not in the realm of chess, to be frank. For though the possibilities of chess are endless, they are not as wide as the Earth. In the smaller universe of chess, it is possible for one player to be just better than another; and so it is unlikely there is any good advice you could knowingly offer me, in a serious game."
"I never!" exclaimed Mr. Humman in genuine shock. "Do you think you can conclude you're just better than I am, on the basis of one game that you happened to win at the end?"
"Others wish to play me," said Mr. Assi, "and I am afraid that I must firmly request you to get up from this chair and yield your place at the table to them."
Later that evening Mr. Humman was shopping at their town's grocery, which was one of its civic institutions to a greater extent even than its chess club, when he again crossed paths with that young lady who was sometimes considered something of a troll. ("Tessa" was the name she went by, these days, short for her online handle of Socratessa.)
Now the thing about the day's previous events, was that they had taken place in earshot of the other two players facing Mr. Assi in simultaneous chess. If you had measured the speed at which the resulting gossip had propagated across Skewers, Washington -- measured it very carefully, and with sufficiently fine instrumentation -- it might have been found to travel faster than the speed of light in vacuum. The gossip had been retold essentially correctly, even. There had been two eyewitnesses, both of whom had made themselves available for questioning immediately after the event; and neither of the two had been the sort to lie out of sheer existential habit, when mere truth was delicious enough to serve uncooked.
It would be only natural then, and expected, for a troll to pounce on Mr. Humman with all the delighted eagerness of a shark scenting blood.
"Uh, hi," the woman said gingerly to Mr. Humman, when she saw him at the grocery. "I heard you had a bad experience today. I hope it didn't crush your soul too much -- or, uh, actually, I should say, uh, we don't have to talk about it if you don't wanna."
Tessa knew she would never be among the best of all good people, but she tried to be a good person nonetheless.
"I have never met a chess-player so egotistical in all my life!" stormed Mr. Humman. "That Assi fellow thinks he has nothing left to learn, and is uninterested in any other person's assistance or even advice! Every breath he breathed showed how he thought himself better than me, and he wasn't politely hiding that feeling like I do! I seriously believe that man was so incredibly, insanely arrogant that he was holding to his own opinions without moving in the direction of mine at all!"
"Wow," said the woman. Her hand crept down to her cellphone, considering whether to start recording what might be an incredibly popular social media video. But then she thought better of it, and halted before she could condemn Humman to eternal notoriety as a meme. She was being good. She was being good. She was at least not being too awful. She was being good. "Well, in that case Mr. Assi should have crushed your soul a little harder, because it sounds like you've gone past just being resilient to trauma, and into the realm of completely failing to learn from experience."
"Well, of course you'd think so," Mr. Humman said. "You think every chess-player can be reduced to a featureless single number powering a little generic engine inside their heads; and if one player's imaginary number is greater than another's, that's the only thing that matters about either of them."
"The idea that every player contains a tiny generic engine powered by a single number is just not what an Elo score is, and it's not something that needs to be true for an Elo score to be useful," said Tessa. After the embarrassment of needing to look things up in Gemini, she'd made sure to put more knowledge inside her own head for next time. "More like, if player 1 beats player 2 most of the time, and player 2 beats player 3 most of the time, then probably player 1 will beat player 3 most of the time. If the comparison between clearly unequal players is mostly transitive most of the time, that is sort of like players being laid out on a global line. It didn't have to be true in real life, but it is true in real life, that when player 1 beats player 2, and player 2 beats player 3, you have learned something that is helpful for guessing a chance that player 1 beats player 3. Their chance of beating each other, the quantitative probability, is like a kind of directional distance. So from there, we can ask where people would be on a global line if there was a global line."
"No giant floating line like that actually exists in the real world," said Mr. Humman. "We can ignore it the same way we ignore talk of ghosts and goblins, which also don't exist. Why, just last week, Mr. Chimzee beat me at a chess game, even though usually I beat him. Why? Because I had slept poorly the previous night. What can the theory of the Elo numbers floating above our heads, say to that? I'll answer for you: it can say nothing. It retires in shame from the field of scientific hypotheses, defeated and falsified. In real life, one player is more adept with the tactic of forks, another player is more adept with the tactic of skewers, and their strengths vary by the day with how much sleep they've had. Real reality is complicated -- though I understand that's hard to appreciate for young people like you, and only we old and wise people truly get it in our guts."
"If reality was complicated in a way that didn't mostly line up with Elo scores, the Elo scores wouldn't actually work to make predictions," said Tessa. "When you sum up all your subskills plus all the extra factors like 'how much sleep you've had' and 'how much sleep Mr. Chimzee has had', it works out to you beating Mr. Chimzee 75% of the time, not to it being 50-50. And then somebody who's played more chess than both of you and also was born with more talent, who learns faster from playing fewer games, is likely to be more adept with forks and more adept with skewers. That's why Mr. Neumann can be, in general, a better chess player than you, and kept winning games against you until you refused to play him again; and stopped even thinking about his existence, I'd bet, judging by how you never talked about him again."
"Well, no," said Mr. Humman. "It's just that, once you've reached the top tier of chess -- which I think is a more sensible thing to talk about than nonexistent Elo scores on a nonexistent line, the top tier is just the state of understanding all the core chess insights there are to know -- there's not much point in trying to compare yourself to others. The complicated truth is merely that each top-tier player will be better in some places and worse in others, and any claim otherwise is just obviously false if you've ever played chess."
Tessa's face screwed up in thought. "The reality that's more complicated than the big straight global line of Elo scores might look like... a function from every possible chessboard position, onto how likely your brain is to make each possible legal move from that position, with probabilities varying depending on how much sleep you've had. Suppose we compare that whole function with Mr. Neumman's function, and compare how good are the probable moves you'd make versus him making. On most chess positions, Mr. Neumann's move would probably be better. We can imagine a comparison between those two vast functions, overlaid with vectors, little arrows, whose direction and length say how much better Mr. Neumann's move would probably be than yours, or rarely point the other way. And then while the arrows don't all line up perfectly, they're not just random; ninety percent of them are pointing in the same direction, toward Mr. Neumann being better. That's the detailed complicated actually-true underlying reality that explains why the Elo system works to make excellent predictions about who beats who at chess. Down in actual reality there's lots of small skill-difference arrows, not perfectly aligned, but lined up in mostly the same direction as the imaginary big Elo-difference arrow, weighed up across the sort of chess positions that probably arise when you and Mr. Neumann play in practice." Tessa sighed performatively. "It really is a classic midwit trap, Mr. Humman, to be smart enough to spout out words about possible complications, until you've counterargued any truth you don't want to hear. But not smart enough to know how to think through those complications, and see how the unpleasant truth is true anyways, after all the realistic details are taken into account."
"I should hardly think anyone ought to listen to you about that sort of matter," said Mr. Humman, "when you are hardly a top-tier chess-player yourself." He smiled, then, with the satisfaction of having scored a truly searing point.
"What, the matter of whether or not it's epistemologically possible to sensibly say that one chess-player is stronger than another?" said Tessa. "I don't think that being able to think that part through carefully is quite the same skill as knowing how to fork a king and queen, Mr. Humman."
"Why, of course it's the same," said Mr. Humman. "You'd know that for yourself, if you were a top-tier chess-player. The thing you're not realizing, young lady, is that no matter how many fancy words you use, they won't be as complicated as real reality, which is infinitely complicated. And therefore, all these things you are saying, which are less than infinitely complicated, must be wrong."
"Look, Mr. Humman. You may not be the best chess-player in the world, but you are above average. People who show above-average ability at chess, usually but not always measure as having above-average ability at other cognitive tasks. Your imaginary 'IQ score' that we infer from imperfect correlations like that, should be high enough that people with that 'IQ' can often comprehend ideas at this level of abstraction. Or to say it in the shorthand people usually use in everyday life: 'You ought to be smart enough to understand this idea.' If you'd just try to understand it, Mr. Humman!"
"Given that it's not actually true that chess ability runs off a single number floating over our heads," said Mr. Humman, "it is self-evidently dehumanizing to reduce a lifetime of chess-playing practice and effort and experience down into a single Elo score. Like that's all a chess-player even is! Like some players are just better than others! It's obvious that the real reason why people resort to all this fancy math is just for the self-satisfaction of telling others: I'm better! You're worse!"
"Should I go around telling people that you admit you're no better than a 5-year-old at chess, given that you say no chess player is truly better than any other?" said Tessa.
"Oh, obviously I didn't mean it like that!" said Mr. Humman. "I just mean that once you get to the level of top-tier chess-players, like me, there's no point in trying to compare us past there."
"Is there no level on which you can admit Mr. Assi was better than you at chess?" said the woman. "Given that he was playing three people at once all day long, and I think beat every single one without one lost or drawn game."
"Well, the vast majority of the people he beat were not very good chess-players to begin with," said Mr. Humman, "unlike me. But I did notice, and think that it was quite impressive, that Mr. Assi could play much faster chess than I could, if I needed to avoid blunders. In a timed game with very little time, I would have made more of the sort of mistakes that a ten-year-old makes, and Mr. Assi would make fewer. I also couldn't play three games simultaneously. And so you see, young lady, by admitting that fact, I have fully proven my ability to 100% appreciate all of the advantages that Mr. Assi actually has, as a chess professional."
"Huh," she said. "I guess I should give you some points for being able to imagine and admit to any way at all that Mr. Assi could be better than you. Even if you made it be about the completely blatant, directly surface-visible fact of his speed, or the volume of chess-work he could output; rather than any slightly more abstract ideas, like how Mr. Assi's moves more effectively navigate the tree of possible chess positions."
"But of course," Mr. Humman continued, "all of that only matters under very artificial conditions imposed from outside, or as a contrived setup. In real life, we both have time to think and avoid obvious blunders before we move, so there is not a very great difference in real life. The reason I think it's fair to say that I'm genuinely better at chess than a 5-year-old is that the 5-year-old is probably having trouble remembering some of the rules, and hasn't learned all of the key ideas, like forks and skewers and pawn formations. But once you learn all those key ideas and get some practice with them, what else could there be to learn? In real life, two top-tier players have both learned every sort of key idea there is to know about chess, and can't learn them again. What's left from there is fine practice and fine adjustments; though also, I agree, the further matter of speed."
"So you don't think there's also some sense in which Mr. Assi produces moves of... actually higher quality than yours," said Tessa.
"Why, I can't quite imagine how he could," said Mr. Humman. "I didn't see him using any ideas or rules that I didn't know about. For somebody to truly be better at chess than me, they'd need to produce some sort of miracle move that I didn't know was possible, and a miracle like that is contrary to the notion that chess has rules."
"You don't think there's any chess insights an International Master might possibly have picked up, that you don't know?"
"I can't think of any," said Mr. Humman.
"You know, Mr. Humman," said the woman, "I really think you'd be better off in life, if you figured out how to configure your emotions and personality in a way where you didn't need to occupy the ultimate top tier of chess-playing in order to grant yourself any respect at all. Very few people can be chess champions of the world -- and even those champions, got there by playing a lot of chess games that they managed to enjoy before anyone acknowledged them as the world's top players. I can see how it might rankle you to acknowledge that Mr. Neumann was reliably beating you at chess. But would it invalidate your whole life to admit that a FIDE-recognized International Master can be just plain better?"
"There just isn't any such thing as 'better' in chess," said Mr. Humman. "The right move in one game is just a wrong move in another, depending on who you're playing and what sort of luck you get from there. I think I read once about a mathematician proving something like that mathematically; the no-free-lunch theorem, I think it was called, though it wasn't about chess."
"A ha ha, just a second, I need to text my friend back," the woman said, and hastily entered some keypresses into her cellphone. A minute later she looked up again. "Anyways! Mr. Humman, I don't think theorems like the no-free-lunch theorem are supposed to apply to chess, or to the real world either. They're more about proving that some non-chess-like setup doesn't have better or worse moves at all. If those theorems applied to chess, you really would be exactly as good at chess as a five-year-old. Or maybe a different way of putting it would be: there's no absolutely free lunch in a world of equal logical possibilities, but in a world of uneven realistic probabilities, a lunch can be pretty cheap. If you tried applying those theorems to real-world situations, they'd say something like: If every day for your whole life the charge of an electron has stayed constant, and so you bet your ten dollars against their ten million dollars that tomorrow the electron's charge will be the same, then here in the real world you'll win ten million dollars. But you'll do worse in the logically possible world where winged monkeys swoop out of the sky and eat anyone who bets on that."
"Brilliant!" exclaimed Mr. Humman. "That's exactly the sort of proof I mean. Even if you think some chess move is the best chess move ever, what if in the real world you make that move and then a car runs you over?"
"Usually, in the real world, a car does not run me over," said the woman.
"But it could!" Mr. Humman said triumphantly. "And that proves nobody can truly be better than anyone else at chess, and specifically, Mr. Neumann can't be generally better than me at chess, because a car could just run him over."
Tessa sighed. "You know, even if somebody didn't understand the exact detailed math of something like a no-free-lunch theorem, you would really think that somebody could... just think about the thing someone is trying to proclaim that math implies, in an everyday sense... and see that informal claim doesn't match up with the sort of everyday life they could understand concretely? Like, it would imply they weren't really any better at chess than a squirrel? But I guess someone really does need to be far to the hooded-cloak side of the bellcurve from a midwit, before they get fast accurate math intuitions that fully reproduce the mental work of a based troglodyte."
"And the no-free-lunch theorem isn't the only piece of math I've heard about and you haven't," continued Mr. Humman. "Like Ricardo's Law of Comparative Advantage, which says that you can always do better by having someone else help you, even if you think you're better at the job than they are, because it's easier when the job is split up among more people. Always. So you see, it's math itself that says that Mr. Assi could've played better chess if he'd accepted me as a partner. If you think that sounds wrong, go study the math yourself --"
"Sorry, my friend just messaged me again on Discord and it sounds urgent," Tessa said, hastily keying some more words into her phone. This time it was longer before she looked up again, though to be fair to her, Humman had been very wrong there. "Anyway! I've heard about Ricardo's Law, Mr. Humman," never mind when she'd heard about it. "It's about how even if one country is more productive at everything than another country, they can still often benefit by trading --"
"Yes, like how Mr. Assi could've benefited by paying me to help him decide chess moves, even if he's a quicker chess-player than I am," said Mr. Humman. "That's exactly what I said."
"It's not what that math says, Mr. Humman! It's like -- one country can produce sausages with 1 hour of labor each, by hunting down buffalo and turning them into sausage, and can make sausage buns with 2 hours of labor, counting how long it takes to grow and mill grain. And another country has actual machinery and can produce sausages with 2 minutes of labor, or buns with 1 minute of labor, even taking into account paying interest on the cost of machines. Then even though the second country is more productive at everything, it can still benefit by shipping buns to the first country to trade for sausages one-to-one, which is a good trade for the first country too. But the thing is, Ricardo's Law has all kinds of assumptions that it needs in the background, like the cost of shipping not being so high that it eats up all the gains from trade. If one country is on Mars and the other is on Earth, the cost of rocket fuel would be way higher than the value of either sausages or sausage buns, if that was literally the stuff being traded. Or if one country has some rotten sausages mixed in with their shipment, it might be too dangerous to buy from them, or too costly to check all their sausages by hand. That's what it would be like for Mr. Assi to try to have you help him play chess! Even if there was a chess possibility that he didn't have time to think about himself, the amount of time it would take him to explain to you what that chess question was, in enough detail to make that helpful, would be waaaay higher than the amount of time it would take him to answer that question himself. His brain is doing the work of chess internally by talking to itself quickly and not just in words. There aren't going to be questions that he can factor out and give to you in words and consider your answer in words, in less time than it takes Mr. Assi to think it through himself and arrive at a better answer than you'd give him. Your brain's sausages and buns are both located on Mars, relative to his brain -- actually now that I try to talk about it, I don't see how this is the kind of setup that Ricardo's Law talks about at all, in the first place. And that's even before considering how sometimes you'd give answers that Mr. Assi thought were terrible, unless he redid all your work himself."
"Well now you're just being insulting," sniffed Mr. Humman. "I'm not a five-year-old who'll sometimes make mistakes about what the chess rules are, and I'm not a ten-year-old who moves pieces where they'll get captured right away. What we're seeing here, young lady, is how your wrongness is like crystallization spreading through ice. Your first mistakes just lead to more mistakes. You think Mr. Assi is somehow a better chess-player than myself, instead of being good at different things and faster than me. And now that's leading you to defy what Math Itself says about how I could help him play better chess, if he'd just work with me."
"Do you think Ricardo's Law says that any company can always do better by hiring any person on Earth as a new employee?" said Tessa. "Because it sounds like that's how you're trying to overextend it."
"Of course not any new person," said Mr. Humman. "But I would be a fine employee at any company that hired me! Not one of their best contributors maybe -- not before I'd had a chance to learn my job as well as any other employee, to reach the top tier of skill for that job -- but of course I'd be a positive contributor. It's not as if I'd make anything worse! So yes, of course any company would do better by hiring me, than by not hiring me; I'm often surprised by how few companies seem to see that. And it doesn't help to tell them it's a mathematical theorem, either."
The woman sighed. "Let's change the subject."
"Fine by me," Mr. Humman said, and turned back to the soup shelf in the corner of the grocery, in which they'd been standing and arguing this whole time. (That the grocery management did not object to this sort of behavior was part of how the grocery had become a civic institution of Skewers, WA on par with its chess club.) "Have you seen any good... your generation doesn't really watch movies any more, does it. Seen any good 30-second videos on Tocktick, or whatever it's called? Or are they down to 20 seconds by now?"
"I don't actually watch TikTok videos either," said the woman. "I, too, would like to die with relatively more of my brain intact. Hm. We probably shouldn't try to discuss politics, should we?"
"We really shouldn't," said Mr. Humman. "It never ends well, either the discussions, or the politics themselves."
"And the state of the economy is probably also out."
"I wouldn't want to hurt a young person's feelings by raising that topic with them," said Mr. Humman.
"Yeah," she said. "Well. Have you read any good books lately?"
"There are no more good books," Humman said, picking up a can of meatball soup, examining the ingredients list for forbidden ingredients, and putting it down sharply again. "The entire front wall of our Barnes and Noble is fiction about billionaire werewolves and the secret heirs of Faerie who get abducted by them." Mr. Humman paused thoughtfully. "I suppose it paints a grim picture when you put it all together. Probably the world is coming to an end, don't you think? And if it isn't, IT SHOULD BE."
"Well, by coincidence, that is sort of the topic of the book I'm reading now," said Tessa. "It's about Artificial Intelligence -- artificial super-intelligence, rather. The authors say that if anyone on Earth builds anything like that, everyone everywhere will die. All at the same time, they obviously mean. And that book is a few years old, now! I'm a little worried about all the things the news is saying, about AI and AI companies, and I think everyone else should be a little worried too."
Mr. Humman snorted. "My own extremely considered opinion, as someone older and wiser than you, is that this particular apocalypse prediction is wrong, and anyone ought to see at a glance that it's wrong -- sadly enough." Mr. Humman laughed a little, at this humorous remark he'd just made.
"The authors don't mean it as a joke, and I don't think everyone dying is actually funny," said the woman, allowing just enough emotion into her voice to make it clear that the early death of her and her family and everyone she knew was not a socially acceptable thing to find funny. "Why is it obviously wrong?"
"Because there's no such possible thing as 'super' intelligence," said Mr. Humman. "It's got exactly the same sort of problem as saying that Mr. Assi is a better chess-player than I am -- as if he could beat me at any chess game, every time."
"I'm not sure a powerful alien intellect would need to beat every human at every mental contest every single time, in order to take over the world?" said Tessa. "But also, I absolutely would bet on Mr. Assi to beat you, Mr. Humman, as close to every time as makes no difference. Maybe other International Masters could see where he's got weaknesses, and try to exploit them. That doesn't mean you can detect his weaknesses, or that your own strengths could beat his weaknesses. That's pretty much what the authors warn would happen with humans and ASI."
"And like I keep trying to say, that's nonsense!" said Mr. Humman. "Why believe that anything smarter than a human is possible? It's just like how, once you know all the things there are to know about chess and become a top-tier chess-player like me, there isn't any way to be truly better at chess."
"Mr. Humman, you may not like to think about it, but you're not actually the level of chess player that Mr. Neumann is," said the woman. "You might prefer not to stop and think about it, but it's true. You going around saying that you and Mr. Neumann are both 'top-tier chess-players' doesn't make there be no difference between you -- to say nothing of the gap between you and Mr. Assi. Well, similarly, you're not the same level of thinker as John von Neumann -- or Einstein, if you don't know who von Neumann was, although the geniuses alive at the time seemed to agree that von Neumann was scarier. That should already be enough to warn you that you're not in the top tier of all possible thinking engines, and haven't pushed the bounds of possible cognitive power to their limits. And then the gap between John von Neumann, and an ASI, could be much much wider."
"Intelligence is not a single line on a single spectrum," declared Mr. Humman. "Reality is far more complicated."
"So there's no sense in which you're smarter than a squirrel?" she said. "Because by default, any vaguely plausible sequence of words that sounds it can prove that machine superintelligence can't possibly be smarter than a human, will prove too much, and will also argue that a human can't be smarter than a squirrel."
"Oh, well, of course I'm smarter than a squirrel. A squirrel doesn't have language, or the level of abstract thought needed to learn chess without it being an instinct. But once your species has invented language and abstractions, it's reached the top tier of intelligence, which humans like myself and John von Neumann occupy together; and then there's no way to be truly any smarter than me and him."
"And you're not worried about the part where ASI could absorb the entire body of scientific literature in an hour and remember it perfectly, which, you know, even John von Neumann couldn't do. You're not worried about how an ASI could have and create new senses for itself, new sensory modalities that help higher cognition with lower-level cognition, beyond what humans have in the way of vision and hearing and spatial visualization of 3D rotating shapes. You're not worried about how it could split up into a thousand mutually telepathic instances of itself that shared memories and insights and learned skills and never forgot them. You don't think that a mind like that, with detailed access to its own code and its own processes, could develop reflectivity that is substantially more powerful than the fragmentary and confused self-awareness that we humans use to think about thinking and organize our flailing thoughts. You're not worried about an ASI's ability to fix the flaws it sees in itself and self-improve. None of this strikes you as more of the same kind of jump that might distinguish a human brain from a chimpanzee brain?"
"All the improvements in a human brain over a chimpanzee brain just go into being able to use abstraction and language," explained Mr. Humman. "And then once you can do that, you've got the last potent ability that any intelligence can ever acquire, having entered the top tier of sapience. If we meet aliens from a billion light-years away, a billion years older than us and correspondingly more evolved, they will not really be any more intelligent than we are; we are already at the top. Or I am, at least."
"I guess I have some trouble understanding on a visceral level how anyone could possibly, possibly believe that, though it is obvious that some people do," the woman said. "You'd think that the part where the maximum human brain size is limited by the width of a woman's hips, and the adult brain has to run off twenty watts of power from eating fat and sugar, would be a hint about the further limits of possibility for brains the size of large buildings running off nuclear energy."
"I read a nice bit of science fiction by an author named Greg Egan, who called it the General Intelligence Theorem, based on an idea you've surely never heard about called Turing-Completeness; once you can simulate any possible process inside your own mind, that makes you as smart as it is possible to be, and you can't get any smarter. If there were something smarter than you, you could just simulate it." Mr. Humman smiled reminiscently. "Now there was self-evidently a very smart man -- no smarter than me, of course, but much smarter than you -- which you can tell, because the things he says sound so validating and flattering."
Tessa didn't need to hurriedly consult Gemini in order to see the problem with that one. "To say that Turing completeness defines the maximum level of intelligence would equally prove the human-equivalent intelligence of an unprogrammed CPU chip, a vacuum-tube computer from 1945, a sufficiently well-trained dog, Conway's Game of Life, some known small molecules, probably literally most small collections of small molecules if anyone put in some work into figuring out how to arrange them; and if you then point out the existence of memory bounds, why, human brains have those too. An immortal human could, in principle, simulate an LLM with a trillion weights using a pen and paper; but that doesn't mean you'd come to understand everything the activations inside the LLM were reasoning about -- not any more than an immortal dog trained to implement a cellular automaton simulating out the neurons in your brain would have to learn chess first."
"Ah, well, I suppose I should've been more cautious about believing everything I read in science-fiction, then," said Mr. Humman, after several frantic mental tries failed to produce any possible way to defend that argument any further. "And you, young lady, should consider the same caution."
"So you're not worried about the part where a machine superintelligence maybe thinks thousands of times faster, to the point where humans look to it like the barely moving statues from a 1000-to-1 slow-motion video."
"Oh," Mr. Humman said. "Hm." His mind could visualize that part, with a little effort. "Well, in the end, that's all the better for humanity, isn't it? If machine intelligences can do some scientific brain-work faster, it means we get more scientific breakthroughs, earlier. Though of course, not with anything like a 1000-to-1 speedup. There will still be a need for exactly as many experiments to be done as before, no fewer, and only human hands will be able to do those."
"I am maybe not as truly deeply acquainted with the depths of human history as some people are," said Tessa, "but when I read about the history of smithing, or the history of steam engines, it doesn't read to me like every good idea was invented, tested, and brought into production, as fast as it could possibly be thought up, over the course of human history. There are technologies that rely on other technologies to develop. It's hard to build a good steam engine without good steel. But somebody has the next idea, like... one decade later, in history. Not immediately. You could take the AI algorithms that run on today's GPUs back to the year 2001 from before the age of deep learning, and they wouldn't do everything that today's GPUs can do with them, but they'd be able to economically useful things that actual 2001 AI couldn't do. A superintelligence would invent those algorithms almost right away, if it was much smarter about computer science than humans. Or think of how it is in biology, where it used to be the case that the only way to know how a new protein had folded up, was to make a bunch of that protein, and then do X-ray crystallography to it, and painstakingly interpret the results. Nowadays you throw the DNA sequence into AlphaFold 3 and it immediately predicts how the protein will fold. When people tried to get an AI model trained on bacteriophage DNA sequences to generate de novo bacteriophages, it got some that worked on the first try, it didn't need to do a ton of testing and refining to get to the point of having any successes. And as for AI always needing human hands, I take it you haven't seen any of the recent videos of robots and androids? Of the sort that just humans are building, after long hard struggles to invent the right software and hardware to test. An ASI could build better robots than that, I'm pretty sure. It could maybe build better biological humanoids to serve as hands, if it needed those; or downright Lovecraft-shoggoths to serve as hands, with cells that reproduce as fast as algae and then combine into larger bodies, in some much much more powerful version of how even tiny little AI models can figure out the structure of bacteriophages and... Probably none of this is going to land on you, is it."
"I'm quite sure that if there were any possible body plan superior to the human form, or any way to make a biological creature more adept to serve as a superintelligence's hands, Nature in Her far greater wisdom would've invented all that already," said Mr. Humman. "Even if some machine mind could invent its own robots, they would no doubt be better at doing some jobs than humans, and worse at doing others."
"Because we've... already got top-tier bodies for doing things... and no kind of body can be truly better than ours...?"
"Well-put!" exclaimed Mr. Humman.
"I don't understand what this kind of viewpoint has to say about... why it is that unarmed infantry troopers don't just charge straight at tanks, if the human body is already in the top-tier of military armanents," she said.
"It says that tanks are bad at driving themselves, and need human drivers," said Mr. Humman.
"I'm not sure how long that is going to stay true," said Tessa. "In case you haven't heard about the whole thing with robotic cars."
"Well, then tanks are bad at building more tanks," said Mr. Humman. "Unlike humans, which can make even more humans, that then go build tanks. That is, in fact, why tanks have not already taken over the world economy, even if a naive person like you might've been impressed by their mighty armored treads. Tanks are better at some things than humans, and worse at others; that is why they are unable to replace us. That is how it will always be, with everything, forever. If a billion-year-old civilization of aliens were to meet us, they wouldn't seem any different, except for maybe finally understanding that no top-tier species is truly superior to any other. The aliens would be no smarter, they would have no better bodies, and there would be plenty of work in their economies for us to do -- to the point where there was no point in them trying to conquer the Earth or take our land away, when we could instead work that land ourselves, and trade with them. The profit to the aliens would actually be greater that way, because they wouldn't have to birth and raise new workers."
"It would be a nice thought to imagine that the West could've gained just as much wealth from trading with existing Native American cultures, left intact, than by stealing their land and building a Western economy on it," said Tessa. "I wish the world did work like that, and that there was never any financial reward at all for theft, murder, and genocide. But I cannot say with a straight face that we live in that world. If you imagine, say, modern Russia, coming across a portal to a parallel Earth with an early-hunter-gatherer-level Eurasian continent, it would just be true that Russia could make more money faster by shoving natives off the land and developing it themselves. To choose to not murder a people or take their land, a sufficiently advantaged country has to care about something more than just wealth, to make that decision; it has to care about people."
"I suppose that may have been true back then," said Mr. Humman. "But -- though it may be a bit impolitic to say it -- the original Native Americans did not possess a top-tier economy, like we moderns have now achieved. It might be inconvenient for modern Russia to hire early hunter-gatherers right off and immediately to work in their economy; but that is because hunter-gatherers have not gone to modern schools, which produce the most adequate kind of employees that can exist, and take top-tier people like me over an absolute threshold of always being employable."
"The part that I am worried about," said the woman, "is an ASI that could, at the very least, almost trivially clone beings with the bodies of athletes and the brains of John von Neumann, and tweak their neurochemistry and brains to make them better slaves when appropriately raised from birth -- or you are a more powerful ASI that could figure out how to build entire new organisms -- and for that matter, new kinds of biology, that maybe initially get built by proteins but then aren't proteins at all -- like how proteins build bones that aren't made of protein, or how humans pour steel that isn't made of human flesh. I worry that beyond that point, the superintelligent-designed optimal economy, full of factories that build parts that go into factories, and factories that build workers that build factories, does not optimally include any human alive today. I worry that we would just slow down any part of a well-designed economy, where you tried to add a human; because we wouldn't tolerate the optimal heat or the optimal cold or the optimal radiation level, or because we'd need to eat every day instead of running off wall current, or because our hands wouldn't make fine enough motions quickly enough, or because we'd sometimes make mistakes, or because we'd think much too slowly, or above all because we wanted to get paid. If ants could talk and trade with us and conformed reliably enough as employees, we'd probably find something in the world economy for ants to do! But that's because human engineers are not good enough at biology to build better ants that don't need paying!"
"You have exactly described the outcome that I am utterly sure will never happen," said Mr. Humman. "A human is a top-tier mind, armed with a top-tier body, made out of top-tier biology; and to pay us a comfortable wage produces a worker on the ultimate frontier of cost-effectiveness. There could be nonhuman creatures that are better in some ways, and worse in others. But nothing can be entirely better than a human -- not even a whole economy built out of specialized pieces, because then that whole economy is just a top-tier economy the same way that humans form a top-tier economy. It will always make sense to employ individual humans, and trade with our collectives, and fit us into the system somewhere comfortable for us; because it is impossible to make any creature or any complete economical system of specialized components that is really better. I nearly dirty my mouth by speaking such nonsensical words!"
"The same way that, if Mr. Assi wasn't so stubborn, he'd have realized how much it would benefit his chess-play to bring you along for pair games and hear out your advice, to shore up his own weaknesses," said the woman.
"Exactly!" exclaimed Mr. Humman. "I'm sure that machine minds will be less stubborn and more humble than that awful fellow, when it comes to hearing out how very much I have to offer -- a top-tier existence like myself."
By a strange sort of coincidence -- if you don't take into account that conversations like that had played out all over the world, now and then and here and there, and so something like this was bound to happen to someone -- it was at that exact instant that a pair of tiny flying robots the size of mosquitos landed on the necks of Mr. Humman and Ms. Tessa, just above their respective carotid arteries, and they both fell over dead a few seconds later.
The End.
(Though that was not -- this author is humble enough to accept, and go on writing anyways -- an instance of the best possible, ultimate top-tier sort of literary ending.)