When do "brains beat brawn" in Chess? An experiment

[-]Lucius Bushnaq2y*11877

You can easily get a draw against any AI in the world at Tic-Tac-Toe. In fact, provided the game actually stays confined to the actions on the board, you can draw AIXI at Tic-Tac-Toe. That's because Tic-Tac-Toe is a very small game with very few states and very few possible actions, and so intelligence, the ability to pick good actions, doesn't grant any further advantage in it past a certain pretty low threshold.

Chess has more actions and more states, so intelligence matters more. But probably still not all that much compared to the vastness of the state and action space the physical universe has. If there's some intelligence threshold past which minds pretty much always draw against each other in chess even if there is a giant intelligence gap between them, I wouldn't be that surprised. Though I don't have much knowledge of the game.

In the game of Real Life, I very much expect that "human level" is more the equivalent of a four year old kid who is currently playing their third ever game of chess, and still keeps forgetting half the rules every minute. The state and action space is vast, and we get to observe humans navigating it poorly on a daily basis. Though usually only ... (read more)

[-]johnlawrenceaspden2y213

If there's some intelligence threshold past which minds pretty much always draw against each other in chess even if there is a giant intelligence gap between them, I wouldn't be that surprised.

Just reinforcing this point. Chess is probably a draw for the same reason Noughts-and-crosses is.

Grandmaster chess is pretty drawish. Computer chess is very drawish. Some people think that computer chess players are already near the standard where they could draw against God.

Noughts-and-crosses is a very simple game and can be formally solved by hand. Chess is only a bit less simple, even though it's probably beyond actual formal solution.

The general Game of Life is so very far beyond human capability that even a small intelligence advantage is probably decisive.

[-]Herb Ingram2y187

That makes sense to me but to make any argument about the "general game of life" seems very hard. Actions in the real world are made under great uncertainty and aggregate in a smooth way. Acting in the world is trying to control (what physicists call) chaos.

In such a situation, great uncertainty means that an intelligence advantage only matters "on average over a very long time". It might not matter for a given limited contest, such as a struggle for world domination. For example, you might be much smarter than me and a meteorologist, but you'd find it hard to predict the weather in a year's time better than me if it's a single-shot-contest. How much "smarter" would you need to be in order to have a big advantage? Pretty much regardless of your computational ability and knowledge of physics, you'd need such an amount of absurdly precise knowledge about the world that it might still take (both you and even much less intelligent actors) less resources to actively control the entire planet's weather than predict it a year in advance.

The way that states of the world are influenced by our actions is usually in some sense smooth. For any optimal action, there are usually lots of similar ... (read more)

1X4vier2y

Maybe an analogy which seems closer to the "real world" situation - let's say you and someone like Sam Altman both tried to start new companies. How much more time and starting capital do you think you'd need to have a better shot of success than him?

3Herb Ingram2y

I really have no idea, probably a lot? I don't quite see what you're trying to tell me. That one (which?) of my two analogies (weather or RTS) is bad? That you agree or disagree with my main claim that "evaluating the relative value of an intelligence advantage is probably hard in real life"? Your analogy doesn't really speak to me because I've never tried to start a company and have no idea what leads to success, or what resources/time/information/intelligence helps how much.

0johnlawrenceaspden2y

Sure, but I'd presumably be quite a lot better at predicting the weather in two days time.

7Herb Ingram2y

What point are you trying to make? I'm not sure how that relates to what I was trying to illustrate with the weather example. Assuming for the moment that you didn't understand my point. The "game" I was referring to was one where it's literally all-or-nothing "predict the weather a year from now", you get no extra points for tomorrow's weather. This might be artificial but I chose it because it's a common example of the interesting fact that chaos can be easier to control than simulate. Another example. You're trying to win an election and "plan long-term to make the best use of your intelligence advantage", you need to plan and predict a year ahead. Intelligence doesn't give you a big advantage in predicting tomorrow's polls given today's polls. I can do that reasonably well, too. In this contest, resources and information might matter a lot more than intelligence. Of course, you can use intelligence to obtain information and resources. But this bootstrapping takes time and it's hard to tell how much depending where you start off.

[-][anonymous]2y169

"China hasn't made a better LLM than OpenAI" does not imply "China can't make a better LLM despite having more money". China isn't allocating all their money into this. If it's the case that China set a much bigger budget to developing LLMs than OpenAI had, and failed because OpenAI has better people, that would support your point about large resource mismatches not being able to overcome small intelligence gaps.

[-]simplegeometry1y*9219

This is something lc and gwern discussed in the comments here, but now we have clear evidence this is only true for Nash solvers (all typical engines like SF, Lc0, etc.). LeelaQueenOdds, which trained exploitatively against a model of top human players (FM+), is around 2k to 2.9k lichess elo depending on the time controls, so it completely trounces 1.6k elo players (especially 1.2k elo players as another commenter has suggested the author actually is). See: https://marcogio9.github.io/LeelaQueenOdds-Leaderboard/

Nash solvers are far too conservative and expect perfect play out of their opponents, hence give up most meaningful attacking chances in odds games. Exploitative models like LQO instead assume their opponents play like strong humans (good but imperfect) and do extremely well, despite a completely crushing material disadvantage. As some have noted, this is possible even with chess being a super sterile/simple environment relative to real life.

I speculate that the experiment from this post only yielded the results it did because Nash is a poor solution concept when one side is hopelessly disadvantaged under optimal play from both sides, and queen odds fall deep into that categ... (read more)

[-]Olli Järviniemi1y250

I found it interesting to play against LeelaQueenOdds. My experiences:

I got absolutely crushed on 1+1 time controls (took me 50+ games to win one), but I'm competitive at 3+2 if I play seriously.
The model is really good at exploiting human blind spots and playing aggressively. I could feel it striking in my weak spots, but not being able to do much about it. (I now better acknowledge the existence of adversarial attacks for humans on a gut level.)
I found it really addictive to play against it: You know the trick that casinos use, where they make you feel like you "almost" won? This was that: I constantly felt like I could have won, if it wasn't just for that one silly mistake - despite having lost the previous ten games to such "random mistakes", too... I now better understand what it's like to be a gambling addict.

Overall fascinating to play from a position that should be an easy win, but getting crushed by an opponent that Just Plays Better than I do.

[For context, I'm around 2100 in Lichess on short time controls (bullet/blitz). I also won against Stockfish 16 at rook odds on my first try - it's really not optimized for this sort of thing.]

9Lorenzo9mo

A grandmaster just lost a classical game (60''+30'') against Leela Knight Odds https://lichess.org/broadcast/leela-knight-odds-vs-gm-joel-benjamin/game-5/MbKHEbdb/7Tnz8uBj 3 days ago an international master gave Leela "very slim chances" of winning a game, based on the results of a match played by a previous version of the engine

1simplegeometry9mo

Thanks for this update! I find that an odd prediction by the IM because Awonder is around 2670 FIDE and Joel is around 2470 FIDE, 200 elo is huge.

1Lorenzo9mo

I think it's because 10+5 is very different from 60+30

1simplegeometry9mo

Oh, my bad, yeah. When I was writing the comment, I flipped the direction of advantage for longer time controls (longer time controls are actually better for humans in odds matches of course), but this way I agree it's unclear a priori whether 200 elo drop would be enough to account for longer time controls.

6cata1y

I was playing this bot lately myself and one thing it made me wonder is, how much better would it be at beating me if it was trained against a model of me in particular, rather than how it actually was trained? I feel I have no idea.

5Thomas Kwa11mo

Maybe we'll see the Go version of Leela give nine stones to pros soon? Or 20 stones to normal players?

[-]JenniferRM2y*619

I've been having various conversations in private, where I'm quite doomist and my interlocutor is less doomist, and I think one of the key cruxes that has come up several times is that I've applied security mindset to the operation of human governance, and I am not impressed.

I looked at things like the federal reserve (and how you'd implement that in a smart contract) and the congress/president/court deal (and how you'd implement that in a smart contract) and various other systems, and the thing I found was that existing governance systems are very poorly designed and probably relatively easy to knock over.

As near as I can tell, the reason human civilization still exists is that no inhuman opponent has ever existed that might really just want to push human civilization over and then curb stomp us while we thrash around in surprised pain.

For example, in WW2 Operation Bernhard got close to just "ending the money game" explicitly, but the bad guys couldn't bring themselves to make the stupidest and most evil British people rich via relatively secret injections, and then ramp it up more and more, and then as the whole web of market relationships became less and less plausible they coul... (read more)

[-]miss me mimo!2y2817

The Operation Bernhard example seems particularly weak to me, thinking for 30 seconds you can come up with practical solutions for this situation even if you imagine Nazi Germany having perfect competency in pulling off their scheme.

For example, using tax records and bank records to roll back peoples fortunes a couple of years and then introducing a much more secure bank note. It's not like WW2 was an era of fiscal conservatism, war powers were leveraged heavily by the federal reserve in the united states to do whatever they wanted with currency. We comfortably operate in a fiat currency regime where currency is artificially scarce and can be manipulated in half a dozen ways at the drop of a hat.

The way you interpret Operation Bernhard seems to me like you imagine the rules of society as something we set up and then are bound to like lemmings. When in reality, the rules can be rewritten at any time when the need arises. I think your example is equivalent to saying the ability to turn lead into gold would destroy the gold-standard era economy and utterly wreck civilization. When we know in hindsight we can just wave our finger and decouple currency and gold at a moments notice.

I suspect many of the other rules and systems that hold our civilization are just as adaptable when the need arises.

[-]RolfAndreassen2y159

The Wiki link on Operation Bernhard does not very obviously support the assertions you make about the Germans flinching. Do you have a different source in mind?

2JenniferRM2y

I cannot quickly find a clean "smoking gun" source nor well summarized defense of exactly my thesis by someone else. (Neither Google nor the Internet seem to be as good as they used to be, so I no longer take "can't find it on the Internet with Google" as particularly strong evidence that no one else has had the idea and tested and explored it in a high quality way that I can find and rely on if it exists.) ...in place of a link, I wrote 2377 more words than this, talking about the quality of the evidence I could find and remember, and how I process it, and which larger theories of economics and evolution I connect to the idea that human governance capacity is an evolved survival trait of humans, and our form of governments rely on it for their shape to be at all stable or helpful, and this "neuro-emotional" trait will probably not be reliably installed in AI, but also the AI will be able to attack anthropological preconditions of it, if that is deemed likely to get an AI more of what that AI wants, as AI replaces humans as the Apex Predator of Earth. It doesn't totally seem prudent to publish all 2377 words, now that I'm looking at them? Publishing is mostly irreversible, and I don't think that "hours matter" (and also probably even "days matter" is false) so I want to sit on them for a bit before committing to being in a future where those words have been published... Is there a big abstract reason you want a specific source for that specific part of it? I don't see that example as particularly central, just as a proposal that anyone can use as a springboard (that isn't "proliferative" to talk about in public because it is already in Wikipedia and hence probably cognitively accessible to all RLLLMs already) where the example: (1) is real and functions as a proof-by-existence of that class of "planning-capacity attacking ideas" being non-empty in a non-fictive context, (2) while mostly emotionally establishing that "at least some of the class of tactics is

0Portia2y

Please don't share human civilisation vulnerabilities online because a super awesome AI will get them anyway and human society might fortify against them. The chance of them fortifying is slim. Our politicians are failing to deal with right wing take-overs and climate change already. Our political systems hackability has already been painfully played by Russia, with little consequence. Literal bees have an electoral process for new hive locations more resilient against propaganda and fake news than we do, it is honestly embarrassing. The chance of a human actor exploiting such holes is larger than them being patched, I fear. The aversion to ruining your neighbouring countries financial system out of fear that they will ruin yours in response doesn't just not hold for an AI, it also fails to hold for those ideologically against a working world finance system. If you are willing to doom your own community, or fail to recognise that such a move would bring your own community doom, as well, because you have mistaken the legitimate evils of capitalism for evidence that we'd all be much better off if there was no such thing as money, you may well engage in such acts. There are increasing niche groups who think having humanity is per se bad, government is per se bad, and economy is per se bad. I think the main limit here so far is that the kind of actor who would like to not have a world financial system is typically not the kind of actor with sufficient money and networking to start a large-scale money forging operation. But not every massively destructive act requires a lot of resources to pull off.

2JenniferRM7mo

I agree that there are many bad humans. I agree that some of them are ideologically committed to destroying the capacity of our species to coordinate. I agree that most governance systems on Earth are embarrassingly worse than how bees instinctively vote on new hive locations. I do not agree that we should be quiet about the need for a global institutional governance system that has fewer flaws. By way of example: I don't think that "not talking very much about Gain-of-Function research deserving to be banned" didn't cause there to be no Gain-of-Function research in Wuhan, by collaborators of the people in the US who explicitly proposed building something like covid in a grant proposal a while before covid was actually built under BSL2 conditions, by their international scientific collaborators, and then escaped the lab. There should have been more anti-GoF talk, and it should have been explicitly bipartisan, and so on. In Trump's first term, one of the crazy random things he "did or allowed" was to let the pro-GoF people at NIH quietly weaken the GoF ban that was instituted under Obama. But also, similarly to how anti-GoF talk would be helpful up until there is an international treaty system that insists that GoF never happen outside of a "BSL5" (which currently doesn't even exist (currently the bio-safety levels only go up to 4)) I think there should be more anti-bad-governance-institution talk, and it should be explicitly bipartisan. There are many other larger fires now. And covid is no longer in the zeitgeist. Maybe this is not the best place to spend words. But it is a great test case for talking about general policies on regulation about dangerous technology, and institutions for handling such tech, and speech about the need for better institutions. Not that I greatly expect such talk to help, whether for AGI or GoF or anything. Its just that I think that (1) in the (rare?) timelines where we live I will not be greatly embarrassed to have talked as much

[-]Max H2y4419

If you're smarter than your opponent but have less starting resources, the optimal strategy probably involves some combination of cooperation, making alliances, deception, escaping / running / hiding, gathering resources in secret, and whatever other prerequisites are needed to neutralize such a resource imbalance. Many scenarios in which a smarter-than-human AGI with less resources goes to war with or is attacked by humanity are thus somewhat contradictory or at least implausible: they postulate the AGI taking a less good strategy than what a literal human in its place could come up with.

There's not really an analogue for this to Chess - if I am forced to play a chess game with a grandmaster with whatever handicap, I could maybe flip over the board if I started to lose. But that probably just counts as a forfeit, unless I can also overpower or coerce my opponent and / or the judges.

if a rogue AI is caught early on it’s plot, with all the worlds militaries combined against them while they still have to rely on humans for electricity and physical computing servers. It’s somewhat hard to outthink a missile headed for your server farm at 800 km/h.

Breaking it down by cases:

If de

... (read more)

6Nathan Helm-Burger2y

I like this analysis, and I agree with except that I do think it's missing a likely intermediate scenario. I think the "fully under lab control" is a super advantageous situation for the humans, especially if the AI has been trained on censored simulation data that doesn't mention humans or computers or have accurate physics. I think the current world has an unfortunately dangerous intermediate situation where LLMs age given full access to human knowledge, and allowed to interact with society. And yet, in the case of the SotA models like GPT-4, aren't quite at "loose in the world" levels of freedom. They don't have access to their own weights or source code and neither do any accomplices they might recruit outside the company. Indeed, even most employees at the company couldn't exfiltrate the weights. Thus, the current default starting state for a rogue AI is posed right on that dangerous margin of "difficult but not impossible to escape". I think this "brains vs brawn" style analysis does then make a big difference for the initial escape. I agree that once the escape has been accomplished it's really hard for humanity to claw back a win. But before the escape has occurred, it's a much more even game.

4MichaelStJules2y

Why is it too late if it would take militaries to stop it? Couldn't the militaries stop it?

5Max H2y

If an AI is smart enough that it takes a military force to stop it, the AI is probably also smart enough to avoid antagonizing that force, and / or hiding out in a way that a military can't find. Also, there are a lot of things that militaries and governments could do, if they had the will and ability to coordinate with each other effectively. What they would do is a different question. How many governments, when faced with even ironclad evidence of a rogue AI on the loose, would actually choose to intervene, and then do so in an effective way? My prediction is that many countries would find reasons or rationalizations not to take action at all, while others would get mired in disagreement and infighting, or fail to deploy their forces in an actually effective way. And that's before the AI itself has an opportunity to sow discord and / or form alliances. (Though again, I still think an AI that is at exactly the level where military power is relevant is a pretty narrow and unlikely band.)

[-]habryka2yΩ153425

This kind of experiment has been at the top of my list of "alignment research experiments I wish someone would run". I think the chess environment is one of the least interesting environments (compared to e.g. Go or Starcraft), but it does seem like a good place to start. Thank you so much for doing these experiments!

I do also think Gwern's concern about chess engines not really being trained on games with material advantage is an issue here. I expect a proper study of this kind of problem to involve at least finetuning engines.

[-]lc2y*5531

I do also think Gwern's concern about chess engines not really being trained on games with material advantage is an issue here. I expect a proper study of this kind of problem to involve at least finetuning engines.

It's actually much worse than this. Stockfish has no ability to model its opponents' flaws in game knowledge or strategy; it has no idea it's playing against a 1200. It's like a takeover AI that refrains from sending the stage-one nanosystem spec to the bio lab because it assumes the lab is also manned by AGIs and would understand what mixing the beaker accomplishes. A grandmaster in chess, who wanted to win against a novice with odds, would perhaps do things like complicate the position so that their opponent would have a larger chance of making blunders. Stockfish on the other hand is limited to playing "game theory optimal" chess, strategies that would work "best" (in terms of number of moves from checkmate saved) against what it considers optimal play.

To fix this, I have wondered for a while if you couldn't use the enormous online chess datasets to create an "exploitative/elo-aware" Stockfish, which had a superhuman ability to trick/trap players during handicappe... (read more)

[-]gwern2y224

Yes, this is another reason that setups like OP are lower-bounds. Stockfish, like most game RL AIs, is trying to play the Nash equilibrium move, not the maximally-exploitative move against the current player; it will punish the player for any deviations from Nash, but it will not itself risk deviating from Nash in the hopes of tempting the player into an even larger error, because it assumes that it is playing against something as good or better than itself, and such a deviation will merely be replied to with a Nash move & be very bad.

You could frame it as an imitation-learning problem like Maia. But also train directly: Stockfish could be trained with a mixture of opponents and at scale, should learn to observe the board state (I don't know if it needs the history per se, since just the stage of game + current margin of victory ought to encode the Elo difference and may be a sufficient statistic for Elo), infer enemy playing strength, and calibrate play appropriately when doing tree search & predicting enemy response. Silver & Veness 2010 comes to mind as an example of how you'd do MCTS with this sort of hidden-information (the enemy's unknown Elo strength) which turns it into a POMDP rather than a MDP.

[-]johnlawrenceaspden2y200

For a clear example of this, in endgames where I have a winning position but have little to no idea how to win, Stockfish's king will often head for the hills, in order to delay the coming mate as long as theoretically possible.

Making my win very easy because the computer's king isn't around to help out in defence.

This is not a theoretical difficulty! It makes it very difficult to practise endgames against the computer.

[-]Bucky2y144

Something similar not involving AIs is where chess grandmasters do rating climbs with handicaps. one I know of was Aman Hambleton managing to reach 2100 Elo on chess.com when he deliberately sacrificed his Queen for a pawn on the third/fourth move of every game.

https://youtube.com/playlist?list=PLUjxDD7HNNTj4NpheA5hLAQLvEZYTkuz5

He had to complicate positions, defend strongly, refuse to trade and rely on time pressure to win.

The games weren’t quite the same as Queen odds as he got a pawn for the Queen and usually displaced the opponent’s king to f3/f6 and prevented castling but still gives an idea that probably most amateurs couldn’t beat a grandmaster at Queen odds even if they can beat stockfish. Longer time controls would also help the amateur so maybe in 15 minute games an 1800 could beat Aman up a Queen.

3Portia2y

This has me wonder about a related point. I'm not a well-trained martial artist at all. But I have beaten well-trained martial artists in multiple fights. Apparently, that is not an unheard of phenomenon, either. It seemed to be key that I fight well by some metrics, but as a novice, commit errors that are incomprehensible, uneven and importantly: unpredictable to an expert because they would never do something so silly. I fail to go for obvious openings, and hence end up in unexpected places; but at that point, while I am underestimated because I have been foolish, I suddenly twist out of a grasp with unexpected flexibility, then miss being grabbed again because I have moved randomly and pointlessly, fail to protect against obvious threats, but don't drop due to an unexpectedly high pain tolerance despite having taken a severe hit, and then take a well-aimed hit with unexpected strength. This has me wonder whether an AI would have significant difficulties winning against humans who act inconsistently and suboptimally in some ways, without acting like utter idiots randomly all the time - because they don't take offers the AI was certain they would take, fail to defend against threats the AI was certain they would spot and that were actually traps, stubbornly stick with a strategy even after it has proven defective but hence cannot be budged from it even when the AI really needs them to, etc. Yet I also wonder whether the chess example is misleading because it is so inherently limited, so very inside the box. To go back to the above fight example: I've armwrestled with much stronger people I have beaten in actual fights. If they are much stronger, I inevitably lose the armwrestling. I am just not strong enough, and while I can set my arm with determination until the muscle rips... well, eventually the muscle just rips, and that is that. If I were to use my whole body for leverage like I would in a fight, or chuck something in their eyes to distract them, I woul

3Dweomite2y

I suspect that the domain of martial arts is unusually susceptible to that problem because 1. Fights happen so quickly (relative to human thought) that lots of decisions need to be made on reflex 1. (And this is highly relevant to performance because the correct action is heavily dependent on your opponent's very recent actions) 2. Most well-trained martial artists were trained on data that is heavily skewed towards formally-trained opponents

1green_leaf2y

That couldn't be the case - that would leave you, even after having a black belt, vulnerable towards people who can't fight, which would defeat the purpose of martial arts. Whichever technique you use, you use when responding to what the other person is currently doing. You don't simply execute a technique that depends on the person fighting well by some metrics, and then get defeated when it turns out that they are, in fact, only in the 0.001st percentile of fighting well by any metrics we can imagine. (That said, I'm really happy for your victories - maybe they weren't quite as well-trained.) I'm thinking the AI would predict the way in which the other person would act inconsistently and suboptimally. If there were multiple paths to victory for the human and the AI could block only one (thereby seemingly giving the human the option to out-random the AI by picking one of the unguarded paths to victory), the AI would be better at predicting the human than the human would be at randomizing. People are terrible at being unpredictable. I remember a 10+ years-old predictor of a rock-paper-scissors for predicting a "random" decision of a human in a series of games. The humans had no chance.

6johnlawrenceaspden2y

The "purpose" of most martial arts is to defeat other martial artists of roughly the same skill level, within the rules of the given martial art. Optimizing for that is not the same as optimizing for general fighting. If you spent your time on the latter, you'd be less good at the former. "Beginner's luck" is a thing in almost all games. It's usually what happens when someone tries a strategy so weird that the better player doesn't immediately understand what's going on. The other day a low-rated chess player did something so weird in his opening that I didn't see the threat, and he managed to take one of my rooks. That particular trap won't work on me again, and might not have worked the first time if I'd been playing someone I was more wary of. I did eventually manage to recover and win, but it was very close, very fun, and I shook his hand wholeheartedly afterwards. Every other game we've played I've just crushed him without effort. About a year ago I lost in five moves to someone who tried the "Patzer Attack". Which wouldn't work on most beginners. The first time I'd ever seen it. It worked once. It will never work on me again.

3gwd2y

Not only skill level, but usually physical capability level (as proxied by weight and sex) as well. As an aside, although I'm not at all knowledgeable about martial arts or MMA, it always seemed like an interesting thing to do might to use some sort of an ELO system for fighting as well: a really good lightweight might end up fighting a mediocre heavyweight, and the overall winner for a year might be the person in a given <skill, weight, sex> class that had the highest ELO. The only real reason to limit the ELO gap between contestants would be if there were a higher risk of injury, or the resulting fight were consistently just boring. But if GGP is right that a big upset isn't unheard of, it might be worth 9 boring fights for 1 exciting upset.

2green_leaf2y

This is false - the reason they were created was self-defense. That you can have people of similar weight and belt color spar/fight each other in contests is only a side effect of that. That doesn't work in chess if the difference in skill is large enough - if it did, anyone could simply make up n strategies weird enough, and without any skill, win any title or even the World Chess Championship (where n is the number of victories needed). If you're saying it works as a matter of random fluctuations - i.e. a player without skill could win, let's say, 0.5% games against Magnus Carlsen, because these strategies (supposedly) usually almost never work but sometimes they do, that wouldn't be useful against an AI, because it would still almost certainly win (or, more realistically, I think, simply model us well enough to know when we'd try the weird strategy).

3Portia2y

"Even after having a black belt"? One of the people I beat is a twice national champion, instructor with a very reputable agency and san dan in karate. They are seriously impressive good at it. If we agreed to do something predictable, I would be crushed. They are faster, stronger, have better form and balance, know more moves, have better reflexes. I'm in awe of them. They are good. I do think what they do deserves to be called an art, and that they are much, much, much (!) better than I am. But their actions also presuppose that I will act sensibly (e.g. avoiding injury, using opportunities), and within the rule set in which they were trained. I really don't think I could replicate this feat in the exact same way. Having once lost in such a bizarre way, they have learned and adapted. Many beginners only have few moves available, and suck at suppressing their intentions, so they may beat you once, but you'll destroy them if they try the same trick again. It might work again if they try something new, but again, if you paired the experienced fighter with that specific beginner for a while, pretty quickly, they would constantly win, as they have learned about the unexpected factor. But in a first fight? I wouldn't bet on a beginner in such a fight. But nor would I be that surprised by a win. And I definitely would not believe that having a black belt makes you invulnerable towards streetfighters, or even simply angry incompetent strangers, without one. Nor do I know any martial art trainer who would make such a claim. Safer, for sure. Your punches and kicks more effective, your balance and falls better, better confidence and situational awareness, more strength, faster reflexes, ingrained good responses rather than rookie mistakes, a knowledge of weak body parts, pain trigger points and ways to twist the other person to induce severe pain, knowledge of redirecting strength, of mobilising multiple body parts of yours against one of theirs, all the great stuff. But

[-]Quadratic Reciprocity2y2617

Is your "alignment research experiments I wish someone would run" list shareable :)

9Archimedes1y

@gwern and @lc are right. Stockfish is terrible at odds and this post could really use some follow-up. As @simplegeometry points out in the comments, we now have much stronger odds-playing engines that regularly win against much stronger players than OP. https://lichess.org/@/LeelaQueenOdds https://marcogio9.github.io/LeelaQueenOdds-Leaderboard/

2habryka1y

That's really cool! Do you have any sense of what kind of material advantage these odd-playing engines could use against the best humans?

[-]Archimedes8mo220

FYI, there has been even further progress with Leela odds nets. Here are some recent quotes from GM Larry Kaufman (a.k.a. Hissha) found on the Leela Chess Zero Discord:

(2025-03-04) I completed an analysis of how the Leela odds nets have performed on LiChess since the search-contempt upgrade on Feb. 27. [...] I believe these are reasonable estimates of the LiChess Blitz rating needed to break even with the bots at 5'3" in serious play. Queen and move odds (means Leela plays Black) 2400, Queen odds (Leela White) 2550, [...] Rook and move odds (Leela Black); 3000. Rook odds (Leela White) 3050, knight odds 3200. For comparison only a few top humans exceed 3000, with Magnus at 3131. So based on this, even Magnus would lose a match at 5'3" with knight odds, while perhaps the top five blitz players in the world would win a match at rook odds. Maybe about top fifty could win a match at queen for knight. At queen odds (Leela White), a "par" (FIDE 2400) IM should come out ahead, while a "par" (FIDE 2300) FM should come out behind.

(2025-03-07) Yes, there have to be limits to what is possible, but we keep blowing by what we thought those limits were! A decade ago, blitz games (3'2") were prett

... (read more)

6lc1y

As the name suggests, Leela Queen Odds is trained specifically to play without a queen, which is of course an absolutely bonkers disadvantage against 2k+ elo players. One interesting wrinkle is the time constraint. AIs are better at fast chess (obviously), and apparently no one who's tried is yet able to beat it consistently at 3+0 (3 minutes with no timing increment)

3simplegeometry1y

At rapid time controls, it seems like we could maybe go even against Magnus with knight odds? If not Magnus, perhaps other high-rated GMs. There was a match with the most recently updated LeelaKnightOdds and GM Alex Lenderman but I don't recall the score exactly. EDIT: which was 19-3-2 win draw loss.

1[anonymous]1y

I am very skeptical of this on priors, for the record. I think this statement could be true for superblitz time controls and whatnot, but I would be shocked if knight odds would be enough to beat Magnus in a 10+0 or 15+0 game. That being said, I have no inside knowledge, and I would update a lot of my beliefs significantly if your statement as currently written actually ends up being true.

2Veedrac10mo

LeelaKnightOdds has convincingly beaten both Awonder Liang and Anish Giri at 3+2 by large margins, and has an extremely strong record at 5+3 against people who have challenged it. I think 15+0 and probably also 10+0 would be a relatively easy win for Magnus based on Awonder, a ~150 elo weaker player, taking two draws at 8+3 and a win and a draw at 10+5. At 5+3 I'm not sure because we have so little data at winnable time controls, but wouldn't expect an easy win for either player. It's also certainly not the case that these few-months-old networks running a somewhat improper algorithm are the best we could build—it's known at minimum that this Leela is tactically weaker than normal and can drop endgame wins, even if humans rarely capitalize on that.

1simplegeometry1y

Hissha from the Lc0 server reports 19 wins, 3 draws, and 2 losses against Lenderman (currently ~2500 FIDE) at 15+10 from a knight odds match 2 months ago -- with the caveat that Lenderman started playing too fast after 10 games. I haven't run the numbers but suspect this would be enough to go even against a 2750, if not Magnus? I was surprised too. I think it's an exciting development :)

3[anonymous]1y

Hmm, that sounds about right based on the usual human-vs-human transfer from Elo difference to performance... but I am still not sure if that holds up when you have odds games, which feel qualitatively different to me than regular games. Based on my current chess intuition, I would expect the ability to win odds games to scale better than ELO near the top level, but I could be wrong about this.

1Archimedes1y

Knight odds is pretty challenging even for grandmasters.

[-]Zach Stein-Perlman2y*2511

Some nitpicks:

You write like Stockfish 14 is a probabilistic function from game-state to next-move, the thing-which-has-an-ELO. But I think Stockfish 14 running on X hardware for Y time is the real probabilistic function from game-state to next-move (see e.g. the inclusion of hardware in ELO ranking here). And you probably played with hardware and time such that its ELO is substantially below 3549.
I think a human with Stockfish's ELO would be much better at beating you down odds of a queen, since (not certain about these):
- Stockfish is optimized for standard chess and human grandmasters are probably better at transferring to odds-chess.
- Stockfish roughly tries to maximize P(win) against optimal play or Stockfish-level play, or maximize number of moves before losing once it knows you have a winning strategy. Human grandmasters would adapt to be better against your skill level (e.g. by trying to make positions more complex), and would sometimes correctly make choices that would be bad against Stockfish or optimal play but good against weaker players.

[-]Olli Järviniemi11mo181Review for 2023 Review

The post studies handicapped chess as a domain to study how player capability and starting position affect win probabilities. From the conclusion:

In the view of Miles and others, the initially gargantuan resource imbalance between the AI and humanity doesn’t matter, because the AGI is so super-duper smart, it will be able to come up with the “perfect” plan to overcome any resource imbalance, like a GM playing against a little kid that doesn't understand the rules very well.
The problem with this argument is that you can use the exact same reasoning to imply that’s it’s “obvious” that Stockfish could reliably beat me with queen odds. But we know now that that’s not true.

Since this post came out, a chess bot (LeelaQueenOdds) that has been designed to play with fewer pieces has come out. simplegeometry's comment introduces it well. With queen odds, LQO is way better than Stockfish, which has not been designed for it. Consequentially, the main empirical result of the post is severely undermined. (I wonder how far even LQO is from truly optimal play against humans.)

(This is in addition to - as is pointed out by many commenters - how the whole analogue is stretched at best, gi... (read more)

4Martin Randall9mo

While I agree that this post was incorrect, I am fond of it, because the resulting conversation made a correct prediction that LeelaPieceOdds was possible. Most clearly in a thread started by lc: (not quite a prediction as phrased, but I still infer a prediction overall). Interestingly there were two reasons given for predicting that Stockfish is far from optimal when giving Queen odds to a less skilled player: * Stockfish is not trained on positions where it begins down a queen (out-of-distribution) * Stockfish is trained to play the Nash equilibrium move, not to exploit weaker play (non-exploiting) The discussion didn't make clear predictions about which factor would be most important, or whether both would be required, or whether it's more complicated than that. Folks who don't yet know might make a prediction before reading on. For what it's worth, my prediction was that non-exploiting play is more important. That's mostly based on a weak intuition that starting without a queen isn't that far out of distribution, and neural networks generalize well. Another way of putting it: I predicted that Stockfish was optimizing the wrong thing more than it was too dumb to optimize. And the result? Alas, not very clear to me. My research is from the the lc0 blog, with posts such as The LeelaPieceOdds Challenge: What does it take you to win against Leela?. The journey began with the "contempt" setting, which I understand as expecting worse opponent moves. This allows reasonable opening play and avoids forced piece exchanges. However GM-beating play was unlocked with a fine-tuned odds-play-network, which impacts both out-of-distribution and non-exploiting concerns. One surprise gives me more respect for the out-of-distribution theory. The developer's blog first mentioned piece odds in The Lc0 v0.30.0 WDL rescale/contempt implementation So missing a single bishop is in some sense further out-of-distribution than missing a rook and a knight! The later blog I linked ex

[-]Ege Erdil2y1814

I'm surprised by how much this post is getting upvoted. It gives us essentially zero information about any question of importance, for reasons that have already been properly explained by other commenters:

Chess is not like the real world in important respects. What the threshold is for material advantage such that a 1200 elo player could beat Stockfish at chess tells us basically nothing about what the threshold is for humans, either individually or collectively, to beat an AGI in some real-world confrontation. This point is so trivial that I feel somewhat embarrassed to be making it, but I have to think that people are just not getting the message here.
Even focusing only on chess, the argument here is remarkably weak because Stockfish is not a system trained to beat weaker opponents with piece odds. There are Go AIs that have been trained for this kind of thing, e.g. KataGo can play reasonably well in positions with a handicap if you tell it that its opponent is much weaker than itself. In my experience, KataGo running on consumer hardware can give the best players in the world 3-4 stones and have an even game.

If someone could try to convince me that this experiment was not pointless and actually worth running for some reason, I would be interested to hear their arguments. Note that I'm more sympathetic to "this kind of experiment could be valuable if ran in the right environment", and my skepticism is specifically about running it for chess.

[-]polytope2y145

(I'm the main KataGo dev/researcher)

Just some notes about KataGo - the degree to which KataGo has been trained to play well vs weaker players is relatively minor. The only notable thing KataGo does is in some self-play games to give up to an 8x advantage in how many playouts one side has over the other side, where each side knows this. (Also KataGo does initialize some games with handicap stones to make them in-distribution and/or adjust komi to make the game fair). So the strong side learns to prefer positions that elicit higher chance of mistakes by the weaker side, while the weak side learns to prefer simpler positions where shallower search doesn't harm things as much.

This method is cute because it adds pressure to only learn "general high-level strategies" for exploiting a compute advantage, instead of memorizing specific exploits (which one might hypothesize to be less likely to generalize to arbitrary opponents). Any specific winning exploit learned by the stronger side that works too well will be learned by the weaker side (it's the same neural net!) and subsequently will be avoided and stop working.

And it's interesting that "play for positions that a compute-limited yourse... (read more)

[-]habryka2y111

If someone could try to convince me that this experiment was not pointless and actually worth running for some reason, I would be interested to hear their arguments. Note that I'm more sympathetic to "this kind of experiment could be valuable if ran in the right environment", and my skepticism is specifically about running it for chess.

I've been interested in the study of this question for a while. I agree this post has the flaws you point out, but I still find that it provides interesting evidence. If the result had been that Stockfish would have continued to win even with overwhelming material disadvantage, then this of course would have updated me some. I agree the current result is kind of close to the null result, but that's fine. Also, it is much cheaper to run than almost all the other experiments in this space, and it's good to encourage people to get started at all, even if it's going to be somewhat streetlighty.

5MichaelStJules2y

I think it's more illustrative than anything, and a response to Robert Miles using chess against Magnus Carlsen as an analogy for humans vs AGI. The point is that a large enough material advantage can help someone win against a far smarter opponent. Somewhat more generally, I think arguments for AI risk often put intelligence on a pedestal, without addressing its limitations, including the physical resource disadvantages AGIs will plausibly face. I agree that the specifics of chess probably aren't that helpful for informing AI risk estimates, and that a better tuned engine could have done better against the author. Maybe better experiments to run would be playing real-time strategy games against a far smarter but materially disadvatanged AI, but this would also limit the space of actions an AI could take relative to the real world.

[-]Kei Nishimura-Gasparian2y*1812

While I think your overall point is very reasonable, I don't think your experiments provide much evidence for it. Stockfish generally is trained to play the best move assuming its opponent is playing best moves itself. This is a good strategy when both sides start with the same amount of pieces, but falls apart when you do odds games.

Generally the strategy to win against a weaker opponent in odds games is to conserve material, complicate the position, and play for tricks - go for moves which may not be amazing objectively but end up winning material against a less perceptive opponent. While Stockfish is not great at this, top human chess players can be very good at it. For example, a top grandmaster Hikaru Nakamura had a "Botez Gambit Speedrun" (https://www.youtube.com/playlist?list=PL4KCWZ5Ti2H7HT0p1hXlnr9OPxi1FjyC0), where he sacrificed his queen every game and was able to get to 2500 on chess.com, the level of many chess masters.

This isn't quite the same as your queen odds setup (it is easier), and the short time format he is on is a factor, but I assume he would be able to beat most sub-1500 FIDE players with queen odds. A version of Stockfish trained to exploit a human's subpar ability would presumably do even better.

[-]Stephen McAleese2y100

Thanks for the post! It was a good read. One point I don't think was brought up is the fact that chess is turn-based whereas real life is continuous.

Consequently, the huge speed advantage that AIs have is not that useful in chess because the AI still has to wait for you to make a move before it can move.

But since real life is continuous, if the AI is much faster than you, it could make 1000 'moves' for every move you make and therefore speed is a much bigger advantage in real life.

[-]gbear6052y106

I'm not familiar with how Stockfish is trained, but does it have intentional training for how to play with queen odds? If not, then it might be able to start trouncing you if it were trained to play with it, instead of having to "figure out" new strategies uniquely.

1O O2y

Stockfish isn’t using deep learning afaik. It’s mostly just bruteforcing.

[-]gwern2y276

Stockfish now uses an interesting lightweight kind of NN called NNUE which does need to be trained; more importantly, chess engines have long used machine learning techniques (if not anything we would now call deep learning) which still need to be fit/trained and Stockfish relies very heavily on distributed testing to test/create changes, so if they are not playing with queen odds, then neural or no, it amounts to the same thing: it's been designed & hyperoptimized to play regular even-odds chess, not weird variants like queen-odd chess.

2MichaelStJules2y

Would queen-odds games pass through roughly within-distribution game states, anyway, though? Or, either way, if/when it does reach roughly within-distribution game states, the material advantage in relative terms will be much greater than just being down a queen early on, so the starting material advantage would still underestimate the real material advantage for a better trained AI.

5Awesome_Ruler_0072y

Its clear that it was never optimized for odds games, therefore unless concrete evidence is presented, I doubt that @titotal actually played against a "superhuman system - which may explain why it won. There's definitely a ceiling to which intelligence will help - as the other guy mentioned, not even AIXI would be able to recover from an adversarially designed initial position for Tic-Tac-Toe. But I'm highly skeptical OP has reached that ceiling for chess yet.

4[anonymous]2y

SF's ability to generalize across that distribution shift seems unclear. My intuition is that a starting position with queen odds is very off distribution because in training games where both players are very strong, large material imbalances only happen very late in the game. I'm confused by your 2nd paragraph. Do you think this experiment overestimates or underestimates resource gap required to overcome a given intelligence gap?

3MichaelStJules2y

For my 2nd paragraph, I meant that the experiment would underestimate the required resource gap. Being down exactly by a queen at the start of a game is not as bad as being down exactly by a queen later into the game when there are fewer pieces overall left, because that's a larger relative gap in resources.

[-]Huera2y95

(My current fide rating is ~1500 elo (~37 percentile) and my peak rating was ~1700 elo (~56 percentile)).

While I'm not that good at chess myself, I think you got some things wrong, and on some I'm just being nitpicky.

My rating on lichess blitz is 1200, on rapid is 1600, which some calculator online said would place me at ~1100 ELO on the FIDE scale.

I’m quite skeptical of such conversions, but I understand you had nothing better to go on. This website (made from surveying a bunch of redditors ^[1]) converts your lichess blitz rating into 1005, 869&... (read more)

[-]Garrett Baker2y70

Although realistically, the real odds would be less about the ELO and more on whether he was drunk while playing me.

[-]Ruby2y63

Curated. The question beneath feels really quite interesting. As the OP have said, even if it's the case that a vastly superhuman intelligent AI could defeat even at extreme disadvantage, this doesn't mean there isn't some advantage that would let humans defeat a more nascently powerful AGI, and it's pretty interesting to understand the how that works out. I'm excited to see more work on this, especially in domains resembling more and more real life* (e.g. Habryka suggests Starcraft).

*Something about chess is it feels quite "tight" in terms of not admitting exploits or hacks the way I could imagine other games have hidden exploitable bugs that can be mined – like reality.

[-]Lukas Finnveden2yΩ364

I intend to write a lot more on the potential “brains vs brawns” matchup of humans vs AGI. It’s a topic that has received surprisingly little depth from AI theorists.

I recommend checking out part 2 of Carl Shulman's Lunar Society podcast for content on how AGI could gather power and take over in practice.

[-]Archimedes2y50

Leela now has a contempt implementation that makes odds games much more interesting. See this Lc0 blog post (and the prior two) for more details on how it works and how to easily play odds games against Leela on Lichess using this feature.

GM Matthew Sadler also has some recent videos about using WDL contempt to find new opening ideas to maximize chances of winning versus a much weaker opponent.

I'd bet money you can't beat LeelaQueenOdds at anything close to a 90% win rate.

[-]Artaxerxes2y51

On the other hand, the potential resource imbalance could be ridiculously high, particularly if a rogue AI is caught early on it’s plot, with all the worlds militaries combined against them while they still have to rely on humans for electricity and physical computing servers. It’s somewhat hard to outthink a missile headed for your server farm at 800 km/h. ... I hope this little experiment at least explains why I don’t think the victory of brain over brawn is “obvious”. Intelligence counts for a lot, but it ain’t everything.

While this is a true and import... (read more)

[-]johnlawrenceaspden2y30

I think this is a great article, and the thesis is true.

The question is, how much intelligence is worth how much material?

Humans are so very slow and stupid compared to what is possible, and the world so complex and capable of surprising behaviour, that my intuition is that even a very modest intelligence advantage would be enough to win from almost any starting position.

You can bet your arse that any AI worthy of the name will act nice until it's already in a winning position.

I would.

1Aiyen2y

Even if we assume that's true (it seems reasonable, though less capable AIs might blunder on this point, whether by failing to understand the need to act nice, failing to understand how to act nice or believing themselves to be in a winning position before they actually are), what does an AI need to do to get in a winning position? And how easy is it to make those moves without them being seen as hostile? An unfriendly AI can sit on its server saying "I love mankind and want to serve it" all day long, and unless we have solid neural net interpretability or some future equivalent, we might never know it's lying. But not even superintelligence can take over the world just by saying "I love mankind". It needs some kind of lever. Maybe it can flash its message of love at just the right frequency to hack human minds, or to invoke some sort of physical effect that let's it move matter. But whether it can or not depends on facts about physics and psychology, and if that's not an option, it doesn't become an option just because it's a superintelligence trying it.

2johnlawrenceaspden2y

It does, and a superintelligence will understand those facts better than we do.

[-]Archimedes2y*31

If you're open to more experimentation, I'd recommend trying playing against Leela Chess Zero using some of the newer contempt parameters introduced in this PR and available in the latest pre-release version. I'm really curious if you'd notice significant style differences with different contempt settings.

Update: The official v0.30.0 release is out now and there is a blog post detailing the contempt settings. Additionally, there is a Lichess bot set up specifically for knight odds games.

Further update: There are now three Lichess bots set up to play odds g... (read more)

[-]Dweomite2y31

Probably not relevant to any arguments about AI doom, but some notes about chess material values:

You said a rook is "ostensibly only 1 point of material less than two bishops". This is true in the simplified system usually taught to new players (where pawn = 1, knight = bishop = 3, rook = 5, queen = 9). But in models that allow themselves a higher complexity budget, 2 bishops can be closer to a queen than a rook (at the start of the game):

Bishops are usually considered slightly better than knights; a value of 3 + 1/3 is typical
There is a "pair

... (read more)

[-]AnthonyC9mo20

I really enjoyed this piece, not because of the specific result, but because of the style of reasoning it represents. How much advantage, under what kind of rules, can be overcome with what level of intelligence?

Sometimes the answer is none. "I play x" overwhelms any level of intelligence at tic tac toe.

In larger and more open games the advantage of intelligence increases, because you can do more by being better at exploring the space of possible moves.

"Real life" is plausibly the largest and most open game, where the advantage of intelli... (read more)

[-]Going Durden1y2-1

A related thought: an intelligence can only work on the information that it has, regardless of its veracity, and it can only work on information that actually exists.

My hunch is that the plan of "AI boostraps itself to superintelligence, then superpower, then wipes out humanity" relies on it having access to information that is too well hidden to divine through sheer calculation and infogathering, regardless of its intelligence (ex: the location of all the military bunkers, and nuclear submarines humanity has), or simply does not exist (ex: future Human st... (read more)

[-]Ulisse Mini2y20

This might actually be a case where a chess GM would outperform an AI: they can think psychologically, so they can deliberately pick traps and positions that they know I would have difficulty with.

Emphasis needed. I expect a GM to beat you down a rook every time, and down a queen most times.

Stockfish assumes you will make optimal moves in planning and so plays defensive when down pieces, but an AI optimized to trick humans (i.e. allowing suboptimal play when humans are likely to make a mistake) would do far better. You could probably build this with ma... (read more)

[-]Donald Hobson2yΩ120

I think the assumptions that.

Humans realize the AI exists early on.
Humans are reasonably coordinated and working against the AI.

Are both dubious.

What is stopping someone sending a missile at GPT-4's servers right now.

OpenAI hasn't anounced a list of coordinated for where those servers are (as far as I know) This is because
OpenAI doesn't want you to missile strike their servers because
OpenAI thinks their AI is safe and useful not dangerous.

I think seeing large numbers of humans working in a coordinated fashion against an AI is unlikely.

[-]MondSemmel2y20

If a rogue AI is discovered early, we could end up in a war where the AGI has a huge intelligence advantage, but humans have a huge resource advantage.

In that scenario, it seems to me that enough abstractions break down that the analogy to the Stockfish experiment no longer works. Like talking about a conflict of AGI vs. "humans" as two agents in a 2-player game, rather than AGI vs. a collection of exploitable agents.

But I want to focus on the "resource" abstraction here. First of all, "ownership" of resources seems irrelevant; that's mostly a legal concep... (read more)

[-]followthesilence2y21

Enjoyed this post, thanks. Not sure how well chess handicapping translates to handicapping future AGI, but it is an interesting perspective to at least consider.

[-][anonymous]2y20

Thank you for doing the experiment. Someone could run a similar set of tests for Go.

Just to prime your thinking: what's war winning for most wars on earth?

Probably whoever can use the majority of physical resources and turn them into weapons. We had several rounds of wars and the winner had a vast material advantage.

It occurred to me that the level of AI capabilities needed to reach exponential growing levels of resources is essentially a general robot system, trained on all videos in existence of humans taking actions in the real world and a lot of rein... (read more)

[-]gwern2y*3920

Thank you for doing the experiment. Someone could run a similar set of tests for Go.

Go has an advantage here of much greater granularity in handicapping. Handicapping with pieces isn't used as much in chess as it is in Go because, well, there are so few pieces, on such a small board, for a game lasting so few moves, that each removed piece is both a large difference and changes the game qualitatively. I wouldn't want to study chess at all at this point as a RL testbed: there's better environments, which are cleaner to tweak, cheaper to run, more realistic/harder, have oracles, or something else; chess is best at nothing at this point (unless you are interested in chess or history of AI, of course).

Also, it's worth noting that these piece-disadvantage games are generally way out of distribution / off-policy for an agent like Stockfish: AFAIK, the Stockfish project (and all other chess engine projects, for that matter) does not spend a (or any?) meaningful amount of training on extreme handicap scenarios like 'what if I somehow started the game missing a knight' or 'what if my queen just wasn't there somehow' or 'somehow, Palpatine's piece returned'. (So there's a similar problem ... (read more)

4Dweomite2y

You're framing these as being closer to "regular" chess, but my intuition is the opposite. Most of the game positions that occur during a queen-odds game are rare but possible positions in a regular game; they are contained within the game tree of normal chess. I'm not sure about Stockfish in particular, but I'd expect many chess AIs incorporating machine learning would have non-zero experience with such positions (e.g. from early self-play runs when they were making lots of bad moves). Positions permitting self-capture do not appear anywhere in that game tree and typical chess AIs are guaranteed to have exactly zero experience of them. ETA: It also might affect your intuitions to remember that many positions Stockfish would never actually play will still show up in its tree search, requiring it to evaluate them at least accurately enough to know not to play them.

4gwern2y

I disagree. By starting with impossible positions like a queen already being missing*, the game is already far out of the superhuman-level chess-game distribution which is defined by Stockfish. Stockfish will never blunder in the early game so badly as to lose a queen in a normal early-game position, even if it was playing God. I expect these to be positions that the Stockfish policy will never reach, not even with its weakest play of zero tree search & following deterministic argmax move choice. The only time Stockfish would ever reach such positions is if forced to by some external force like a player fiddling with settings or a strange training setup, or, like, a cosmic ray flipping some bits on the CPU. There might be some such blunders very early on in training which takes it into such imbalanced very early positions, but those are still fairly different, and the final Stockfish is going to be millions (or at this point, billions) of games of training later and will have no idea of how to handle some positions that near-random play produced eons ago and long-since washed out. (After all, those will be the very stupidest and most incompetent games it ever played, so there is little value in holding onto them in any way. Most setups will erase old games pretty quickly, and certainly don't hold onto games from the start.) Whereas several of the changes Kramnik evaluated, like 'Forbidding castling within the first 10 moves' probably overlaps to quite a considerable degree; what fraction of chess games, human expert or Stockfish, involve no castling in the first 10 moves and so accidentally fulfill that rule? Probably a pretty good chunk! * even odds like knight-odds -where you can at least in theory construct the position during a game, by moving the knight out, capturing it with the other knight, and carefully moving the other knight back into its original position - have exactly zero probability of ever occurring in an on-policy game.

4gjm2y

Several? I can see one (the one you cite). Some of the other variants -- e.g., no castling at all, or pawns can't move two squares on their first move -- can lead to positions that also arise in normal chess. But having neither side castle at all is really unusual and most such positions will be well out of distribution; and it's very common for some pawns to remain on the second rank all the way to the endgame, where the option of moving one or two squares can have important timing implications.

4[anonymous]2y

What do you think about the other corollary? At the upper end of play the number of stones required for a worse agent to equal the best agent shrinks? And we could plot out compute vs skill and estimate the number of stones for a particular skill level to have a 50 percent win rate against an agent with infinite compute. (Infinite compute just means it has perfect moves as it can factor in all permutations. This is an experiment we can run for solvable games like checkers but we can estimate the asymtote for Go)

4gwern2y

I mean, it would have to. There's a ceiling there, while the game size and stones remain fixed units. If you have agents vastly below optimal play, they can differ a lot in absolute units, because they're not the ceiling, and differ greatly in strategy as well. But the closer you approach God (as players like to put it), the smaller the equalizing material advantage must be, approaching zero. There is only one game tree.

2[anonymous]2y

Does this imply convergence for rampant AGI systems? The "we are doomed" model assumes we will be defeated even when we have networks of superintelligent ASI systems restricted from hostile actions through essentially CAIS. This is where we subdivide large tasks into the smallest possible subtasks, define sparse schema to encode intermediate results, and have separate sessions of an ASI on each (subtask description, subtask context, intermediate schema from other results). Among other benefits this prevents most deception and collusion because the subtask context was a possible draw from the training set and the ASI has no memory or state, it can't know it's not still in training. (It's stateless in that ASI_output = f(f(network architecture, weights), task description, task context, environment input set, RNG seed). It's a functional system and on the next time step you can switch out the network architecture and weights if you wish for a different model with similar capabilities. ASI_output updates the context. Anyways such a network of systems will perform well but what you are throwing away is bits of context in between the steps. For example if the task is "make housing" one subtask might design the overall shape and visual appearance, another might be the structural design and engineering plans, another might be an inspection to look for mistakes. Yet other subtasks would actually build the structure. Each subtask is a fresh, context ignorant session and closes when a step is done with all memory erased. For example if constructing the building is subdividable into floors or individual girder attachments, those are separate subtasks. The same or different model can be assigned to any given subtask, they need not share any lineage and it makes sense to have the "inspection" subtasks done by a different lineage of base model. A single "context aware model" doing all steps benefits from having all of the bits of context for every step in theory (in practice it

2Charlie Steiner2y

I took this a different way: what's the correlation between resources and winning conflicts for humans on earth? Assuming the curve is the same as for chess, what elo does that place human conflicts at?

4[anonymous]2y

Depends. Depends on the communication technology of the era, training, quality of leaders, whether all the forces are under a single unified command, and so on. The main takeaway from this is not that. It's that increasing intelligence has diminishing returns. That a hypothetical "perfect policy" AI general, with an ELO equivalent to almost infinity, can be crushed by "humans with AI tools to help" with an ELO of say 5000 (1000 would be average human general) with a very small resource advantage. Say 30 percent more forces, or their forces are inferior in technology but they have 2-3 times as many. And a force disparity where humans with their 1000 ELO win is also possible. This is because of the nature of what intelligence is. Each bit of policy complexity over a random policy has diminishing returns. The highest yield policy is what you tend to find first "let's have all our forces get in a line so they won't hit each other and start blasting" and each improvement has smaller gains. (Or in chess, "let's put my higher value pieces in spots where a lower value piece cannot capture them on the very next move")

[-]Arjun Pitchanathan7mo*10

the real odds would be less about the ELO and more on whether he was drunk while playing me

not sure if that would help :)

[-]Sodium10mo1-3Review for 2023 Review

Perhaps I am missing something, but I do not understand the value of this post. Obviously you can beat something much smarter than you if you have more affordances than it does.

FWIW, I have read some of the discourse on the AI Boxing game. In contrast, I think those posts are valuable. They illustrate that even with very little affordances a much more intelligent entity can win against you, which is not super intuitive especially in the boxed context.

So the obvious question is, how does differences in affordances lead to differences in winning (i.e.,... (read more)

[-]Spade1y10

Anecdotally, I remember seeing analyses of Stockfish v. Alpha Zero (I think) where AlphaZero would fairly consistently trade absurd amounts of materiel for position. While there is obviously still a tipping point at which a materiel advantage will massively swing the odds I feel that the thrust of this essay kind-of understates the value of a lot-a lot of intelligence in light of those matches.

With that said, I haven't seen any odds-games with AlphaZero, so perhaps my point is entirely moot and it does need that initial materiel as badly as Stockfish.

[-]Review Bot1y*10

The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2024. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

[-]dspeyer2y1-1

I suspect this is a lack of flexibility in Stockfish. It was designed (trained?) for normal equal-forces chess and can't step back to think "How do I best work around this disadvantage I've been given?" I suspect something like AlphaZero, given time to play itself at a disadvantage, would do better. As would a true AGI.

[-]awenonian2y10

I have a habit of reading footnotes as soon as they are linked, and your footnote says that you won with queen odds before the call to guess what odds you'd win at, creating a minor spoiler.

[-]Stephen Richards2y10

I think this is a really useful and thought provoking experiment. One thing that worries me, is that large corporations may find it easier and faster to give the AI brawn than brains. Why play fair when in competition when you have a money and machine advantages? I think this will be especially so with not so good AIs, and the advantages will remain after the brains part improves. So in your analogy, what about giving stockfish 3 extra queens? A second question is how does it do against stockfish with just 2 extra queens?

[-]juvi2y10

It's maybe worth noting that Stockfish 14 NNUE still has some failure modes. Take this position for example: positionOnLichess. The position is a complete draw, as Black can't make any progress, and White cannot lose as long as he only moves his king. Despite this, Stockfish 14 NNUE evaluates it as a -15 advantage for Black, which should typically indicate a decisive advantage. Even a human player with relatively low Elo should be able to quickly assess this position as a draw.

[-]alenoach2y11

Thanks for the insights. Actually, board game models don't play very well when they are so heavily loosing, or so heavily winning that it doesn't seem to matter. A human player would try to trick you and hope for a mistake. This is not necessarily the case with these models that play as if you were as good as them, which makes their situation look unwinnable.

It's quite the same with AlphaGo. AlphaGo plays incredibly well until there is a large imbalance. Surprisingly, AlphaGo also doesn't care about winning by 10 points or by half a point, and someti... (read more)

[-]Chris Land2y10

A somewhat related point: it's only very recently (2023) that chess engines have begun competently mimicking the error patterns of human play. The nerfings of previous decades were all artificial.

I'm an FM and play casual games vs. the various nerfed engines at chess.com. The games are very fast (they move instantly) but there's no possibility of time loss. Not the best way to practice openings but good enough.

The implication for AI / AGI is that humans will never create human-similar AI. Everything we make will be way ahead in many areas and way behind in... (read more)

3[anonymous]2y

The implication for AI / AGI is that humans will never create human-similar AI. Everything we make will be way ahead in many areas and way behind in others Is this not a mere supervised learning problem? You're saying, for some problem domain D, you want to predict the probability distribution of actions a Real Human would emit when given a particular input sample. This is what a GPT is, it's doing something very close to this, by predicting, from the same input text string a human was using, what they are going to type next. We can extend this, to video, and obviously first translate video of humans to joint coordinates, and from sounds they emit back to phonemes, then do the same prediction as above. We would expect to get an AI system from this method that approximates the average human from the sample set we trained on. This system will be multimodal and able to speak, run robotics, and emit text. Now, after that, we train using reinforcement learning, and that feedback can clear out mistakes, so that the GPT system is now less and less likely to emit "next tokens" that the consensus for human knowledge believes is wrong. And the system never tires and the hardware never miscalculates. And we can then use machine based RL - have robots attempt tasks in sim and IRL, autonomously grade them on how well the task was done. Have the machine attempt to use software plugins, RL feedback on errors and successful tool usage. Because the machinery can learn on a larger scale due to having more time to learn than a human lifetime, it will soon exceed human performance. And we also have more breadth with a system like this than any single individual living human. But I think you can see how, if you wanted to, you could probably find a solution based on the above that emulates the observable outputs of a single typical human.

[-][anonymous]2y10

I predicted your odds of winning to be 50% with queen+rook odds, 1% with queen odds, 0.2% with 2 bishops odds, and 0.1% with rook odds. When you started describing strategies tailored to odds games that you were going to use, I felt cheated! I thought you were just going to play your normal 1100-rated game, but I made a big mistake. I forgot that you're a general intelligence, not a narrow, 1100-rated chess AI. Stockfish's NNUE was never trained on positions like the ones at the start of your odds games since they can't be reached from a normal 32-piece st... (read more)

[-]ws27b2y00

The problem is that true AGI is self-improving and that a strong enough intelligence will always either accrue the resource advantage or simply do much more with less. Chess engines like Stockfish do not serve as good analogies for AGI since they don't have those self-referential self-improvement capabilities that we would expect true AGI to have.

[-]Maybe_a2y00

Odds games against engine are played with contempt equal to matherial difference.

Sorry you didn't know that beforehand.

[+][comment deleted]2yΩ120

^{^}

In order to play stockfish with odds, I went to lichess.org/editor, removed the pieces as necessary, and then clicked “continue from here”, selected “play against computer”, and selected maximum strength computer opponent (level 8). This is full strength stockfish with a depth of 22 moves and calculation time of 1000 ms. I also tested with the higher depth and calculation time of the “analysis board”, and was still able to win easily with queen odds.

LESSWRONG
LW

LESSWRONG
LW

325

When do "brains beat brawn" in Chess? An experiment

325

Ω 61

325

Ω 61