Arimaa was an earlier attempt to do this. Developed in 2003, a computer beat humans in 2015. This site summarizes some of its anti-AI properties as

  • On average there are over 17,000 possible moves compared to about 30 for chess; this significantly limits how deep computers can think, but does not seem to affect humans.
  • Opening books are useless since the starting position is not fixed. There are over 64 million ways to start the game.
  • End game databases are not helpful since a game can end with all pieces still on the board.
  • Research papers on Arimaa suggest it is more of a strategic and positional game with less emphasis on tactics.

Riddles

Riddle: What month of the year has 28 days?
Answer: All of them

Riddle: What is full of holes but still holds water?
Answer: A sponge

Riddle: What is always in front of you but can’t be seen?
Answer: The future

Riddle. What can you break, even if you never pick it up or touch it?
Answer: A promise

Riddle: A man who was outside in the rain without an umbrella or hat didn’t get a single hair on his head wet. Why?
Answer: He was bald.

Riddle: I shave every day, but my beard stays the same. What am I?
Answer: A barber

Riddle: You see a boat filled wit... (read more)

[This comment is no longer endorsed by its author]Reply
1Max Hodges1moHow about if you have to solve brain teasers by visual analogy. For example: a card shows a drawing of a bear and a 12-inch ruler; answer is "BAREFOOT." A pair of dice showing the value of 2 (one and one); answer: "SNAKE EYES." The word "READ" between two lines; answer: "READ BETWEEN THE LINES." The word "agent" twice; answer: "DOUBLE AGENT." A picture of an Apple and the number 3.14158. You get the point.
1Long try3moTks Kaj. I can see that this designer tried to fuck AIs up by the brute force way, which is not efficient and, well, not elegant. The game also kind of suffers from the same problem as Esperanto, that is it's way too "eurocentric". Those summaries from the site sound dubious. Of course that affects humans. This is like sacrificing most of your 2nd goal to get a tiny little bit ahead on your 1st goal. Absurd. Many strategy/abstract games, even chess, can end with all pieces alive. Reviews I read suggest otherwise. Moreover, the game claims that it's among the highest rated on BGG. Following the link reveals that it's down in the 40ish or 50ish ranks, below Go, Xiangqi, Shogi, and even Chess, which it aspires to improve from. Besides, there's a pattern I noticed from reading the reviews. Those high scores for Animaa usually come from earlier years, 2000s. Conversely, the recent ones are dominated by negative views. In them we can see those repeated complaints about slow pace, boring feel and stripping off of chess' aesthetics... So, I'd argue that Animaa isn't really an attempt to do what I asked in the question. It went solely for the 1st goal while completely ignoring the 2nd goal, which weigh about 40-45% of importance IMO. After all, the human element is just something we're having an edge over AIs. And what is a game if it doesn't have people playing??

[ Question ]

A game designed to beat AI?

by Long try 1 min read17th Mar 202028 comments

12


It's no surprise LW is dominated by COVID questions right now. My guess is that many of us are holding their ground at home and have more spare time than usual. So here is a question for you: if you are to design a 1v1-type board game and your purpose is to confuse the AIs, to make it as difficult for them as possible, to level the playground between humans & machines, or at least to prolong the period when human players have an upper hand over AIs; then what designs would you use, what elements would you introduce into the game?

Of course, the 2nd most important goal is to make an interesting game, one that fascinate people and keep them playing whenever they have free time - during pandemics, for example.

New Answer
Ask Related Question
New Comment

8 Answers

Arimaa was an earlier attempt to do this. Developed in 2003, a computer beat humans in 2015. This site summarizes some of its anti-AI properties as

  • On average there are over 17,000 possible moves compared to about 30 for chess; this significantly limits how deep computers can think, but does not seem to affect humans.
  • Opening books are useless since the starting position is not fixed. There are over 64 million ways to start the game.
  • End game databases are not helpful since a game can end with all pieces still on the board.
  • Research papers on Arimaa suggest it is more of a strategic and positional game with less emphasis on tactics.

Excellent question! Once again, late to the party, but here are my thoughts:

It's very hard to come up with any board game where humans would beat computers, let alone an interesting one. Board games, by their nature, are discretized and usually perfect information. This type of game is not only solved by AI, but solved by essentially a single algorithm. Card games with mixed strategy equilibrium like Poker do a little better, although Poker has been solved the algorithm doesn't generalize to other card games without significant feature engineering.

If I were to design a board game to stump AIs, I would use these elements:

  • Incomplete information, with the possibility of information gathering ("scouting") at a cost (like in Starcraft 2) to invalidate Markov property
  • Lengthy gameplay (number of moves) to make the credit assignment problem as bad as possible for RL agents
  • Patterns that require abstract reasoning to discern (e.g. the pigeonhole principle lets you conclude immediately that 1001 pigeons don't fit in 1000 pigeonholes; an insight that can't practically be learned through random exploration of permutations)

The last element in particular is a subtle art and must be used with caution, because it trades off intractability for RL against intractability for traditional AI: If the pattern is too rigid the programmer could just hard-code it into a database.

If we considered video games instead, the task becomes much easier. DOTA 2 and Starcraft 2 AIs still can't beat human professionals at the full game despite the news hype, although they probably can beat the average human player. Some games, such as Chronotron or Ultimate Chicken Horse, might be impossible for current AI techniques to even achieve average human level performance on.

Manual dexterity. I'm pretty sure I can whoop any AI at Jenga, for the next 3 years or so. And the more fiddly, the bigger my expected advantage - Men at Work is an example of an even more challenging game, with many more possible game states.

Machine learning is bad at situations where it is provided with limited training data. Therefore I would design a game with frequently-changing rules. In particular, I would create an expansion set for Betrayal at House on the Hill.

Betrayal at House on the Hill is about exploring a haunted house. The gimmick is you do not know the rules of the game before you begin playing. There are many different rules the haunted house might obey.

Humans could beat machines for a long time if the following two eratta were applied:

  1. An intelligence may not have access to information ahead of time about the expansion packs' rules. (It is fair play for the AI's programmers to have access to the base ruleset but not the special iteration-specific rulesets.)
  2. An intelligence only gets points for winning the first time it encounters a particular ruleset.

An AI would need to read the specialized rules on-the-spot and then understand the semantics well enough to devise a strategy. Then the computer would have to execute this strategy correctly on its first try. No software in existence today can do anything like this.

Not only could humans crush machines at this board game, today's best machine learning software cannot even play this game (follow the rules) without its programmers' reading the complete rulebook ahead of time, which is cheating.

As for goal #2, Betrayal at House on the Hill is my favorite board game.

Each player is provided with a board and a male assistant. They're not allowed to use anyone else's help. The winner is the first one to produce a human baby.

I believe this meets the second goal, too.

Riddles


Riddle: What month of the year has 28 days?
Answer: All of them

Riddle: What is full of holes but still holds water?
Answer: A sponge

Riddle: What is always in front of you but can’t be seen?
Answer: The future

Riddle. What can you break, even if you never pick it up or touch it?
Answer: A promise

Riddle: A man who was outside in the rain without an umbrella or hat didn’t get a single hair on his head wet. Why?
Answer: He was bald.

Riddle: I shave every day, but my beard stays the same. What am I?
Answer: A barber

Riddle: You see a boat filled with people, yet there isn’t a single person on board. How is that possible?
Answer: All the people on the boat are married.

Riddle: A man dies of old age on his 25 birthday. How is this possible?
Answer: He was born on February 29.

Riddle: I have branches, but no fruit, trunk or leaves. What am I?
Answer: A bank

Riddle: What can’t talk but will reply when spoken to?
Answer: An echo

Riddle: The more of this there is, the less you see. What is it?
Answer: Darkness

How about if you have to solve brain teasers by visual analogy. For example: a card shows a drawing of a bear and a 12-inch ruler; answer is "BAREFOOT." A pair of dice showing the value of 2 (one and one); answer: "SNAKE EYES." The word "READ" between two lines; answer: "READ BETWEEN THE LINES." The word "agent" twice; answer: "DOUBLE AGENT." A picture of an Apple and the number 3.14158. You get the point.

Couldn't you just pick a game that was most similar to the unsolved problems in AI? For example, you could say that the AI box experiment is a game.