Arimaa was an earlier attempt to do this. Developed in 2003, a computer beat humans in 2015. This site summarizes some of its anti-AI properties as
- On average there are over 17,000 possible moves compared to about 30 for chess; this significantly limits how deep computers can think, but does not seem to affect humans.
- Opening books are useless since the starting position is not fixed. There are over 64 million ways to start the game.
- End game databases are not helpful since a game can end with all pieces still on the board.
- Research papers on Arimaa suggest it is more of a strategic and positional game with less emphasis on tactics.
Riddle: What month of the year has 28 days?
Answer: All of them
Riddle: What is full of holes but still holds water?
Answer: A sponge
Riddle: What is always in front of you but can’t be seen?
Answer: The future
Riddle. What can you break, even if you never pick it up or touch it?
Answer: A promise
Riddle: A man who was outside in the rain without an umbrella or hat didn’t get a single hair on his head wet. Why?
Answer: He was bald.
Riddle: I shave every day, but my beard stays the same. What am I?
Answer: A barber
Riddle: You see a boat filled wit...
How about if you have to solve brain teasers by visual analogy. For example: a card shows a drawing of a bear and a 12-inch ruler; answer is "BAREFOOT." A pair of dice showing the value of 2 (one and one); answer: "SNAKE EYES." The word "READ" between two lines; answer: "READ BETWEEN THE LINES." The word "agent" twice; answer: "DOUBLE AGENT." A picture of an Apple and the number 3.14158. You get the point.
Tks Kaj. I can see that this designer tried to fuck AIs up by the brute force way, which is not efficient and, well, not elegant. The game also kind of suffers from the same problem as Esperanto, that is it's way too "eurocentric".
Those summaries from the site sound dubious.
On average there are over 17,000 possible moves compared to about 30 for chess; this significantly limits how deep computers can think, but does not seem to affect humans.
Of course that affects humans. This is like sacrificing most of your 2nd goal to get a tiny little bi...
Excellent question! Once again, late to the party, but here are my thoughts:
It's very hard to come up with any board game where humans would beat computers, let alone an interesting one. Board games, by their nature, are discretized and usually perfect information. This type of game is not only solved by AI, but solved by essentially a single algorithm. Card games with mixed strategy equilibrium like Poker do a little better, although Poker has been solved the algorithm doesn't generalize to other card games without significant feature engineering.
If I were to design a board game to stump AIs, I would use these elements:
The last element in particular is a subtle art and must be used with caution, because it trades off intractability for RL against intractability for traditional AI: If the pattern is too rigid the programmer could just hard-code it into a database.
If we considered video games instead, the task becomes much easier. DOTA 2 and Starcraft 2 AIs still can't beat human professionals at the full game despite the news hype, although they probably can beat the average human player. Some games, such as Chronotron or Ultimate Chicken Horse, might be impossible for current AI techniques to even achieve average human level performance on.
In addition to your ideas, I would add long tales and chaotic systems. It's hard to train an AI on 1,000,000 datapoints when the value function of the 1,000,001st datapoint could outweigh all the previous cumulative results.
To generalize this mathematically, a board game ought to have a value function that never converges no matter how many times you play the game.
Thanks a bunch Maxim! I remember you in my hypothetical "drop all water on Earth" question - so, a usually late guy but always arrives with excellent answers :)
1 of the main differences between board & card games that I can discern is that 1 has perfect info, as you pointed out, the other not. Thus if we integrate imperfect info into a board game, can programmers just combine the 2 algorithms to solve it, or they will have to find another approach?
Unlike the Earth water question, I have some difficulties understanding the technical terms full...
Manual dexterity. I'm pretty sure I can whoop any AI at Jenga, for the next 3 years or so. And the more fiddly, the bigger my expected advantage - Men at Work is an example of an even more challenging game, with many more possible game states.
Good idea! Maybe but something with more variety than Jenga. I'd bet hard-cash a dedicated team could make a Jenga champion in fraction of that time. Sounds like a fun challenge. Here's are a few impressive robot dexterity projects:
https://openai.com/blog/learning-dexterity/
https://www.youtube.com/watch?v=kVmp0uGtShk
Hmm. Well, anything physical can be a challenge to AI, since we don't have many real-life machines playing games physically. While technically the idea rings true, my question didn't intend to explore much of this approach :)
Machine learning is bad at situations where it is provided with limited training data. Therefore I would design a game with frequently-changing rules. In particular, I would create an expansion set for Betrayal at House on the Hill.
Betrayal at House on the Hill is about exploring a haunted house. The gimmick is you do not know the rules of the game before you begin playing. There are many different rules the haunted house might obey.
Humans could beat machines for a long time if the following two eratta were applied:
An AI would need to read the specialized rules on-the-spot and then understand the semantics well enough to devise a strategy. Then the computer would have to execute this strategy correctly on its first try. No software in existence today can do anything like this.
Not only could humans crush machines at this board game, today's best machine learning software cannot even play this game (follow the rules) without its programmers' reading the complete rulebook ahead of time, which is cheating.
As for goal #2, Betrayal at House on the Hill is my favorite board game.
How about if you have to solve brain teasers by visual analogy. For example: a card shows a drawing of a bear and a 12-inch ruler; answer is "BAREFOOT." A pair of dice showing the value of 2 (one and one); answer: "SNAKE EYES." The word "READ" between two lines; answer: "READ BETWEEN THE LINES." The word "agent" twice; answer: "DOUBLE AGENT." A picture of an Apple and the number 3.14158. You get the point.
I think this approach tries to use puns to confuse AI... but it'll get old quickly for humans. Once the card is answered, it can no longer be of much value next times.
Each player is provided with a board and a male assistant. They're not allowed to use anyone else's help. The winner is the first one to produce a human baby.
I believe this meets the second goal, too.
You could achieve a similarly lopsided result faster with the in-person Turing test, albeit at the expense of goal #2.
Each player guesses whether the opponent is human or machine. Each player gets one point for a correct guess and one point for convincing the opponent that he/she/it is a human.
To beat humans at this, we would need the following developments:
Riddles
Riddle: What month of the year has 28 days?
Answer: All of them
Riddle: What is full of holes but still holds water?
Answer: A sponge
Riddle: What is always in front of you but can’t be seen?
Answer: The future
Riddle. What can you break, even if you never pick it up or touch it?
Answer: A promise
Riddle: A man who was outside in the rain without an umbrella or hat didn’t get a single hair on his head wet. Why?
Answer: He was bald.
Riddle: I shave every day, but my beard stays the same. What am I?
Answer: A barber
Riddle: You see a boat filled with people, yet there isn’t a single person on board. How is that possible?
Answer: All the people on the boat are married.
Riddle: A man dies of old age on his 25 birthday. How is this possible?
Answer: He was born on February 29.
Riddle: I have branches, but no fruit, trunk or leaves. What am I?
Answer: A bank
Riddle: What can’t talk but will reply when spoken to?
Answer: An echo
Riddle: The more of this there is, the less you see. What is it?
Answer: Darkness
Couldn't you just pick a game that was most similar to the unsolved problems in AI? For example, you could say that the AI box experiment is a game.
Er... I'm not sure I follow. In the sentence, if the word "you" means "the individual me" then no, I don't think the AI box ex is a game. It's merely a thought experiment, and actually a pretty stupid one. If a box is designed to completely separate an AI from the real world then allowing it to interact with outside personnel destroys the purpose of the box in the 1st place. It's about as much a game to me as Roko's basilisk.
If the word "you" mean "people in general" then no, unsolved AI prob...
It's no surprise LW is dominated by COVID questions right now. My guess is that many of us are holding their ground at home and have more spare time than usual. So here is a question for you: if you are to design a 1v1-type board game and your purpose is to confuse the AIs, to make it as difficult for them as possible, to level the playground between humans & machines, or at least to prolong the period when human players have an upper hand over AIs; then what designs would you use, what elements would you introduce into the game?
Of course, the 2nd most important goal is to make an interesting game, one that fascinate people and keep them playing whenever they have free time - during pandemics, for example.