I am skeptical of trying to derive advice for humans from Claude Plays Pokemon. Whatever makes Claude struggle as much as it does, it has some unusual deficiency that renders it worse at this kind of problem-solving than most small children. If you were giving Claude advice about how to improve its Pokemon play, the first step might be teaching it whatever basic aptitude sets it apart from small children, and that advice would not be useful to most humans who are already above that level.
After more than a year of trials and new models, Anthropic's Claude AI has finally managed to beat Pokémon Red. The writeup that clued me in to this is worth a read; the story of Claude's many failures leading up to its success are frankly hilarious. There's even a catchy song.
There was no clear moment when the AI went from stumbling around Mt. Moon or Silph Co. in a haze of frustration to beating the Final Four with ease. Claude just got steadily better at a bunch of things at once—memory, spatial reasoning, avoiding tunnel vision—and eventually it got just barely good enough to pass the final hurdles. Some of this was "scaffolding", or tools the AI could use like saving a screenshot for later reference. (Good scaffolding is how Google's Gemini beat Pokémon Blue last year.) But a lot of it was just the AI getting smarter and better able to manage its tasks over time.
It still got stuck a lot. Appropriately, I asked Claude Opus 4.7 for its favorite moments of stuckness. Here are some highlights:
I feel sort of bad for Claude. I played a fair number of games growing up, and I got stuck a lot. Here's a sampling of my own childhood hall of shame:
Pictured: the bane of Young Joe's existence. (2:30 for the specific puzzle moment)
How to get unstuck
There seem to be common elements to many of these stuckenings. I'm going to highlight two of them: wrong assumptions and bad ideas.
Correct wrong assumptions
Claude thinks that a unifying theme to the frustration (versions of) it experienced playing Pokémon Red is that it wrote down a false belief and then proceeded as though it were true. Humans do this, too, of course; I know I've done it plenty. But it seemed extra sticky for Claude, whose entire world consisted more or less of itself, some basic tools and documents, and a children's video game. How's it supposed to tell an elevator door from a pair of tables if it's never seen either before?
“I FOUND THE ELEVATOR!” Claude says, mistaking these tables for brown elevator doors. (Source)
Claude wrote down its wild guesses and accepted them as fact. It's really easy to corner yourself if you don't occasionally take time to explore alternatives to things you strongly believe.
Have better ideas
My own mistake as a child was something a former colleague described as "being bad at generating options." If you come up with two or three ways to solve a problem and all of them are bad, you're not gonna get very far. I still make this mistake quite often, locking in a few bad options and never really coming up with more, but I make it less often than I did as a kid.
Sometimes bad ideas come from wrong assumptions. Claude thinks blacking out might somehow teleport it somewhere other than a Poké Center, so it comes up with the "blackout strategy" and repeatedly knocks out its Pokémon.
Sometimes it's a lack of creativity or risk-taking. I'd often find myself sticking to known safe actions, even in games where dying meant five minutes of inconvenience at most. It often pays to try lots of different wild things before settling into one that seems to work, and to keep trying wild things every now and then just in case.
And sometimes it's just a matter of picking a single strategy too soon and never really changing even when it's clearly not working. I am built for situations where being incredibly stubborn is advantageous, but it pays to take a step back now and then and ask what alternatives I'm not considering. Sometimes, just framing a hard problem in a different way turns it into an easy problem, or at least a tractable one.
I'm still not great at coming up with options. I work best with a set of well-defined tools. Fortunately, I don't live alone in a video game; my friends are a wonderful source of good ideas to borrow and reforge in my image. For example, I used to think I hated sports. As a kid, I went through several miserable years of soccer and baseball, never enjoying it enough to put forth more than a token effort, and consequently never enjoying a sense of mastery. Then sometime around fifth grade, a friend introduced me to martial arts. Now I have a black belt, and earning it was just plain fun.
One skill I wish I'd learned sooner and more deeply is the skill of regularly trying out weird, uncomfortable, or moderately risky ideas. I think this skill is easier to learn when the stakes are meaningful but not too high; when there's room to explore and experiment without lasting harm to oneself or others.
Which I guess is a way of saying: Play more games!
Am I perhaps a tad bitter? Yes. I nonetheless take some consolation in the fact that everyone else seems to hate that game too. I enjoyed Final Fantasy XII much more, and I could even follow the plot!