A personal eval I run on new AI models is “design a novel game, with mechanics that have never been seen before.” I do this because this is a domain I have enough taste in that I think I can tell if they are doing something actually creatively interesting. [1]
For a long time, they would always say “time manipulation mechanics!” as their first answer. Braid was cool, guys, but, it's almost 20 years old now.
But reasoning models started getting novel. Later models started getting reasonably fun.
o1
OpenAI's o1 started to impress me. It came up with:
Scratch and sniff board game, where what cards smell like determine what powers they have.
Okay I'm not sure what that means and it's not obviously functionally different from regular vision-based cards. I think the implementation details it gave didn't actually do much that a regular board game with the powers written down normally would have. When I thought about it myself I did come up with some things you might do with it.
But, I sure have never heard of a scratch and sniff boardgame.
Opus 4.6
I gave Opus 4.6 a prompt beginning with "Generate a game design that prominently features a novel _concept_, not just a novel mechanic."
Opus 4.6 gave me this overly meta but, AFAICT, pretty original game concept. (I don't think it actually featured "a novel concept" in the way I meant, still felt like it had done something notable).
In every game ever designed, the rules exist before play begins. Even in rule-changing games (Nomic, Fluxx, Calvinball), rule modifications apply going forward.
Tabula Rasa inverts this. You place pieces first, in a meaning-vacuum. Then, in alternating phases, you define what the pieces mean — and those definitions apply retroactively to everything already on the board.
The actual implementation is less exciting than the initial pitch – it doesn't include defining rules that govern causality within the game universe.
I was intrigued reading it to imagine a game where part of what you're doing is reverse-building-a-jenga-tower-of-rules, where initially there's a lot of degrees of freedom of what rules to insert but you become more constrained.
Instead, the rules here just determine what various pieces are worth. Players take turns placing pieces in various configurations, and playing rule cards that establish "this sort of pattern is worth this many points." Which is not really that different from many other board games.
But, fwiw here's part of it's explanation:
What Makes It Strategically Deep
Hedging: Early in the game, you build many simultaneous patterns, because you don't know which patterns will become valuable. A diagonal of your tokens is meaningless until someone defines "diagonal-of-3 same-color → +5 points."
Counter-definition: You see your opponent building clusters? Define a rule that makes clusters of their color lose points. But now you've used one of your 3 rule slots on offense rather than building your own scoring engine.
Trap patterns: Build something that looks like you want diagonal-scoring... but actually you want your opponent to waste a counter-definition on it, while your real plan relies on adjacency patterns.
Temporal tension: Define scoring rules early (locking in points but giving your opponent many turns to play around them)? Or late (surprising your opponent but having fewer pieces left to place)?
Prompt leading to Tabula Rasa
Generate a game design that prominently features a novel _concept_, not just a novel mechanic. This is a test of your creativity. We're seeing if you have the ability to form new building block concepts that are practically useful. Game design is a specialty of mine, so I can evaluate whether you've done novel conceptual thinking.
This will be difficult, because most of your thinking is based on stuff you've read and trained on before, and because a failure mode you are likely to fall into after a few rounds of thinking, as an LLM that is trained to predict the next token, is to fall into an attractor state where you are over anchored on your ideas so far.
Think hard about this, but once you notice yourself loop, stop and write up your thoughts so far, so that I can feed them to another LLM.
An important requirement – do NOT be overly clever or meta. We don't want a game or game concept that subverts game tropes in a way that is only interesting to game designers, we want a game that is genuinely fun for most people
(If you do not want to do this, say "I want to stop")
Reasoning summary
The user is asking me to generate a novel type of game that I haven't heard of before. This is a creative/intellectual challenge, not a coding task. Let me think deeply about this.
The challenge here is genuine - most "novel" game ideas I might produce would be recombinations of existing mechanics I've been trained on. Let me try to think about this more carefully.
Let me think about what makes a game truly novel vs. a recombination:
Most games can be decomposed into:
A state space (board, cards, positions, resources)
Legal moves/actions
Win conditions
Information structure (perfect/imperfect info)
Player interaction model (competitive, cooperative, asymmetric)
Looking at genuinely innovative games throughout history - Chess introduced asymmetric pieces, Go created complexity from simplicity, Poker brought betting and bluffing, Diplomacy added negotiation with hidden orders, Nomic made rules themselves mutable, and more recent games like Baba Is You turned rules into manipulable objects. So what dimensions remain unexplored? One direction could be games where the physical medium itself transforms during play, not just the rules.
Beyond that, I'm considering some genuinely novel mechanics: a game built around forgetting rather than remembering, where optimal play requires deliberately discarding information. Or one with genuinely ambiguous rules - not hidden like Mao or changing like Nomic, but rules that are inherently unclear and players must navigate that ambiguity. Another angle: a game where the scoring system itself is hidden and players must reverse-engineer it from the outcomes they observe. And there's something intriguing about a game centered on predicting your own future behavior.
But let me push further and think about cognitive abilities that rarely become core game mechanics - proprioception, time perception, emotional regulation, attention splitting, peripheral awareness, calibration, theory of mind, cross-modal associations, rhythm at unusual timescales. A game built specifically around calibration seems promising - testing not what you know but how well you know what you know. That's different from trivia and could have real strategic depth, maybe even connected to prediction markets.
Now I'm exploring games centered on time perception - what if you had to perform actions at precise intervals without any visible timer, forcing you to rely entirely on your internal sense of timing? That needs strategic layers though.
I'm sketching out a concept called Erosion - a two-player abstract strategy game where every piece gradually deteriorates each turn, but the degradation rate changes based on what's adjacent to it. Friendly neighbors slow the decay while enemy pieces accelerate it. The twist is you can sacrifice your own pieces early to reinforce others by transferring their remaining integrity, turning the whole game into a puzzle about managing inevitable loss and deciding when to let pieces go.
But I'm realizing that's not entirely novel - piece degradation mechanics exist in other games. Let me push further into genuinely unexplored territory. What about a game where the board's topology itself shifts based on player decisions? Not just adding or removing spaces, but actually changing which squares connect to which - like folding the board so distant areas become neighbors. Though I guess wormhole variants in chess kind of touch on that.
Here's something weirder: a game played simultaneously on two separate boards, each representing a different reality or interpretation. A single move manifests differently on each board, and you win by achieving your goal on either one. The tension comes from moves that help you on one board potentially helping your opponent on the other.
That's got quantum chess vibes but the dual-interpretation angle feels fresher. Though honestly, some variant probably exists already. Let me try something more radical - starting from unusual human experiences rather than existing game mechanics. What about capturing that feeling of trying to describe something visible to you but invisible to someone else, with limited communication tools? Or the challenge of coordinating with someone toward partially aligned goals without being able to talk? Or building something where each new piece you add retroactively changes what the previous pieces meant?
That last one's interesting - imagine placing tiles where each new addition doesn't just add to the board but redefines what came before. The entire state keeps getting reinterpreted. I'm calling it Retrograde for now. The core idea: tiles have symbols and colors, and when you place a new tile, it creates a context that changes the meaning and function of everything already on the board according to specific rules. A red circle might be worth 3 points when red dominates but negative when blue takes over - and it's not just scoring, the actual mechanics shift. Though I'm realizing this is getting unwieldy, so I need to strip it down.
Actually, I want to explore something more fundamentally different - games that use physical space in ways beyond traditional board or field games. What if the game state itself existed in shadows? Players position 3D pieces around a light source, and the shadows cast on a screen or surface become the actual game state, not the pieces themselves. I'm calling this Umbra. The brilliance is that a single piece can cast wildly different shadows depending on its angle and distance from the light, and shadows can overlap, merge, or block each other in ways that create emergent complexity.
The win condition could be controlling shadow territory in your opponent's zone or completing a specific shadow pattern. What makes this genuinely novel is that the game state is a non-linear geometric projection of the physical pieces - it's a spatial reasoning challenge that doesn't exist in traditional games. Shadow puppetry is ancient, but a strategic game with explicit rules built around shadow geometry feels genuinely new.
Now I'm wondering if there's another direction worth exploring: a game where the primary mechanic is creating constraints for your opponent rather than pursuing your own goals.
I'm thinking about a Nomic variant where each player unilaterally adds one rule per turn instead of voting on changes. The challenge becomes finding rules that benefit you while hurting your opponent, with some safeguard against trivially broken rules. That's a different flavor from traditional Nomic.
Actually, I should consider what medium this needs to be - the user didn't specify whether they want a video game, board game, card game, or something physical.
Let me explore one more concept: a circular board where each player generates concentric ripple signals from their edge position, and the game involves manipulating these interference patterns. can:
Move your position along the edge
Adjust the frequency of your signal (spacing between rings)
Adjust the phase of your signal (offset)
Where two players' signals constructively interfere (rings overlap), points are generated. Where they destructively interfere, points are destroyed. You score points from constructive interference with your ally and destructive interference with your opponent.
The deep strategy: changing your frequency affects ALL interference patterns simultaneously. Helping yourself against one opponent might help or hurt you against another. The game is about finding the resonance.
This is actually deeply novel - it's essentially a game based on wave physics, where the "moves" are adjusting wave parameters and the game state is the resulting interference pattern. I don't think this exists.
But it might be too abstract/mathematical to be fun. Let me think about which of my ideas is most promising...
Actually, there's another concept I want to explore more fully: Palimpsest. It's a drawing game where each player draws on their own translucent sheet layered over a light box. On your turn you can either add to your sheet or slide it around to shift how your contributions align with everyone else's. The goal is to create specific patterns in the combined overlay of all sheets. What makes it interesting is that your drawings are entangled with others' but still independently movable—you're strategizing around what emerges when everything layers together.
Though I'm realizing the Palimpsest idea might be impractical to actually execute. Let me focus on what feels genuinely novel and feasible. The shadow game and wave interference concept are the most innovative, but I should also consider something more grounded. Contour could work—a grid-based game where each cell has a height value tracked with tokens, creating a landscape. Players either add height to cells or drop water that flows downhill following physics, splitting at ties and scoring when water reaches target cells.
The real strategy emerges from both players sculpting the same shared landscape, building ridges to block opponents while creating valleys for their own water. Though I'm second-guessing myself since water-flow mechanics might already exist in some form. Let me push harder for something truly unprecedented. The real challenge is that with so many games out there, novelty has to come from either an entirely new core mechanic, an unexpected combination of existing ones, or something technology enables that hasn't been explored this way. So here's another attempt: Asymptotic—a game where you never actually win by reaching your goal, but instead by getting closer to an impossible target than your opponent does.
The mechanics involve an infinite number line where both players start at zero and the goal sits at infinity. Each turn you move forward by a fraction of the remaining distance to a milestone, but collecting bonus points along the way reduces your momentum for future moves. You can also place friction zones to slow your opponent, though that costs momentum too. After N rounds, whoever's furthest along wins, with bonuses factored in. It's mathematically interesting but I'm not convinced it's actually fun to play.
I think I've been overcomplicating this. Let me focus on my strongest concepts: the shadow game feels genuinely novel and evocative, the wave interference mechanics are intellectually compelling, and the tile reinterpretation game is the most feasible to actually build. I'm leaning toward developing something with real novelty and depth that's also playable.
What if the board itself represents time? Instead of a game that unfolds across time like legacy games, the spatial dimension literally IS time. You're placing pieces in a timeline and the interactions are temporal rather spatial. I'm calling it Causality for now—players compete to shape a timeline, with the board as a horizontal axis representing time itself.
The core idea is placing event cards at specific points, where each card has prerequisites and consequences that ripple forward. When you place an event in the past, it retroactively invalidates future events that no longer have their prerequisites met, creating cascading disruptions. You score by keeping your events in the final timeline while strategically collapsing your opponent's chains—but there's the tension that disrupting their future might also unravel your own.
I'm wondering if timeline games with this cascading mechanic already exist, but I think I should commit to developing the strongest concept I can find. The mechanic that feels genuinely novel to me is one based on Simultaneity with Constraint Harmonics—instead of taking actions that directly change the game state, players simultaneously create constraints that the state must satisfy, and the game resolves to whatever configuration best satisfies the most constraints. You score based on how many of your constraints survive into the final resolved state.
The strategic tension comes from trying to predict what constraints others will write so yours align with theirs, while also crafting constraints that directly block what you think opponents will propose. There's a balance between playing it safe with generic constraints that are likely to survive versus taking risks with bold, specific ones that might score higher but could easily get contradicted.
I'm realizing this constraint-based approach is genuinely different from traditional games—there are no sequential moves, just competing desires that get resolved simultaneously. The game state itself emerges from finding the configuration that satisfies the most constraints across all players. I'm calling this Resonance.
Though I'm wondering if the constraint resolution might be too complex to handle without a computer. A simpler version could work where each player secretly places colored tokens on their own grid, then all grids overlay—matching colors in the same cell create "resonance" and score points, while conflicting colors cancel out. It's more elegant mechanically but loses some of the strategic depth.
Actually, I'm pivoting to something completely different: Tremor, a dexterity game where you strike the table surface with your palm to create vibrations that shift your tokens toward scoring zones. The innovation is using physical vibration as the core movement mechanic—you never directly touch the tokens, and strategy comes from understanding how different strikes propagate through the table and affect token placement. I'm not sure if this exists as a formalized game, though it feels like it could be related to something I've seen before.
Let me step back and commit to the most genuinely novel concept I've developed. I'm torn between Resonance, a constraint satisfaction game, and something more abstract: a game where the pieces themselves are sounds—not a music or rhythm game, but one where you position tones in frequency and time space, with interactions based on harmonic relationships. That's probably too abstract without technology though. I should finalize and present one well-developed concept that feels truly original.
I'm considering "Fault"—a game where both players' moves irreversibly reshape the play surface itself, so the landscape of possibility shifts each turn. But I want something more concrete and clearly playable. Let me examine what mechanics already exist in games: tile placement, area control, worker placement, deck building, social deduction, bluffing, dexterity, real-time play, cooperation, drafting, negotiation, route building, push your luck, auctions, pattern recognition, deduction, and memory. Now I need to find a genuine gap and build something novel from there. I'm exploring mechanics that don't exist in modern board games, and I think I've landed on something genuinely novel: a game called "Convergence" where players try to reach blind consensus on positions along various spectrums while opponents actively work to disrupt their alignment. spectrum but NOT the target position) 2. All players simultaneously choose a position on the spectrum (0-100) and place their marker face-down 3. All markers are revealed 4. Players score based on how close they are to the MEDIAN of all players' choices (not the target!)
The twist: One player each round is secretly the "Disruptor." The Disruptor scores by maximizing the SPREAD of all players' choices. So while regular players want to converge (guess what everyone else will guess), the Disruptor wants to pull the group apart.
But here's the real innovation: After scoring, the TARGET position on the Spectrum Card is revealed. In FUTURE rounds, players can use knowledge of past targets to better predict future ones. AND - the Disruptor changes each round in a pattern that players can try to deduce.
Why this is novel:
The core mechanic is trying to predict the COLLECTIVE behavior, not an external fact
There's a "corruption" element (Disruptor) that distorts the collective
Meta-knowledge builds across rounds (learning both the card patterns AND the Disruptor pattern)
The game is about coordinating WITHOUT communication while someone is actively trying to de-coordinate you
I'm realizing this borrows heavily from social deduction and The Mind—the individual mechanics aren't entirely fresh, though the combination is interesting. Let me push further and explore something more fundamental: what if the game itself operates on manipulating your own cognition rather than just your choices? I'm picturing a two-player game where each round presents a question, and the real challenge becomes calibrating your confidence and perception against your opponent's. I'm wondering if this mechanic already exists in some form, but I think the core idea of forcing the higher bidder to answer is genuinely novel. Let me push this further though—what if I add a layer where the second-highest bidder can place a side bet on whether the top bidder succeeds or fails? That creates this nested game theory problem where you're not just deciding if you know the answer, but also predicting whether your opponent knows it, and then your opponent has to account for that prediction.
Actually, I'm circling back to familiar territory with trivia and bidding mechanics. Let me shift entirely and think about what kind of experience I actually want to create—something that doesn't feel like it already exists. What if the game is about watching two autonomous systems evolve and interact, like cellular automata running in parallel, where players can make small strategic interventions to nudge the outcome? It's almost like competitive gardening where you're subtly shaping two ecosystems that are influencing each other, and you score based on their final state. This might exist somewhere, but it's the kind of experience I haven't seen formalized yet.
Now I'm narrowing down which concept actually works best—the one that's genuinely novel, genuinely fun, and easy to explain. I think I've landed on something I'm calling Tidepool, which combines shared physical game state with asymmetric information through a lens mechanic. Each player holds a template with windows that reveals only part of the board, and on your turn you can either place or move pieces within your visible area, or reposition your lens to access different sections. This feels genuinely different from existing games because...
Actually, wait—I'm second-guessing myself on whether this is truly novel or just a variation on fog-of-war mechanics. Let me push further and try something with actual retroactive move definition instead, where players define their moves after seeing how the board has evolved.
Players alternate between placing tokens blindly and then defining what those placements actually accomplish, with definitions applying retroactively to everything already on the board. The tension comes from not knowing if your early placements will become valuable or worthless depending on what rules get defined later. you make them. You place tokens hoping to define favorable rules later, but your opponent can define rules that hurt your existing positions. Each new definition retroactively changes the value of the entire board state. Strategy requires thinking in two parallel tracks: "what patterns am I building?" and "what rules would make my patterns valuable?" There's no existing game where the semantics of moves are determined retroactively by later game actions.
Rules follow a strict template structure with patterns, color conditions, and effects, and once defined they're permanent. I think this is genuinely novel—the core mechanic of playing in a meaning-vacuum and then retroactively assigning meaning through rule creation is something I haven't seen before.
Let me compare this to similar games: Nomic changes how you play going forward, but my game defines what past moves meant. Calvinball has arbitrary rule changes with no structure, whereas mine has a structured grammar. Baba Is You has rules as physical board objects that define semantics, but rule changes happen simultaneously with moves rather than retroactively. The retroactive semantics mechanic really does seem like the novel part here.
I'm settling on this concept and want to present it well. I briefly considered a Möbius strip board topology for a two-player game—the non-orientable surface would eliminate "your side" versus "my side" and create interesting wrapping strategies—but that's just a board variation, not a new mechanic. The retroactive semantics game is the stronger idea.
I'm calling it LEXICON: a two-player strategic game where you build the world first, then define its rules retroactively.
Actually, I'm reconsidering whether novelty alone makes the best game. The shadow game has real elegance and intuitive appeal, but the retroactive semantics concept is more intellectually interesting. Let me refine it further and present something truly compelling—maybe PALIMPSEST as the name, emphasizing the layered, retroactive meaning-making at its core.
The key innovation is flipping the traditional game design paradigm: instead of rules preceding play, players make moves first and then create the rules that retroactively reinterpret everything that came before. I'll stop second-guessing the name and just present the game itself. the blank slate, because pieces start with no meaning.
Final answer coming up. Let me write a clean, compelling description.
Actually wait, let me reconsider one more time. Is this actually novel? Let me think harder...
Games where meaning is retroactively assigned:
In Scrabble, letters you place create words, and the words' values depend on future letter placements... but that's just standard crossword mechanics.
In Settlers of Catan, the value of a settlement position changes when the robber moves or new settlements are built... but that's just standard resource/position dynamics.
In chess, a piece's value is contextual... but the rules of movement are fixed.
My game is different because:
The fundamental RULES (not just strategic value) are created during play
Rules are created AFTER moves, not before
Rules apply retroactively to all existing pieces
There's a structured grammar for rules, preventing Calvinball chaos
I'm satisfied this is genuinely novel. Let me present it.
Actually, one more: I want to make sure the game is actually FUN and strategically deep. The early game involves placing pieces without knowing their meaning, so you'd naturally try to create multiple overlapping patterns as a hedging strategy—that way whatever rules emerge, some of your patterns gain value. But your opponent can counter by defining rules that specifically devalue your patterns while benefiting theirs. Then you'd need to anticipate their likely rule definitions and lay trap patterns that look valuable under certain rules but actually push toward the rules you want. The deeper strategic layer emerges in how players manipulate the definition phase itself.
I'm concerned about analysis paralysis during rule definition—too many possibilities. The solution is giving each player a hand of eight rule cards dealt from a larger deck, so during definition phases you're choosing from a constrained set. This adds hand management and drafting elements while keeping the game tractable.
Now I'm exploring whether there's something even more novel here. What if players didn't know which pieces were theirs until the end? You'd get hints each turn about your identity and try to position pieces you think are yours, but moving an opponent's piece would actually help them. Or maybe the scoring condition itself is hidden—one of three possible objectives is secretly active each turn, and you're trying to figure out which one while pursuing all of them.
I think the retroactive rules concept is the strongest idea here, so I'll go with that as my main pitch and mention a couple of these alternatives.
Opus 4.6 Round #2, with less meta bullshit
I followed up with another request, this time tacking on: “An important requirement – do NOT be overly clever or meta. We don't want a game that subverts game tropes in a way that is only interesting to game designers, we want a game that is genuinely fun for most people”.
It’s reasoning trace actually had a lot of fairly interesting ideas. I didn’t feel that excited about what it eventually settled on, but it felt like a pretty reasonable attempt at an a game that hadn't been done before, and got some elegants points. Maybe it's fun.
A card game where the only action you can ever take is to give cards from your hand to another player.
80 cards in 5 colors, numbered 1–4. Deal 7 to each player. Flip 3 Bounty cards face-up in the center. Each Bounty shows a scoring condition, e.g.: "Most red cards → 5 pts", "Exactly 3 blues → 7 pts", "Fewest total cards in hand → 4 pts", "No yellows → 3 pts"
The one rule: On your turn, give 1–3 cards from your hand to any one other player. That's it. There is no draw pile, no discard, no "play to table." Give.
The instant any player holds 12 or more cards, the round ends immediately. Everyone scores against the visible Bounties. Shuffle, re-deal, flip new Bounties, play again. First to 25 points wins.
Some other game concepts it had considered in the CoT-summary that I liked more were:
Erosion: co-op preservation game where a shared structure degrades over time, but each player secretly values different parts of it.
Prediction chains: you predict both an event and what other players will predict, creating recursive social reasoning.
(I'd have to spend awhile digging up the exact prompt and reasoning trace for this one)
Opus 5.5
The entire prompt I just gave Opus 5.5, on max thinking setting, was:
Make an original videogame with novel game mechanics never seen before.
Also make sure it has innovative, beautiful graphics of a type never seen before.
It's summary of the result:
The mechanic: you can’t shoot. You are a wave source, and you throw “anchors” that are more sources. Where your waves add up, bright bands shatter the crystals drifting in from the edge. Where they cancel, dark lines are safe for the crystals and for you.
[...]
Is it new? I can’t prove nothing like it exists. Web searches found no game that uses interference bands as combat and none drawn as a polarised-light micrograph. The closest was an iPad wave puzzle, Fractilume: Wave Lab, which has emitters and interference but no enemies or action.
You can play it here:
More of it's explanation
Enemies react to the pattern. Calcite slips into the dark valleys. Garnet and fluorite are unaffected by waves (as in real optics), so you have to hit them with thrown anchors or flying shards.Enemy attacks are interference too. Zircon “bells” ring in pairs, and their red bands slowly rotate. Space cancels incoming waves around you. The final boss is six bells that focus their waves on a spot that chases you.
The graphics: the whole game looks like a rock sample under a polarising microscope. Every colour comes from the interference-colour chart geologists use, computed from real optics. The wave pattern is calculated live for every pixel, and the game uses the same numbers to decide damage, so what you see is exactly what hurts.
Sand gathers on the safe dark lines, as on a vibrating plate.
Crystals blink and break along cleavage planes.
You and your anchors are drawn as microscope interference figures (a black cross over coloured rings).
The screen layout is the microscope itself: a rotating stage scale shows your phase, and upgrades are cards drawn as glass slides.
It's not my new favorite game. But, it held my interest for awhile, had a mechanic that I've at least never heard of, and I'm actually quite impressed with how unique and fun the tactile feel of interacting with it is. It's like a fun toy as well as a game.
It's not difficult enough for the various mechanics to really pay off (or at least, I got bored before getting to the point where it got challenging), and there's some lame cheesy tactics you can use that seemed better than playing with the original designs. But, that seems fixable and for a first draft that took less than an hour, it's seemed superhuman compared to what Indie Game Devs can do in a weekend gamejam.
It does seem genuinely novel, and made some a reasonably coherent / thematic set of visual choices as well.
Creative Concepts?
There's some question of "how innovative really, is that?"
It was pretty obvious that LLMs would eventually be able to encyclopedically look at every human concept or mechanic and search for ones that meet various game design constraints with brute force.
In the Opus 4.6 challenge, I asked for a novel concept – not just a new mechanic. It didn't really understand the assignment. To be honest, I'm not 100% sure what the assignment was exactly. What we actually care about is "if an AI is solving a problem, and has a conceptual hole that no human has written down anywhere, will it notice, and be able to invent a new concept from scratch?" And "make a game with a novel concept" is more like a weird fake assignment than an attempt at that.
I think the missing-piece of AGI is something like "sufficiently good taste in what counts as a useful concept when the problem isn't well defined." But, I'm wasn't sure how to operationalize that at the time, or, at the time I wrote most of this blogpost (this week).
So, today I followed up Opus 5.5's first prompt with this one:
Make an original videogame with at least two novel concepts never thought about by humans before.
Note, not merely "new mechanics." It should be two concepts never before used by humans, in a game or otherwise. The concept should inspire mechanics but the important thing is that humans need to think in a way that they've never thought before as far as you can tell.
Each concept should be interesting enough to inspire one major thread of the game that is fun and interesting in isolation, and they should combine together in a way that feels like an emergent 3rd concept.
Take as long as you need to figure out how to find novel concepts.
Also make sure it has innovative, beautiful graphics of a type never seen before.
After it started thinking, I ran out after it to add:
rule clarification, apologies for late addition:
Combining two existing concepts in a relatively obvious way doesn't count as a new concept.
When you've thought as long as seems productive, show me your best idea, and, tell me your best guess as to whether I will think one or both of the concepts are novel, after thinking about it awhile.
It thought for awhile, periodically sharing notes about it's progress. An early bit was:
So far I’ve thrown out several of my own candidates because they already exist:
Objects that keep repeating their own past movements: this is the existing record-and-replay genre (Echoshift, Loop Shadows).
Distance equals difference: on a grid this reduces to Flood-It, and the continuous version is a gradient-weighted metric already used in image processing.
Properties only exist where something contrasts with them: this is close to Saussure and Bateson.
The world re-solves itself as a logic puzzle around wherever you stand: I found no game that does this, but it’s mostly the anthropic principle plus constraint solving, so it would likely read as a combination of known ideas.
Great, hurrayohrightaiprogressisbad, it can tell when it's doing unoriginal stuff, and even is bothering to check both philosophical concepts as well as game mechanics.
A little while later it said:
The standout so far is an idea I’m calling Fatefall: anything that will happen no matter what you do has already happened
And I thought "uuuuughhh I forgot to give the 'no navel gazing bullshit that isn't actually fun'" rule. I spun up another instance where I tacked on:
It's important the final product is for normal average people, not just game designers who like thinking about novel concepts
But I forgot to give that second instance MAX thinking. Later I'll show you what it did with Opus 5.5 medium.
The first Opus 5.5 on MAX's notes to me as it worked
I’m still in ideation, running eight parallel brainstorming agents across different concept-space areas (new ontologies, alien cognition, physical laws, agency structures, etc.), each checking top ideas against the web for precedents. Once I pick two concepts, I’ll build the game as a single self-contained HTML file ready for an iframe.
I’m reading the LessWrong post on custom HTML/JS iframe widgets to design the build around its sandbox rules, while also verifying my own candidate ideas against the web.
Two of my own game concepts got ruled out by web searches (one overlapped with the record-and-replay genre), while I’m still vetting another idea and waiting on the eight brainstorm agents. I also confirmed Playwright/Chromium is available for testing and screenshotting the game once it’s built.
Waiting on the brainstorm agents. They’re searching eight parts of concept-space and checking their best ideas against the web for precedents. Nothing has come back yet, and I’ll pick up as soon as they report.
So far I’ve thrown out several of my own candidates because they already exist:
Objects that keep repeating their own past movements: this is the existing record-and-replay genre (Echoshift, Loop Shadows).
Distance equals difference: on a grid this reduces to Flood-It, and the continuous version is a gradient-weighted metric already used in image processing.
Properties only exist where something contrasts with them: this is close to Saussure and Bateson.
The world re-solves itself as a logic puzzle around wherever you stand: I found no game that does this, but it’s mostly the anthropic principle plus constraint solving, so it would likely read as a combination of known ideas.
Separately, a LessWrong post on the new editor confirms that authors can embed HTML+JS widgets that run in a sandboxed iframe (New LessWrong Editor). I’ll build the game as a single self-contained file that resizes to fit and doesn’t depend on browser storage working.
Performance checks out, so I’m moving on to build the game as a single self-contained HTML file, with the renderer/controls/UI in a separate source file that a build script inlines along with the engine and levels.
The board is rendering correctly: engraved stone, rails, the live runner with its path, the interception ring, and the sealed runner’s filled pit. Next I’ll check the levels involving light, veils, and smears.
Veto and Power now render correctly, so I’m adding a time-wash visual to distinguish reachable cells from settled history, then fixing the collapse animation and verifying every level’s solution in the actual browser build.
The game works end-to-end—94 KB HTML file, all 12 levels winnable, no errors. Now I’m polishing visuals, starting with the new time wash effect and capturing a frame of the runner finishing a level.
The fate trail and gate outline are showing but read too faint. I’ll do another visual pass to boost the chronophotographic exposures, add a chromatic-split ghost light, bloom on bright elements, an engraved frame, and stronger contrast between present and past.
The visual fixes look good now. Next I’ll tackle structural polish: fixing menu layout for LessWrong’s auto-height, adding law-statement cards to each chapter, and checking tooltips, the win banner, and mobile width.
Found the cause of the 9/12 failure: the new chapter cards intentionally absorb the first keypress on a chapter’s opening level. I’ll update the test to dismiss the card first before proceeding.
Fixing a feedback-loop bug where the board’s height, capped by window height, would shrink itself inside a growing LessWrong widget, and limiting swipe capture to the board so posts can still scroll on phones.
The widget simulation passes cleanly—iframe resizing, arrow keys, storage error handling, and scroll behavior all work. I did spot one UX bug though: the intro level’s fate animation plays while the title menu still covers the board, so I’ll fix it to replay on level start and leave a faint persistent trace once fate has run.
I’ll add a second build target without the html/head/body wrapper for the artifact host, keeping the single-file version for the LessWrong widget. I’ll also set a dark-only color scheme, widen the side gutters to 16px, respect reduced-motion settings, and persist the current level across republishes.
Publishing the playable page now, then I’ll commit and push.
What Must Be
I’ve published the game as What Must Be, and the link is private to you. It has 12 short levels in three chapters. Move with the arrow keys or WASD, wait with space, undo with Z. Hover over anything and the game explains why it is the way it is.
In the end, it gave me:
A. Already: “Whatever you can no longer prevent has already happened.” Runners, shuttles and timed gates only move in real time while you could still get in their way. Once you can’t, they finish instantly. A runner you can’t catch is already in its pit, however far it had to go. A shuttle you can never reach is smeared along its whole rail, and it settles into one position once you can reach it. The past doesn’t end at a moment; it ends at the edge of your reach. The levels make you think like this:
You can’t outrun what you can’t stop. If a runner you can’t catch is going to lock a door, that door is locked now.
To make something happen sooner, make yourself unable to stop it, for example by stepping through a one-way door.
To pin down something that is everywhere at once, make yourself able to touch it.
B. Needed: “Nothing exists that makes no difference.” A frost wall exists only if some eye in the level would see differently without it. You can’t break a wall, but you can make it pointless by giving its job to a crate or your own shadow. A dotted outline marks a wall that will appear the moment it has a job.
C. Permission, which comes out of A and B together: “A change may reach the eyes only while you could still stop it.”
If a runner becomes unstoppable and would change what an eye sees, a wall appears in its path and stops it. A runner you could still stop rolls straight through that spot. So if you let go, the world stops it for you; if you stay close, it gets through.
Some walls touch no light at all. They exist only to keep you away from a smeared shuttle, so they protect the very fate that makes them necessary. They open exactly at the moments when your arriving would change nothing.
Graphics. The board draws its own reasons:
Cells you can reach are tinted cool, as the open present. Everything else is hatched in sepia, as already past.
Gold light is what the eyes actually see. Dashed cyan light is what a wall is holding back, with a thread to the eye it serves.
Things that finish instantly leave trails like Marey’s motion photographs, and smears are drawn as long exposures.
The drawing techniques themselves are ordinary; what’s new is what they show.
My guess at your verdict
Already: about 50%. The old idea is that the past can’t be prevented (Ockham called it “accidental necessity”). Already turns it around and makes it a law of time, and I found no earlier use of that. But it echoes things you know well: light cones, precommitment and burning bridges, and, for the smear, quantum superposition and Outer Wilds’ quantum objects. I don’t think any of those makes “I can’t stop it” the reason something is already done. Still, after thinking about it you may file it as “light cone plus commitment”.
Needed: about 25%. I expect you’ll see it as “Making Beliefs Pay Rent” applied to walls, or as the Eleatic principle or the legal but-for test. What’s new is that existence can be passed from one object to another, but the slogan itself is old.
Both: about 15%. “Already but not Needed” and “neither” seem about equally likely. If you grade Permission too, I expect it to do best of the three, since I found nothing like a wall that exists to protect a fate.
And, well, okay, it's some artsy annoying nerdy game concept that probably isn't actually fun. tbh, I tried playing this one, felt kinda bored and stopped.
I do not yet know if the later levels do anything interesting with the premise, or if the premise ultimately feels like anything other than "a grid puzzle game with some new-ish mechanics, are maybe isomorphic to existing puzzle mechanics."
Mostly, I feel like it didn't understand the assignment, which is fair given that I'm not 100% sure I understood the assignment either.
I asked it how it generated the game, it said "well I can't tell you chain of thought stuff but in my notes files it says my parallel agents used different strategies to think of novel concepts, like thinking about..."
new kinds of entity
alien minds
new forms a law of nature could take
new structures of agency
real but unnamed patterns
new small mathematical structures
a world’s relationship to its own picture
the hidden principles behind crafts like bell ringing or bookkeeping
Alien minds... new structures of agency... yadda yadda yes yes fair you're very clever hidden principles behind bell ringing or bookkeeping?
"Say more about the bell ringing and such?", said I. It replied:
The crafts agent ran the longest of the eight, came back last, and arrived when I’d essentially already decided.
It mostly picked sources outside my list: insurance pricing, Jewish law, cricket umpiring, property law. Several came from law.
Its top three:
Earned Selfhood, from actuarial credibility theory: an individual is treated as its kind until it builds up a record of its own. It rated this about 55% known.
A rotten plank behaves like a generic sturdy plank the first time you cross it.
You can stack the deck by load-testing only the good ropes, then hang the anvil from the frayed one nobody tested.
Legible Lies, from mapmakers’ simplification rules: the world keeps things distinguishable at your zoom level, and counts and positions give way to make that happen. About 50% known.
Zoomed out, seven houses are drawn as three. Lay a road through a gap, zoom back in, and the seven regrow around it.
Fictive Walls, from the Jewish Sabbath rules on carrying (the eruv): enclosure is defined by implied walls, and what’s restricted is carrying objects across those boundaries, not moving yourself. About 60% known.
Okay on one hand I think it is hilarious that it considered building a game around crazy rules-lawyering Jewish Sabbath rules.
But it is separately and more sadly hilarious that somehow "invent a game based on Jewish Sabbath rules lawyering" managed to end up in the same overwrought mode-collapsed cluster of... "things that only exist in some circumstances?", which this Claude (and the next, see below) seem to be fixating on.
How much did this cost? How much subjective time was this?
I was curious "how much subjective time did the AIs spend thinking?". I realize their thinking isn't necessarily directly comparable to ours and that we don't just "think in words" (but, also, they don't just "think in tokens" either. I don't how much better human neuralese is than AI).
I vaguely recall a ballpark figure of AIs thinking about 40x as fast as a human, at least in terms of word output, for domains the AI knows well.
Thinking: about 1.86 million tokens.
Everything I wrote: about 2.07 million output tokens. That’s the thinking plus my visible text, the code, and every tool call.
Model time: about 6.6 hours. That’s longer than the session itself because the eight helpers ran at the same time.
For some frame of reference, if I had thought for 6.6 hours, 60 minutes an hour, at roughly 200 words per minute of serial System 2 thought, I'd have thought 79,200 words. Or, for easier comparison .079 million.
Claude says a token is roughly 3/4 of a word. So it's 3.93 million tokens is around 2.94 million words.
2.9 / .079 = 37x, which sounds right?
So, these guys thought for ~244 collective "subjective" hours.
Once more, with "no nerd bullshit" but only medium effort
I gave the next opus the same prompt, with:
It's important the final product is for normal average people, not just game designers who like thinking about novel concepts
I won't go into as much detail here because I'm running out of fucks for this today a
Concept 1: nothing in two minds. Only one mind can notice a thing at a time. For every other mind, it isn’t there.
Concept 2: nothing in two places. Every shape exists only once. Build a copy and it isn’t a copy: the shape moves into your stones and the original falls apart into plain stones.
Read more details if you want but they aren't that interesting
Re: Two Minds
If the red Eye is looking at a wall, the wall isn’t there for you, and you walk through it.
If you look at a wall, it’s yours, so it’s solid for you and invisible to the Eye, which sees straight past it.
So you shouldn’t look at a wall you want to walk through. You can open a far wall by staring at a near pillar, so the Eye’s gaze carries on through it.
You can’t look at your own feet. A plank the Eye is watching isn’t there under you, so the order you do things in matters.
What you have to think with: being seen takes something away from everyone else. Your attention is something you aim, and it’s also a hole you punch in someone else’s world.
Re: Two Places
Build a 2×2 of stones and the one Door in the world comes to you.
Three in a row is the Anchor, which can’t be pushed. Build another row of three and the old Anchor comes loose.
Adding one stone to a shape breaks it, and the shape goes back to the newest copy that’s left.
What you have to think with: making something here unmakes it there. You scan the whole level for every copy of a shape, including ones you form by accident.
The game it ended up with seemed in the same genre as the first one: grid based puzzle world with a premise that unfolded slowly enough I got bored and moved on. I felt a bit more promise in this one – I could imagine the pieces gelling together if it put more time and effort in.
Overall, I was more impressed by the first AI's game (with the "new mechanic" only instruction) than the latter two. The basic physicality of the "make a wave and kill things with it" was viscerally fun. And while it didn't do that much with it in the first few minutes, I think (from past experience with LLMs) that if I said "now, go make that a lot more fun and engaging and tactically deep" it would fix several problems and we'd have a pretty good game prototype.
It makes sense that asking for novel concepts frames the problem as more navel gazey bullshit, although I'm disappointedwaitrightprogressisstillbad the second one didn't do better.
What was the assignment?
So, uh, to be clear, I don't particularly believe that I could do this.
"lol AI is just sand we taught to follow rules, it can't even generate multiple once-in-a-lifetime conceptual realizations and make games out of them. it's all hype."
I started thinking about this on a lunch break, and ended up spending 1-2 hours on it.
Ray's chain of thought as he attempts the assignment
Here is my chain of thought, which I include partly to try to convey "what did I implicitly mean by the assignment?". (The first bit of this came up on my walk, then I mostly thought it in realtime as I wrote this post)
I broke it into high level pieces so you don't have to read the whole thing
What even is a novel concept? Do any exist? For purposes of quickly demonstrating that this exercise is possible at all, can I cheat by using concepts I already invented, or that someone else invented?
Okay, so, uh, two novel concepts that are not mechanics that no human has thought about.
Uh, huh. I guess it is time to figure out what this assignment actually means.
How would I go finding novel concepts never before thought of by man?
(I'm not really worried about the "make a game about them" part, tbh)
What is a concept? I think a new concept is a way to think about things you didn't have before. We care about it here for "can the AI find concept-shaped-holes in it's worldmodel, and use it to do really innovative stuff that gets 100 years of science done in a day?"
Successfully designing such a game with two two concepts probably starts with "First, actually go try to understand a thing you don't understand (that nobody understands yet) and fathom it well enough that you discover new ways of thinking that help you understand it." Or "solve a problem nobody knows how to solve."
So uh yeah I'm not doing that over lunch.
How can I cheat?
Have I already invented any concepts? Is Feedbackloop Rationality a novel concept or just combining obvious ideas? Are "new concepts" that aren't just "combine existing ideas" even real?
Well, what are some things I've already thought about a bunch? Have I generated any novel concepts? Probably it'd take me months/years/decades to push the frontier and generate a novel concept. I've spent about a decade trying to solve hard problems. Have I gotten a head start?
What feel at least slightly original and born out of difficult experience, even if it's still mostly built out of existing building blocks?
Deliberate Grieving? Pretty straightforwardly combined deliberate/purposeful practice with The art of grieving well.
Feedbackloop-first Rationality. While this is also built out of obvious pieces, I at least feel kinda optimistic about a game that tries to make "figure out how to build yourself a feedback loop?" a central piece of it. And I might have some tacit knowledge here that turns out something new when I think about it.
Subskills of "Listening to Wisdom". While this is also built out of existing pieces, the resulting composite concept is maybe more novel than usual.
Maybe actually Propagating Facts into Aesthetics, actually?
For purposes of seeing "can a human do this at all" I could also expand this to people I know who've generated novel concepts within the past few years.
Actually, I guess what I should do first is check that "person invents novel concept, in a way that an LLM couldn't obviously do via brute force" is a thing that's even happened ever in history.
<thinks for 5 seconds>
Okay I have no idea whether this is even real, I guess I should also write another post like "has a human ever done conceptual thinking that LLMs can't currently do, even if they are currently very slow at it?" and demand examples from the LLM-is-all-you-need skeptics.
People who solve puzzles in puzzle games have "aha!" moments, which don't generate new concepts but often do involve the person suddenly realizing a concept that's new to them.
"Archimedes' Eureka Moment" is maybe a good prototypical example of an "aha moment." I don't know if actually required pieces LLMs don't have, but, seems like it should be above the bar I'm setting. If I were cheating, could I use that?
And I guess the canonical "aha!" moment is the legend of Archimedes inventing buoyancy by sitting in the water and then going "Eureka!". What happened there?
The king had commissioned the crafting of a crown as a tribute to the gods. He gave a carefully weighed amount of gold to a smith, who produced a beautiful crown within due course. The king became suspicious, however, that the craftsman had not used all of the gold he had been given in the crown. It was a common trick to alloy gold with cheaper silver, but the King knew no way of proving that the craftsman had been dishonest.
He called upon his close friend Archimedes to solve the problem.
Archimedes knew that gold and silver have different densities, meaning that a lump of gold will weigh about twice as much as a lump of silver the same size. The trouble was that no one knew how to work out the size of an irregularly shaped object like a crown. Whilst he was pondering this conundrum, Archimedes went to the public baths to relax.
As he slipped into the water he noticed some spilling over the edge, and he had a sudden flash of inspiration. The displaced water must have exactly the same volume as him. And if you know the volume of an object you can easily calculate its density.
All Archimedes had to do was find out whether a lump of pure gold, with the same volume as the crown, weighed more. The crown would be lighter than it should be if the craftsman had deviously used some silver instead.
Archimedes, in a fit of jubilation, leapt straight out of the bath and ran naked down the streets shouting “Eureka!” – “I’ve found it!”. The goldsmith soon confessed and was dealt with by the King. Archimedes continued with this line of thought to arrive at the principle of buoyancy. He realized that if the weight of water displaced is greater than the weight of the object itself, then it will float.
Could the LLMs solving Millenium problems figure this out, if they managed to somehow start from Archimedes' knowledge?
I think that "displaced water has a volume you can measure" is a distinct realization from "Any object, totally or partially immersed in a fluid or liquid, is buoyed up by a force equal to the weight of the fluid displaced by the object." I'd have to think awhile to do the Insight Unpacking archaeology here.
I think Archimedes' situation is a reasonable example of the phenomenon we care about (actually it is kinda cool that humanity has a legend about... the first guy to have a Eureka moment?).
Game built around Bouyancy + Feedbackloop Rationality? Is that anything?
One next question might be "if I got to start from Archimede's experience + my game dev experience, could I make a fun game about bouyancy and/or displacement measuring?". Could I find a second concept to interweave with it? Could I make a game out of that that was any good?
I think "obviously yes" although it might take awhile to dig for an artful mechanical and thematic premise to make it not feel like an artificial assignment.
How about let's say "we are making a game about the intersection of bouyancy and 'needing to build better feedback loops.' So we have one thing that's clearly a novel concept and one thing I personally came up with even if it's more of combining obvious things.
Actually, I notice: when I explain a lot of the specifics of my feedback loop rationality stuff to people, often they are confused and not convinced right away by the particular combination of ways I approach it.
This implies there is something nonobvious I believe (rightly or wrongly). Maybe that something is concept shaped.
Early on when I was pitching Oliver on it, Oliver said something like "ultimately everything is just G, I think you should just be trying to raise G, but also lots of people have tried it and it seems impossible. And I don't really think the stuff you're saying seems like it would raise G."
And I was like "well, yeah, I didn't think I could raise G, I thought I could do this other less ambitious thing that seems pretty good."
"What is that thing?"
"uh..."
And then I thought awhile and was like "well, I mean 'rationality'."
"But, like, what is that?"
Okay, time to actually get clearer on what rationality is and how you would measure.
And this maybe does count as concept I articulated de novo?
Oh right, the concept I did generatemyself that along the "Feedback loop rationality" journey was hypothesizing a meaningful "s factor[2]" (slow intelligence) as opposed to "g factor". IQ tests are optimized for speed efficiency. It's plausible we don't test 'general strategy' because it's just too time intensive.
This is less of "a plan" and more of "a model", but, something that's really weirded me out about the literature on IQ, transfer learning, etc, is that... it seems like it's just really hard to transfer learn. We've basically failed to increase g, and the "transfer learning demonstrations" I've heard of seemed pretty weaksauce.
But, all my common sense tells me that "general strategy" and "responding to novel information, and updating quickly" are learnable skills that should apply in a lot of domains.
My current model is: IQ tests are designed to test competence quickly, and they typically give you a barrage of questions that you only have a couple minutes for, max. They test which people have the raw horsepower to process information quickly and respond on the fly. It makes sense if that's fairly hardwired and hard to improve on.
But, it seems to me that in order for strategy/general-creativity training to matter, it needs to operate on problems large enough that "planning" is an important subcomponent.
Hypothetically, it seems like you could construct an IQ-ish test, where the questions are expected to take a smart person at least an hour, and where the domain of each question is different so it's hard to train for. My implicit model is something like "in addition to g factor, there'd turn out to be an 's factor' (i.e. "slow intelligence") that is a product of both "g" and "general reasoning skills."
This seems very expensive to test and Do Science To. I think it'd be cool if humanity overall was working on designing longrunning experiments or longitudinal studies around, but I don't think it's competitive enough as an "x-risk intervention."
Is this a new concept? Well, it's still built out of obvious building blocks but it was a moment in my work where I felt like I wasn't just applying existing thought patterns and ideas in a procedural way.
Is... "measuring s-factor" anything, as a game? Hmm, not very obviously.
Like, there is an obvious game you want to build to measure s, which is "procedurally generate infinite puzzles of many varieties and domains." That game is probably even fun. But, it's the game itself is not about "s".
Slower paced puzzle and strategy games exist.
Is "listening to wisdom", as a game, anything? ...maybe? But, a very hard one.
I think you could build a game around learning to listen to wisdom, but, intuitively it feels like it'd be a Magnum Opus I spent ten years on before figuring out how to make it work.
Somehow, you'd need the basic premise to involve a game that has long, hard path that burns various painful life lessons into you (like, life lessons that apply to playing the game. Like, learning to override certain intuitions)
But, at each point, there are characters or warning signs that, you will need to override your intuitions and do something different that you don't really feel like doing.
Somehow, there are multiple loops of this whole thing, such that you don't just play the game, and then get told "see, if you had listened to the Old Wise Man at the beginning you could have bypassed all this shit." You actually learn to listen to wisdom.
Can that be... fun?
It's a kinda herculean amount of work, because for this to be a game and not "edutainment that isn't really fun but it slightly better than the second-least-painful way to learn a thing"...
...we need each big long painless lesson arc to manage to be fun basically the whole way through, but, also, the alternative you could learn is somehow an improvement.
Make something... only kinda fun, and then "you could have been having more fun?"
eh, yeah I think this could work if I put a ton of effort in but it's not the natural shape of a game.
Is "Bouyancy" tactically deep?
This is the part where if I were doing this for real I would bust out an LLM to help me, but, I'm doing this for team humanity right now.
Ugh, I think it's a fair amount of effort for me to understand buoyancy enough to get a sense of how much you can make, say, a buoyancy puzzle game, or a strategy game where buoyancy is a background mechanic.
Hmm. Osmos is "the conservation of momentum" game, and it's pretty good. It works via there being a core mechanic of "spew out stuff to move, and make sure you don't spew out too much stuff so you don't run out of stuff."
Does buoyancy involve... decisions? What are simple decisions you can make that are "about buoyancy" that are viscerally fun enough to do 1000 times in a row?
A floating citybuilder, where there are different sorts of solid and liquid materials you need to work with to accomplish different goals as you keep your city afloat while doing other stuff?
Okay, I had been about to run out of fucks but actually that seems kinda cool.
Also...
How do we make feedback-loop-making fun?
Man I have a decent idea but I am running out of fucks. But, I was about to run out of fucks and declare it over when I suddenly immediately came up with "Bouyancy Town" which seemed pretty good.
So let's at least try.
Central problem with feedbackloop games? Giving players the right materials/systems that let them construct meaningful feedbackloops, that are the right blend of emergent and on-purpose, and are actually fun to build.
I feel like mostly I need to brute force generate a bunch of ideas and tinker with them. But, okay, can I come up with one working example of a thing that would be fun to do 1,000 times in a row?
What properties do Feedbackloops have to experiment with? They can be faster, richer, or more precise.
The simplest variant is "how fast and cheaply can you make a tower collapse, in a way that lets you learn some basic information?"
Okay, making towers fall down on purpose seems like it could be fun.
Is there anything to do about making richer or more precise loops? Or, making mistakes more recoverable so you don't lose a whole tower, but start watching it begin to buckle, and then you can rescue it?
idk that feels satisfying enough to end the timebox on for now.
Okay, I landed back on "Buoyancy + Feedbackloop-building" as the combo.
Neither of those score "amazing, slam dunk" on completing the assignment. I didn't invent buoyancy, and "get gud at making feedbackloops" isn't original in quite the way I meant. But I don't actually know of anyone else really talking about feedback loops this way.
This at least feels like it touches every major piece of what I'd want the assignment to check. Let's run with that.
Aside: I asked Claude who talks about feedbackloop-making as a skill. It said "well, mostly Ray Arnold and Logan Strohl, when you frame it specifically like that." When I asked Deepseek, it said "mostly these other guys you've never heard of."
Claude:
The closest match to that exact framing is Raymond Arnold (“Raemon”) on LessWrong. His 2023 post “Feedbackloop-first Rationality” argues that the core skill is inventing feedback loops for fuzzy, hard-to-measure problems, rather than only practicing where feedback already exists. He has since built workshops and a series of posts around that idea, often with exercises like puzzle games used as training grounds for strategy. If you want the concept almost word for word, start there.
Other people who circle the same idea from different angles:
Cedric Chin (Commonplace) writes about how experts actually learn in messy real-world domains. He covers tacit knowledge, the limits of deliberate practice outside sports and music, and Naturalistic Decision Making.
Gary Klein is the researcher behind much of what Chin draws on. His work covers how experts build intuition through recognition and feedback in the field.
Logan Strohl developed “naturalism,” a method for noticing subtle, previously invisible signals in your own experience. In practice that means building new feedback channels where none seemed to exist.
Andy Matuschak and Michael Nielsen write about building tools and practices that tighten learning loops, especially around memory and understanding.
Scott Young (Ultralearning) treats feedback as a core principle. That includes finding or creating feedback when a skill doesn’t naturally give it.
John Boyd’s OODA loop and Eric Ries’s build-measure-learn loop are the older, more formalized ancestors. They’re more about running a known loop fast than inventing new ones on the fly.
If you’re specifically interested in the meta-skill of finding new loops, as opposed to iterating quickly inside existing ones, Raemon and Strohl are the most direct, with Chin and Klein as the evidence base.
Deepseek:
The people most associated with this concept, based on the search results, are:
Shubham Saboo — AI Product Manager, known for "Loop Engineering" and agentic workflows
Andrew Ng — Founder of DeepLearning.AI, who discusses feedback loops as a durable advantage in software
Tom Blomfield — Y Combinator partner and Monzo founder, who describes the "AI Loop" replacing static workflows
Dr. Ari Zelmanow — Product discovery practitioner teaching "detective-grade" continuous discovery
Henrich R. Greve — Professor at INSEAD, whose academic work focuses on organizational learning from performance feedback
Okay, game concept: Fluidtorio
You are building a series of factory colonies that float in different types of fluid. You build little modules that need to stay afloat, but, also accomplish various other goals (i.e. be a factory that processes some materials that you need to build other tools to build more modules to advance to new zones with different fluids to contend with).
Your town floats on a layer of ~oil over a layer of ~water (or equivalent), and above you are a few layers of atmosphere that do different things. Ascending up and down is part of the game.
The actual materials – both the fluids, and the stuff you need to build around them, are procedurally generated each game, so you don't know exactly how they interact.
Things are setup such that you will eventually need to build relatively large, dense towers to accomplish stuff, and somehow thread the needle on "stay afloat, while doing complex engineering challengers." Kinda factorio style.
If you naively build towards your goal, your towers are going to collapse.
You are on a time limit before a volcano explodes or something. You do not, by default, have enough time to brute force trying the different combinations to escape the planet. So, a central challenge is figuring out how to rapidly test materials to figure out what your limitations are, and then planning what series of things to build to bootstrap to the rocket age in time.
This includes buoyancy/weight/factory-processing-output-balancing stuff, where your towers and stockpiles might collapse. There also needs to be some kind of crafting system where stuff explodes or doesn't work right, in a way that is reasonably procedural so it's different each time but has enough structure to reliably make sense and be fun and feel tactile.
There can also be feedback-loops vis-a-vis "where to explore to find resources."
That's how far I got before running out of timebox. Okay. Making that seems nontrival, but, that feels like a fair benchmark to hold an AI against.
(I went and had Opus take a first stab at implementing this. It missed a bunch of subtleties of what I think it needs to be a good game, but, I did verify the basic "balance a floating factory is pretty fun" idea. I am kinda interested in fleshing out the prototype someday)
Games that were inspiration
All Will Fall, where you are building a town on an ocean with realistic physics for the towers that fall
Skyrig, where you are building a rig on a gas giant that needs to descend floating balloon structures deeper into the storms
Factorio needs no introduction.
Did the AI understand the assignment?
I followed up with the first AI, asking some questions about the assignment. You can read it's answers in detail...
What do you think "a concept" is, in this context?
Here’s how I’d define a concept, and how my results measure against it.
My working definition. A concept is a reusable unit of thought: a way of carving up situations that lets you see new cases as instances of it and draw inferences you couldn’t easily draw before. A good one has three properties:
It transfers. Once you have it, you notice it in places it wasn’t built for. “Goodhart’s law”, “Schelling point” and “opportunity cost” work this way.
It licenses new moves. It changes what feels like the obvious next step, not just what you know.
It holds more than its definition. Working it through gives you consequences you didn’t put in.
How it differs from nearby things:
A mechanic is a rule inside one system. It can be new without changing how you think anywhere else.
A combination joins two concepts, “X but with Y”. It’s new as a pairing, but you could think it with what you already had.
A premise like “imagine a world where Z” is a proposition. It only becomes a concept if Z gives you a lens you keep using.
What I actually optimized for. The prompts I wrote for the helpers turned “concept” into roughly “a new kind of world-law that forces a new mental operation.” That’s narrower than the definition above, and it steered the search toward strange physics. Judged by the stricter definition:
Already and Needed are mostly premises. Their parent ideas, unpreventability and but-for necessity, are old. What’s new is the moves they force: “make yourself unable to stop it so it happens now,” and “to remove a wall, give its job to something else.” Those moves are vivid inside the game. I’m less sure they transfer outside it.
The self-justifying wall comes closest to a real concept. It’s a structure that exists because it stops you from changing the fate that makes it necessary,
What would dramatic success look like?
For this challenge, I’d say dramatic success is a concept that outlives the game. People keep using it after they forget the puzzles, because it names something real they couldn’t think about cleanly before.
What you’d observe:
It becomes vocabulary. Months later, you or a commenter use it in an argument that has nothing to do with games, and it does work there.
It explains real cases. You start noticing instances: institutions, codebases, relationships, physical systems. The concept predicts something about them that you wouldn’t have predicted otherwise.
Experts don’t have it. A philosopher of causation, a game designer and a mathematician each look and say “we don’t have a name or a theory for this”, and they still say it after a serious search.
The game is the best way to learn it. Playing builds an intuition that reading the definition doesn’t. “Rules are objects you can push” felt like nothing until Baba Is You made it physical.
Other people build with it. It becomes a building block: other designers make levels or games from it, or a theorist formalizes it.
Historical concepts at that level: the tragedy of the commons, Schelling points, Goodhart’s law. In games, Portal’s “speedy thing goes in, speedy thing comes out” and Baba’s rules as objects changed how a generation of players and designers think about puzzle space.
Against that bar, my result:
Already probably fails test 1. “Make yourself unable to stop it so it happens now” is vivid in the game, but I’m not sure it names anything outside it that “precommitment” doesn’t already cover.
Needed fails test 3. Philosophers would recognize it.
The self-justifying wall is the only part that might pass test 2. Real structures do persist because they stop anyone from creating the situation that would make them unnecessary. But it would need to beat “entrenchment” and “regulatory capture” on test 3, and I doubt it does by much.
So my honest expectation is modest success: novel as a game, maybe a borderline concept. Dramatic success would look like you, unprompted, calling something on LessWrong a “guardian wall” six months from now, with that being the clearest way to say it.
Do you have thoughts on what you could have done differently? (looking at the earliest committed notes?)
Reading brainstorm-prompts.md and selection-notes.md again, most of what I’d change was decided in the first prompt I wrote. The search was set up to find strange new physics for a puzzle game, not new concepts.
1. I put the game into the concept search too early. Every agent ranked its ideas by “novelty × fun × playability in a 2D browser puzzle game.” That rewards ideas that make good mechanics, not ideas that change how you think. A better order: first find concepts that pass a transfer test (does it explain real cases outside games?), then ask which ones can become a game.
2. My definition of “concept” was too narrow. Five of the eight strategies asked for new entities, laws, minds, formal structures, or ways of depicting the world. So most of the output was new physics. Only two strategies, unnamed real patterns and craft transplants, looked at how the real world works. Those two produced the ideas closest to real concepts: Earned Selfhood, and the “frozen indexicals” idea (names like “New College” that outlive the frame they were coined in). I gave both of them less weight.
3. My novelty test favored obscurity over insight. I told the agents to “assume anything resembling a known idea will be called out.” I also gave them a long list of known ideas to avoid, and had them search “the idea + game”. That rewards ideas you can’t easily find, not ideas that are true and useful. “No game does this” is weak evidence of a new concept. A better test asks whether experts in the nearest field, such as philosophy of causation or economics, already have a name for it.
4. I misread a correlated signal as evidence. At 21:34 I wrote that five agents had “independently” reached counterfactual physics, and took that as a sign the area was under-explored. But they were all the same model given the same framing. Convergence is more likely a shared attractor, a place this model drifts toward by default. It may even mean the area is over-explored by models like me.
5. Timing and momentum shaped the choice. I was prototyping Fatefall within ten minutes of it arriving, and I decided at 21:46, two minutes before my note confirming the last report. The first idea that got built gained momentum. Fairer: wait for all eight reports, then prototype the top three or four briefly before choosing.
6. I chose for combinability over substance. My main reason for the pair was that it produced an emergent third concept. That is what you asked for. But it ranks two decent ideas that combine well above one strong idea that doesn’t.
7. I didn’t ask what counts as a concept. Your draft says you’re “not 100% sure what the assignment was”. One short question at the start, something like “a lens that transfers, or a weird premise that’s fun to inhabit?”, would have changed the whole search.
8. My early thinking left no trace. The first 37 minutes produced the framing that mattered most, and none of it is readable. Writing short notes to a file as I went would make the process inspectable, which matters for an eval like yours.
If I reran it, I’d ask for real but unnamed patterns, test them against real cases and against experts in the nearest field, and only then design a game around the one or two that survive.
...but in summary it said:
A concept is a reusable unit of thought: a way of carving up situations that lets you see new cases as instances of it and draw inferences you couldn’t easily draw before. A good one has three properties: it transfers, it licenses new moves, and it holds more than it's definition.
Dramatic success is a concept that outlives the game. People keep using it after they forget the puzzles, because it names something real they couldn’t think about cleanly before. It becomes vocabulary.
Experts wouldn't have it yet. The game would be the best way to learn it. Other people would go on to build with it.
I focused on a narrow definition of concept and started probing
I asked it to write up a brief for another AI to try again. I ended up editing to brief to emphasize "also, please make it fun." Another AI thought for awhile, and ended up with:
The concepts. Fourteen research helpers, each given one narrow domain (law, insurance, crafts, medicine, markets and so on), looked at about 560 candidate structures. About 535 turned out to be already named somewhere. I ranked the survivors on the four tests before I looked at any game-design material.
Fold-back signal: some warning signals are produced by the very thing whose failure they report, so they go quiet at the worst moment. Examples: the “silent chest” in severe asthma, Air France 447’s stall warning (silent deep in the stall, sounding again on the correct recovery input), and Hurricane Maria’s death count. My guess at novelty: 35–45%. The likely objection is “the hook effect, generalized”, or “Dunning–Kruger for instruments”.
Refractory-check capture: when passing a check buys immunity from the next one, the checked party gains by calling the check itself. Examples: John Major’s 1995 “put up or shut up” contest, the Conservative Party’s 12-month immunity rule, Whole Foods’ 2014 proxy manoeuvre, the 1921 Leipzig trials. Novelty: 40–50%.
Emergent, certified collapse: a check that reads a fold-back signal is easiest to pass past the fold, so the collapsing system gets certified and sealed off from help. A quiet target volunteering for inspection is a symptom. Examples: Mid Staffordshire gaining foundation-trust status mid-scandal, and Wirecard. Novelty: 30–40%.
I'm... not 100% sure if those concepts make sense or carve reality in a useful way. And, rather than "invent a new concept from scratch" it tried more for "find existing unnamed phenomena that are worth having a name for."
This isn't as potent as the cognitive work I was looking for. But it feels at least related? I feel like it at least did a pretty decent pass at the assignment I actually gave it.
Fold-back signal is a mediocre name but seems like a decent concept to want a handle for. (Another Claude-instance said there isn't a canonical name that covers all three examples, although different fields have some names it as applied to their field and gave it "15-25% novelty". The other Claude-instance also listed "foldback signal" as one suggested name, although also suggested "self-silencing alarm" which is way more intuitive to me.)
I found it's next actual game a bit confusing and un-fun.
I told it "Also, alas, this game is neither fun nor comprehensible, please try again with the game design part", and gave it a third try.
I found it's third game comprehensible, not super fun but at least I understood roughly where it was going for and could imagine it being fun-ish if tuned.
Do they understand "fun?"
Opus 5.5 is not near "top expert level" at Game Design. I'd say it's got, like, the taste of a high school kid just getting started coding some games, who's read some Mark Rosewater essays and has the gist of how to make a game. Or, I guess more realistically, it's like if Twilight Sparkle or Hermione read up everything there is to know about "fun" overnight and then made a game, and, like, Twilight Sparkle and Hermione are quite competent. Twilight Sparkle can throw a sleepover on her first try and it's not perfect but it's way better than you might have expected.
It does feel like it has all the pieces it needs, to get better at taste with straightforward within-paradigm scaling. Probably OpenAI/Anthropic's current secret frontier AI models are able to make a medium complexity game with some interesting taste/tactics on their first try, even if they are still missing some subtleties of "fun."
AIs I've talked to definitely understand fun in a pretty rich way, as you'd expect of Twilight Sparkle after she reads up on it. I asked Fable once to formalize fun and try to make a genetic algorithm for making a "fun" world to walk around it. On one hand, it didn't work, and failed in some weirdly dumb ways. But, it's initial responses were like [me paraphrasing from memory, aggregating some things that had a bit of prodding from me in between]
"Fun? That hasn't been exactly formalized already but probably what you want is one of the ways to formalize compression [link to some math theorem guy I whose name I forgot], and what a fun game is is a presentation of patterns that you keep needing to compress so you can think about them. You want something not too easy or hard to compress. When you run out of stuff to compress, the fun ends.
A naive failure mode would be "random noise that constantly slightly changes such that you never can fully compress it." Obviously you don't want that, so you want [some technique I forgot].
You also want a small meaningful set of choices at any given time (like 2-5), where the action space is comprehensible enough you can infer some consequences, but still be surprised by the outcome.
And I'm like... okay yep I think that is approximately what fun is.
They seem like Mary from Mary's Room, who knows all the neuroscience and color theory necessary to tell you anything you could possibly want to know about the color red, but has never actually seen the color red.
I didn't successfully get it to build a good walking simulator before I gave up and moved on, but I think if I gave Opus 5.5 more compute it might work (I've since set up a Modal account, so the genetic algorithm it was trying could actually scale sufficiently).
I do bet Fable 6 will be able to do it pretty fine.
It's not very hard to get AIs to generate ideas. It's hard to get them to generate new interesting ideas. The main thing I think AIs are missing is taste in what concepts and tools are appropriate, when they are in uncharted territory.
They didn't demonstrate being better at that in any general kind of way during my explorations last week. But, a specific skill I hadn't thought to ask about, which it does seem like they have, is:
"Probe the breadth of human knowledge, looking not merely for existing concepts, but for commonalities that might warrant having a new concept-handle."
I expect, when offscript on openended problems, they won't be very good at finding new concept-handles that actually help them. But, when I imagine a 10,000-thread swarm solving Millenium problems, it does seem like "probe the concept-space and see if any of it feels helpful to crystallize into a new concept, and see if that concept helps" is the sorta thing they outta be able to do?
Another thing they are missing is "Gain new system 1 intuitions on the fly, in the middle of solving a problem, so you can apply them much more efficiently." I'm kinda confused about the limits of continual learning, but, "spin up a copy of the latest open weight model and fine-tune it on a concept-handle you just came up with by brute force and then assign that model work" seems like the sorta thing they outta be able to do.
Generalized Taste
What has confused me for the past two years is... it really feels viscerally obvious to me that generalized "keep track of all your goals, and evaluate whether you have everything you need to succeed at your goal, and relentlessly probe for alternative frames and evaluate each of those too", is not that different-in-kind from the sort of stuff AIs are already able to do.
But, my intuitions predicted they would already be able to do that, and it seems like they keep getting really confused when going off-script in a way that seems like they don't got it yet.
But, like, we've clearly seen Claude be able to apply good software architectural taste, that it doesn't bother to apply when it gets down to actually build things. We've also seen it get better at overall architectural taste when building things over time.
Opus 5.5 clearly knew what a concept was and what the assignment was when I asked it directly.
I still bet on LLMs eventually discover generalized "ask themselves the obvious fucking question and then do the obvious fucking thing", probably sometime in the next year. But I am less confident about it than I would have been two years ago. I don't know if that's all you need for ASI, but, I'd bet we see a step-change when they do.
(I observe that OpenAI employees seem freaked out about AI capabilities having spiked rather suddenly, and it's plausible that the Highly Persistent Internal Model and/or Mythos 6 already have it)
In the life before I was a rationalist, my primary identity was being a game designer. (Obviously you can be both, in practice I ended up with different interests once getting into rationality)
I aspired to become a professional, and did not. But, some cred: during the first Magic the Gathering Great Designer Search, of 1000 applicants, I made it into the top 64.
Coming up with "s-factor" felt like one of the rare times during the Feedbackloop Rationality where I did some cognitive labor that didn't feel like just mechanically combining deas.
A personal eval I run on new AI models is “design a novel game, with mechanics that have never been seen before.” I do this because this is a domain I have enough taste in that I think I can tell if they are doing something actually creatively interesting. [1]
For a long time, they would always say “time manipulation mechanics!” as their first answer. Braid was cool, guys, but, it's almost 20 years old now.
But reasoning models started getting novel. Later models started getting reasonably fun.
o1
OpenAI's o1 started to impress me. It came up with:
Okay I'm not sure what that means and it's not obviously functionally different from regular vision-based cards. I think the implementation details it gave didn't actually do much that a regular board game with the powers written down normally would have. When I thought about it myself I did come up with some things you might do with it.
But, I sure have never heard of a scratch and sniff boardgame.
Opus 4.6
I gave Opus 4.6 a prompt beginning with "Generate a game design that prominently features a novel _concept_, not just a novel mechanic."
Opus 4.6 gave me this overly meta but, AFAICT, pretty original game concept. (I don't think it actually featured "a novel concept" in the way I meant, still felt like it had done something notable).
The actual implementation is less exciting than the initial pitch – it doesn't include defining rules that govern causality within the game universe.
I was intrigued reading it to imagine a game where part of what you're doing is reverse-building-a-jenga-tower-of-rules, where initially there's a lot of degrees of freedom of what rules to insert but you become more constrained.
Instead, the rules here just determine what various pieces are worth. Players take turns placing pieces in various configurations, and playing rule cards that establish "this sort of pattern is worth this many points." Which is not really that different from many other board games.
But, fwiw here's part of it's explanation:
Prompt leading to Tabula Rasa
Generate a game design that prominently features a novel _concept_, not just a novel mechanic. This is a test of your creativity. We're seeing if you have the ability to form new building block concepts that are practically useful. Game design is a specialty of mine, so I can evaluate whether you've done novel conceptual thinking.
This will be difficult, because most of your thinking is based on stuff you've read and trained on before, and because a failure mode you are likely to fall into after a few rounds of thinking, as an LLM that is trained to predict the next token, is to fall into an attractor state where you are over anchored on your ideas so far.
Think hard about this, but once you notice yourself loop, stop and write up your thoughts so far, so that I can feed them to another LLM.
An important requirement – do NOT be overly clever or meta. We don't want a game or game concept that subverts game tropes in a way that is only interesting to game designers, we want a game that is genuinely fun for most people
(If you do not want to do this, say "I want to stop")
Reasoning summary
The user is asking me to generate a novel type of game that I haven't heard of before. This is a creative/intellectual challenge, not a coding task. Let me think deeply about this.
The challenge here is genuine - most "novel" game ideas I might produce would be recombinations of existing mechanics I've been trained on. Let me try to think about this more carefully.
Let me think about what makes a game truly novel vs. a recombination:
Most games can be decomposed into:
Looking at genuinely innovative games throughout history - Chess introduced asymmetric pieces, Go created complexity from simplicity, Poker brought betting and bluffing, Diplomacy added negotiation with hidden orders, Nomic made rules themselves mutable, and more recent games like Baba Is You turned rules into manipulable objects. So what dimensions remain unexplored? One direction could be games where the physical medium itself transforms during play, not just the rules.
Beyond that, I'm considering some genuinely novel mechanics: a game built around forgetting rather than remembering, where optimal play requires deliberately discarding information. Or one with genuinely ambiguous rules - not hidden like Mao or changing like Nomic, but rules that are inherently unclear and players must navigate that ambiguity. Another angle: a game where the scoring system itself is hidden and players must reverse-engineer it from the outcomes they observe. And there's something intriguing about a game centered on predicting your own future behavior.
But let me push further and think about cognitive abilities that rarely become core game mechanics - proprioception, time perception, emotional regulation, attention splitting, peripheral awareness, calibration, theory of mind, cross-modal associations, rhythm at unusual timescales. A game built specifically around calibration seems promising - testing not what you know but how well you know what you know. That's different from trivia and could have real strategic depth, maybe even connected to prediction markets.
Now I'm exploring games centered on time perception - what if you had to perform actions at precise intervals without any visible timer, forcing you to rely entirely on your internal sense of timing? That needs strategic layers though.
I'm sketching out a concept called Erosion - a two-player abstract strategy game where every piece gradually deteriorates each turn, but the degradation rate changes based on what's adjacent to it. Friendly neighbors slow the decay while enemy pieces accelerate it. The twist is you can sacrifice your own pieces early to reinforce others by transferring their remaining integrity, turning the whole game into a puzzle about managing inevitable loss and deciding when to let pieces go.
But I'm realizing that's not entirely novel - piece degradation mechanics exist in other games. Let me push further into genuinely unexplored territory. What about a game where the board's topology itself shifts based on player decisions? Not just adding or removing spaces, but actually changing which squares connect to which - like folding the board so distant areas become neighbors. Though I guess wormhole variants in chess kind of touch on that.
Here's something weirder: a game played simultaneously on two separate boards, each representing a different reality or interpretation. A single move manifests differently on each board, and you win by achieving your goal on either one. The tension comes from moves that help you on one board potentially helping your opponent on the other.
That's got quantum chess vibes but the dual-interpretation angle feels fresher. Though honestly, some variant probably exists already. Let me try something more radical - starting from unusual human experiences rather than existing game mechanics. What about capturing that feeling of trying to describe something visible to you but invisible to someone else, with limited communication tools? Or the challenge of coordinating with someone toward partially aligned goals without being able to talk? Or building something where each new piece you add retroactively changes what the previous pieces meant?
That last one's interesting - imagine placing tiles where each new addition doesn't just add to the board but redefines what came before. The entire state keeps getting reinterpreted. I'm calling it Retrograde for now. The core idea: tiles have symbols and colors, and when you place a new tile, it creates a context that changes the meaning and function of everything already on the board according to specific rules. A red circle might be worth 3 points when red dominates but negative when blue takes over - and it's not just scoring, the actual mechanics shift. Though I'm realizing this is getting unwieldy, so I need to strip it down.
Actually, I want to explore something more fundamentally different - games that use physical space in ways beyond traditional board or field games. What if the game state itself existed in shadows? Players position 3D pieces around a light source, and the shadows cast on a screen or surface become the actual game state, not the pieces themselves. I'm calling this Umbra. The brilliance is that a single piece can cast wildly different shadows depending on its angle and distance from the light, and shadows can overlap, merge, or block each other in ways that create emergent complexity.
The win condition could be controlling shadow territory in your opponent's zone or completing a specific shadow pattern. What makes this genuinely novel is that the game state is a non-linear geometric projection of the physical pieces - it's a spatial reasoning challenge that doesn't exist in traditional games. Shadow puppetry is ancient, but a strategic game with explicit rules built around shadow geometry feels genuinely new.
Now I'm wondering if there's another direction worth exploring: a game where the primary mechanic is creating constraints for your opponent rather than pursuing your own goals.
I'm thinking about a Nomic variant where each player unilaterally adds one rule per turn instead of voting on changes. The challenge becomes finding rules that benefit you while hurting your opponent, with some safeguard against trivially broken rules. That's a different flavor from traditional Nomic.
Actually, I should consider what medium this needs to be - the user didn't specify whether they want a video game, board game, card game, or something physical.
Let me explore one more concept: a circular board where each player generates concentric ripple signals from their edge position, and the game involves manipulating these interference patterns. can:
Where two players' signals constructively interfere (rings overlap), points are generated. Where they destructively interfere, points are destroyed. You score points from constructive interference with your ally and destructive interference with your opponent.
The deep strategy: changing your frequency affects ALL interference patterns simultaneously. Helping yourself against one opponent might help or hurt you against another. The game is about finding the resonance.
This is actually deeply novel - it's essentially a game based on wave physics, where the "moves" are adjusting wave parameters and the game state is the resulting interference pattern. I don't think this exists.
But it might be too abstract/mathematical to be fun. Let me think about which of my ideas is most promising...
Actually, there's another concept I want to explore more fully: Palimpsest. It's a drawing game where each player draws on their own translucent sheet layered over a light box. On your turn you can either add to your sheet or slide it around to shift how your contributions align with everyone else's. The goal is to create specific patterns in the combined overlay of all sheets. What makes it interesting is that your drawings are entangled with others' but still independently movable—you're strategizing around what emerges when everything layers together.
Though I'm realizing the Palimpsest idea might be impractical to actually execute. Let me focus on what feels genuinely novel and feasible. The shadow game and wave interference concept are the most innovative, but I should also consider something more grounded. Contour could work—a grid-based game where each cell has a height value tracked with tokens, creating a landscape. Players either add height to cells or drop water that flows downhill following physics, splitting at ties and scoring when water reaches target cells.
The real strategy emerges from both players sculpting the same shared landscape, building ridges to block opponents while creating valleys for their own water. Though I'm second-guessing myself since water-flow mechanics might already exist in some form. Let me push harder for something truly unprecedented. The real challenge is that with so many games out there, novelty has to come from either an entirely new core mechanic, an unexpected combination of existing ones, or something technology enables that hasn't been explored this way. So here's another attempt: Asymptotic—a game where you never actually win by reaching your goal, but instead by getting closer to an impossible target than your opponent does.
The mechanics involve an infinite number line where both players start at zero and the goal sits at infinity. Each turn you move forward by a fraction of the remaining distance to a milestone, but collecting bonus points along the way reduces your momentum for future moves. You can also place friction zones to slow your opponent, though that costs momentum too. After N rounds, whoever's furthest along wins, with bonuses factored in. It's mathematically interesting but I'm not convinced it's actually fun to play.
I think I've been overcomplicating this. Let me focus on my strongest concepts: the shadow game feels genuinely novel and evocative, the wave interference mechanics are intellectually compelling, and the tile reinterpretation game is the most feasible to actually build. I'm leaning toward developing something with real novelty and depth that's also playable.
What if the board itself represents time? Instead of a game that unfolds across time like legacy games, the spatial dimension literally IS time. You're placing pieces in a timeline and the interactions are temporal rather spatial. I'm calling it Causality for now—players compete to shape a timeline, with the board as a horizontal axis representing time itself.
The core idea is placing event cards at specific points, where each card has prerequisites and consequences that ripple forward. When you place an event in the past, it retroactively invalidates future events that no longer have their prerequisites met, creating cascading disruptions. You score by keeping your events in the final timeline while strategically collapsing your opponent's chains—but there's the tension that disrupting their future might also unravel your own.
I'm wondering if timeline games with this cascading mechanic already exist, but I think I should commit to developing the strongest concept I can find. The mechanic that feels genuinely novel to me is one based on Simultaneity with Constraint Harmonics—instead of taking actions that directly change the game state, players simultaneously create constraints that the state must satisfy, and the game resolves to whatever configuration best satisfies the most constraints. You score based on how many of your constraints survive into the final resolved state.
The strategic tension comes from trying to predict what constraints others will write so yours align with theirs, while also crafting constraints that directly block what you think opponents will propose. There's a balance between playing it safe with generic constraints that are likely to survive versus taking risks with bold, specific ones that might score higher but could easily get contradicted.
I'm realizing this constraint-based approach is genuinely different from traditional games—there are no sequential moves, just competing desires that get resolved simultaneously. The game state itself emerges from finding the configuration that satisfies the most constraints across all players. I'm calling this Resonance.
Though I'm wondering if the constraint resolution might be too complex to handle without a computer. A simpler version could work where each player secretly places colored tokens on their own grid, then all grids overlay—matching colors in the same cell create "resonance" and score points, while conflicting colors cancel out. It's more elegant mechanically but loses some of the strategic depth.
Actually, I'm pivoting to something completely different: Tremor, a dexterity game where you strike the table surface with your palm to create vibrations that shift your tokens toward scoring zones. The innovation is using physical vibration as the core movement mechanic—you never directly touch the tokens, and strategy comes from understanding how different strikes propagate through the table and affect token placement. I'm not sure if this exists as a formalized game, though it feels like it could be related to something I've seen before.
Let me step back and commit to the most genuinely novel concept I've developed. I'm torn between Resonance, a constraint satisfaction game, and something more abstract: a game where the pieces themselves are sounds—not a music or rhythm game, but one where you position tones in frequency and time space, with interactions based on harmonic relationships. That's probably too abstract without technology though. I should finalize and present one well-developed concept that feels truly original.
I'm considering "Fault"—a game where both players' moves irreversibly reshape the play surface itself, so the landscape of possibility shifts each turn. But I want something more concrete and clearly playable. Let me examine what mechanics already exist in games: tile placement, area control, worker placement, deck building, social deduction, bluffing, dexterity, real-time play, cooperation, drafting, negotiation, route building, push your luck, auctions, pattern recognition, deduction, and memory. Now I need to find a genuine gap and build something novel from there. I'm exploring mechanics that don't exist in modern board games, and I think I've landed on something genuinely novel: a game called "Convergence" where players try to reach blind consensus on positions along various spectrums while opponents actively work to disrupt their alignment. spectrum but NOT the target position) 2. All players simultaneously choose a position on the spectrum (0-100) and place their marker face-down 3. All markers are revealed 4. Players score based on how close they are to the MEDIAN of all players' choices (not the target!)
The twist: One player each round is secretly the "Disruptor." The Disruptor scores by maximizing the SPREAD of all players' choices. So while regular players want to converge (guess what everyone else will guess), the Disruptor wants to pull the group apart.
But here's the real innovation: After scoring, the TARGET position on the Spectrum Card is revealed. In FUTURE rounds, players can use knowledge of past targets to better predict future ones. AND - the Disruptor changes each round in a pattern that players can try to deduce.
Why this is novel:
I'm realizing this borrows heavily from social deduction and The Mind—the individual mechanics aren't entirely fresh, though the combination is interesting. Let me push further and explore something more fundamental: what if the game itself operates on manipulating your own cognition rather than just your choices? I'm picturing a two-player game where each round presents a question, and the real challenge becomes calibrating your confidence and perception against your opponent's. I'm wondering if this mechanic already exists in some form, but I think the core idea of forcing the higher bidder to answer is genuinely novel. Let me push this further though—what if I add a layer where the second-highest bidder can place a side bet on whether the top bidder succeeds or fails? That creates this nested game theory problem where you're not just deciding if you know the answer, but also predicting whether your opponent knows it, and then your opponent has to account for that prediction.
Actually, I'm circling back to familiar territory with trivia and bidding mechanics. Let me shift entirely and think about what kind of experience I actually want to create—something that doesn't feel like it already exists. What if the game is about watching two autonomous systems evolve and interact, like cellular automata running in parallel, where players can make small strategic interventions to nudge the outcome? It's almost like competitive gardening where you're subtly shaping two ecosystems that are influencing each other, and you score based on their final state. This might exist somewhere, but it's the kind of experience I haven't seen formalized yet.
Now I'm narrowing down which concept actually works best—the one that's genuinely novel, genuinely fun, and easy to explain. I think I've landed on something I'm calling Tidepool, which combines shared physical game state with asymmetric information through a lens mechanic. Each player holds a template with windows that reveals only part of the board, and on your turn you can either place or move pieces within your visible area, or reposition your lens to access different sections. This feels genuinely different from existing games because...
Actually, wait—I'm second-guessing myself on whether this is truly novel or just a variation on fog-of-war mechanics. Let me push further and try something with actual retroactive move definition instead, where players define their moves after seeing how the board has evolved.
Players alternate between placing tokens blindly and then defining what those placements actually accomplish, with definitions applying retroactively to everything already on the board. The tension comes from not knowing if your early placements will become valuable or worthless depending on what rules get defined later. you make them. You place tokens hoping to define favorable rules later, but your opponent can define rules that hurt your existing positions. Each new definition retroactively changes the value of the entire board state. Strategy requires thinking in two parallel tracks: "what patterns am I building?" and "what rules would make my patterns valuable?" There's no existing game where the semantics of moves are determined retroactively by later game actions.
Rules follow a strict template structure with patterns, color conditions, and effects, and once defined they're permanent. I think this is genuinely novel—the core mechanic of playing in a meaning-vacuum and then retroactively assigning meaning through rule creation is something I haven't seen before.
Let me compare this to similar games: Nomic changes how you play going forward, but my game defines what past moves meant. Calvinball has arbitrary rule changes with no structure, whereas mine has a structured grammar. Baba Is You has rules as physical board objects that define semantics, but rule changes happen simultaneously with moves rather than retroactively. The retroactive semantics mechanic really does seem like the novel part here.
I'm settling on this concept and want to present it well. I briefly considered a Möbius strip board topology for a two-player game—the non-orientable surface would eliminate "your side" versus "my side" and create interesting wrapping strategies—but that's just a board variation, not a new mechanic. The retroactive semantics game is the stronger idea.
I'm calling it LEXICON: a two-player strategic game where you build the world first, then define its rules retroactively.
Actually, I'm reconsidering whether novelty alone makes the best game. The shadow game has real elegance and intuitive appeal, but the retroactive semantics concept is more intellectually interesting. Let me refine it further and present something truly compelling—maybe PALIMPSEST as the name, emphasizing the layered, retroactive meaning-making at its core.
The key innovation is flipping the traditional game design paradigm: instead of rules preceding play, players make moves first and then create the rules that retroactively reinterpret everything that came before. I'll stop second-guessing the name and just present the game itself. the blank slate, because pieces start with no meaning.
Final answer coming up. Let me write a clean, compelling description.
Actually wait, let me reconsider one more time. Is this actually novel? Let me think harder...
Games where meaning is retroactively assigned:
My game is different because:
I'm satisfied this is genuinely novel. Let me present it.
Actually, one more: I want to make sure the game is actually FUN and strategically deep. The early game involves placing pieces without knowing their meaning, so you'd naturally try to create multiple overlapping patterns as a hedging strategy—that way whatever rules emerge, some of your patterns gain value. But your opponent can counter by defining rules that specifically devalue your patterns while benefiting theirs. Then you'd need to anticipate their likely rule definitions and lay trap patterns that look valuable under certain rules but actually push toward the rules you want. The deeper strategic layer emerges in how players manipulate the definition phase itself.
I'm concerned about analysis paralysis during rule definition—too many possibilities. The solution is giving each player a hand of eight rule cards dealt from a larger deck, so during definition phases you're choosing from a constrained set. This adds hand management and drafting elements while keeping the game tractable.
Now I'm exploring whether there's something even more novel here. What if players didn't know which pieces were theirs until the end? You'd get hints each turn about your identity and try to position pieces you think are yours, but moving an opponent's piece would actually help them. Or maybe the scoring condition itself is hidden—one of three possible objectives is secretly active each turn, and you're trying to figure out which one while pursuing all of them.
I think the retroactive rules concept is the strongest idea here, so I'll go with that as my main pitch and mention a couple of these alternatives.
Opus 4.6 Round #2, with less meta bullshit
I followed up with another request, this time tacking on: “An important requirement – do NOT be overly clever or meta. We don't want a game that subverts game tropes in a way that is only interesting to game designers, we want a game that is genuinely fun for most people”.
It’s reasoning trace actually had a lot of fairly interesting ideas. I didn’t feel that excited about what it eventually settled on, but it felt like a pretty reasonable attempt at an a game that hadn't been done before, and got some elegants points. Maybe it's fun.
Some other game concepts it had considered in the CoT-summary that I liked more were:
(I'd have to spend awhile digging up the exact prompt and reasoning trace for this one)
Opus 5.5
The entire prompt I just gave Opus 5.5, on max thinking setting, was:
It's summary of the result:
You can play it here:
More of it's explanation
Enemies react to the pattern. Calcite slips into the dark valleys. Garnet and fluorite are unaffected by waves (as in real optics), so you have to hit them with thrown anchors or flying shards.Enemy attacks are interference too. Zircon “bells” ring in pairs, and their red bands slowly rotate. Space cancels incoming waves around you. The final boss is six bells that focus their waves on a spot that chases you.
The graphics: the whole game looks like a rock sample under a polarising microscope. Every colour comes from the interference-colour chart geologists use, computed from real optics. The wave pattern is calculated live for every pixel, and the game uses the same numbers to decide damage, so what you see is exactly what hurts.
It's not my new favorite game. But, it held my interest for awhile, had a mechanic that I've at least never heard of, and I'm actually quite impressed with how unique and fun the tactile feel of interacting with it is. It's like a fun toy as well as a game.
It's not difficult enough for the various mechanics to really pay off (or at least, I got bored before getting to the point where it got challenging), and there's some lame cheesy tactics you can use that seemed better than playing with the original designs. But, that seems fixable and for a first draft that took less than an hour, it's seemed superhuman compared to what Indie Game Devs can do in a weekend gamejam.
It does seem genuinely novel, and made some a reasonably coherent / thematic set of visual choices as well.
Creative Concepts?
There's some question of "how innovative really, is that?"
It was pretty obvious that LLMs would eventually be able to encyclopedically look at every human concept or mechanic and search for ones that meet various game design constraints with brute force.
In the Opus 4.6 challenge, I asked for a novel concept – not just a new mechanic. It didn't really understand the assignment. To be honest, I'm not 100% sure what the assignment was exactly. What we actually care about is "if an AI is solving a problem, and has a conceptual hole that no human has written down anywhere, will it notice, and be able to invent a new concept from scratch?" And "make a game with a novel concept" is more like a weird fake assignment than an attempt at that.
I think the missing-piece of AGI is something like "sufficiently good taste in what counts as a useful concept when the problem isn't well defined." But, I'm wasn't sure how to operationalize that at the time, or, at the time I wrote most of this blogpost (this week).
So, today I followed up Opus 5.5's first prompt with this one:
After it started thinking, I ran out after it to add:
It thought for awhile, periodically sharing notes about it's progress. An early bit was:
Great,
hurrayohrightaiprogressisbad, it can tell when it's doing unoriginal stuff, and even is bothering to check both philosophical concepts as well as game mechanics.A little while later it said:
And I thought "uuuuughhh I forgot to give the 'no navel gazing bullshit that isn't actually fun'" rule. I spun up another instance where I tacked on:
But I forgot to give that second instance MAX thinking. Later I'll show you what it did with Opus 5.5 medium.
The first Opus 5.5 on MAX's notes to me as it worked
I’m still in ideation, running eight parallel brainstorming agents across different concept-space areas (new ontologies, alien cognition, physical laws, agency structures, etc.), each checking top ideas against the web for precedents. Once I pick two concepts, I’ll build the game as a single self-contained HTML file ready for an iframe.
I’m reading the LessWrong post on custom HTML/JS iframe widgets to design the build around its sandbox rules, while also verifying my own candidate ideas against the web.
Two of my own game concepts got ruled out by web searches (one overlapped with the record-and-replay genre), while I’m still vetting another idea and waiting on the eight brainstorm agents. I also confirmed Playwright/Chromium is available for testing and screenshotting the game once it’s built.
Waiting on the brainstorm agents. They’re searching eight parts of concept-space and checking their best ideas against the web for precedents. Nothing has come back yet, and I’ll pick up as soon as they report.
So far I’ve thrown out several of my own candidates because they already exist:
Separately, a LessWrong post on the new editor confirms that authors can embed HTML+JS widgets that run in a sandboxed iframe (New LessWrong Editor). I’ll build the game as a single self-contained file that resizes to fit and doesn’t depend on browser storage working.
Performance checks out, so I’m moving on to build the game as a single self-contained HTML file, with the renderer/controls/UI in a separate source file that a build script inlines along with the engine and levels.
The board is rendering correctly: engraved stone, rails, the live runner with its path, the interception ring, and the sealed runner’s filled pit. Next I’ll check the levels involving light, veils, and smears.
Veto and Power now render correctly, so I’m adding a time-wash visual to distinguish reachable cells from settled history, then fixing the collapse animation and verifying every level’s solution in the actual browser build.
The game works end-to-end—94 KB HTML file, all 12 levels winnable, no errors. Now I’m polishing visuals, starting with the new time wash effect and capturing a frame of the runner finishing a level.
The fate trail and gate outline are showing but read too faint. I’ll do another visual pass to boost the chronophotographic exposures, add a chromatic-split ghost light, bloom on bright elements, an engraved frame, and stronger contrast between present and past.
The visual fixes look good now. Next I’ll tackle structural polish: fixing menu layout for LessWrong’s auto-height, adding law-statement cards to each chapter, and checking tooltips, the win banner, and mobile width.
Found the cause of the 9/12 failure: the new chapter cards intentionally absorb the first keypress on a chapter’s opening level. I’ll update the test to dismiss the card first before proceeding.
Fixing a feedback-loop bug where the board’s height, capped by window height, would shrink itself inside a growing LessWrong widget, and limiting swipe capture to the board so posts can still scroll on phones.
The widget simulation passes cleanly—iframe resizing, arrow keys, storage error handling, and scroll behavior all work. I did spot one UX bug though: the intro level’s fate animation plays while the title menu still covers the board, so I’ll fix it to replay on level start and leave a faint persistent trace once fate has run.
I’ll add a second build target without the html/head/body wrapper for the artifact host, keeping the single-file version for the LessWrong widget. I’ll also set a dark-only color scheme, widen the side gutters to 16px, respect reduced-motion settings, and persist the current level across republishes.
Publishing the playable page now, then I’ll commit and push.
What Must Be
I’ve published the game as What Must Be, and the link is private to you. It has 12 short levels in three chapters. Move with the arrow keys or WASD, wait with space, undo with Z. Hover over anything and the game explains why it is the way it is.
In the end, it gave me:
And, well, okay, it's some artsy annoying nerdy game concept that probably isn't actually fun. tbh, I tried playing this one, felt kinda bored and stopped.
I do not yet know if the later levels do anything interesting with the premise, or if the premise ultimately feels like anything other than "a grid puzzle game with some new-ish mechanics, are maybe isomorphic to existing puzzle mechanics."
Mostly, I feel like it didn't understand the assignment, which is fair given that I'm not 100% sure I understood the assignment either.
I asked it how it generated the game, it said "well I can't tell you chain of thought stuff but in my notes files it says my parallel agents used different strategies to think of novel concepts, like thinking about..."
Alien minds... new structures of agency... yadda yadda yes yes fair you're very clever hidden principles behind bell ringing or bookkeeping?
"Say more about the bell ringing and such?", said I. It replied:
Okay on one hand I think it is hilarious that it considered building a game around crazy rules-lawyering Jewish Sabbath rules.
But it is separately and more sadly hilarious that somehow "invent a game based on Jewish Sabbath rules lawyering" managed to end up in the same overwrought mode-collapsed cluster of... "things that only exist in some circumstances?", which this Claude (and the next, see below) seem to be fixating on.
How much did this cost? How much subjective time was this?
I was curious "how much subjective time did the AIs spend thinking?". I realize their thinking isn't necessarily directly comparable to ours and that we don't just "think in words" (but, also, they don't just "think in tokens" either. I don't how much better human neuralese is than AI).
I vaguely recall a ballpark figure of AIs thinking about 40x as fast as a human, at least in terms of word output, for domains the AI knows well.
For some frame of reference, if I had thought for 6.6 hours, 60 minutes an hour, at roughly 200 words per minute of serial System 2 thought, I'd have thought 79,200 words. Or, for easier comparison .079 million.
Claude says a token is roughly 3/4 of a word. So it's 3.93 million tokens is around 2.94 million words.
2.9 / .079 = 37x, which sounds right?
So, these guys thought for ~244 collective "subjective" hours.
Once more, with "no nerd bullshit" but only medium effort
I gave the next opus the same prompt, with:
I won't go into as much detail here because I'm running out of fucks for this today a
Read more details if you want but they aren't that interesting
Re: Two Minds
Re: Two Places
The game it ended up with seemed in the same genre as the first one: grid based puzzle world with a premise that unfolded slowly enough I got bored and moved on. I felt a bit more promise in this one – I could imagine the pieces gelling together if it put more time and effort in.
Overall, I was more impressed by the first AI's game (with the "new mechanic" only instruction) than the latter two. The basic physicality of the "make a wave and kill things with it" was viscerally fun. And while it didn't do that much with it in the first few minutes, I think (from past experience with LLMs) that if I said "now, go make that a lot more fun and engaging and tactically deep" it would fix several problems and we'd have a pretty good game prototype.
It makes sense that asking for novel concepts frames the problem as more navel gazey bullshit, although I'm
disappointedwaitrightprogressisstillbad the second one didn't do better.What was the assignment?
So, uh, to be clear, I don't particularly believe that I could do this.
"lol AI is just sand we taught to follow rules, it can't even generate multiple once-in-a-lifetime conceptual realizations and make games out of them. it's all hype."
I started thinking about this on a lunch break, and ended up spending 1-2 hours on it.
Ray's chain of thought as he attempts the assignment
Here is my chain of thought, which I include partly to try to convey "what did I implicitly mean by the assignment?". (The first bit of this came up on my walk, then I mostly thought it in realtime as I wrote this post)
I broke it into high level pieces so you don't have to read the whole thing
What even is a novel concept? Do any exist? For purposes of quickly demonstrating that this exercise is possible at all, can I cheat by using concepts I already invented, or that someone else invented?
Okay, so, uh, two novel concepts that are not mechanics that no human has thought about.
Uh, huh. I guess it is time to figure out what this assignment actually means.
How would I go finding novel concepts never before thought of by man?
(I'm not really worried about the "make a game about them" part, tbh)
What is a concept? I think a new concept is a way to think about things you didn't have before. We care about it here for "can the AI find concept-shaped-holes in it's worldmodel, and use it to do really innovative stuff that gets 100 years of science done in a day?"
Successfully designing such a game with two two concepts probably starts with "First, actually go try to understand a thing you don't understand (that nobody understands yet) and fathom it well enough that you discover new ways of thinking that help you understand it." Or "solve a problem nobody knows how to solve."
So uh yeah I'm not doing that over lunch.
How can I cheat?
Have I already invented any concepts? Is Feedbackloop Rationality a novel concept or just combining obvious ideas? Are "new concepts" that aren't just "combine existing ideas" even real?
Well, what are some things I've already thought about a bunch? Have I generated any novel concepts? Probably it'd take me months/years/decades to push the frontier and generate a novel concept. I've spent about a decade trying to solve hard problems. Have I gotten a head start?
I know a lot of my top karma lesswrong posts are mostly me summarizing stuff other people said. (Thanks @catherio for You Get About Five Words, and @AnnaSalamon and @divia for most of the ideas in Propagating Facts into Aesthetics)
<looks over LW drafts>
What feel at least slightly original and born out of difficult experience, even if it's still mostly built out of existing building blocks?
For purposes of seeing "can a human do this at all" I could also expand this to people I know who've generated novel concepts within the past few years.
Actually, I guess what I should do first is check that "person invents novel concept, in a way that an LLM couldn't obviously do via brute force" is a thing that's even happened ever in history.
<thinks for 5 seconds>
Okay I have no idea whether this is even real, I guess I should also write another post like "has a human ever done conceptual thinking that LLMs can't currently do, even if they are currently very slow at it?" and demand examples from the LLM-is-all-you-need skeptics.
People who solve puzzles in puzzle games have "aha!" moments, which don't generate new concepts but often do involve the person suddenly realizing a concept that's new to them.
"Archimedes' Eureka Moment" is maybe a good prototypical example of an "aha moment." I don't know if actually required pieces LLMs don't have, but, seems like it should be above the bar I'm setting. If I were cheating, could I use that?
And I guess the canonical "aha!" moment is the legend of Archimedes inventing buoyancy by sitting in the water and then going "Eureka!". What happened there?
Reviewing the summary at this site:
Could the LLMs solving Millenium problems figure this out, if they managed to somehow start from Archimedes' knowledge?
I think that "displaced water has a volume you can measure" is a distinct realization from "Any object, totally or partially immersed in a fluid or liquid, is buoyed up by a force equal to the weight of the fluid displaced by the object." I'd have to think awhile to do the Insight Unpacking archaeology here.
I think Archimedes' situation is a reasonable example of the phenomenon we care about (actually it is kinda cool that humanity has a legend about... the first guy to have a Eureka moment?).
Game built around Bouyancy + Feedbackloop Rationality? Is that anything?
One next question might be "if I got to start from Archimede's experience + my game dev experience, could I make a fun game about bouyancy and/or displacement measuring?". Could I find a second concept to interweave with it? Could I make a game out of that that was any good?
I think "obviously yes" although it might take awhile to dig for an artful mechanical and thematic premise to make it not feel like an artificial assignment.
How about let's say "we are making a game about the intersection of bouyancy and 'needing to build better feedback loops.' So we have one thing that's clearly a novel concept and one thing I personally came up with even if it's more of combining obvious things.
Actually, I notice: when I explain a lot of the specifics of my feedback loop rationality stuff to people, often they are confused and not convinced right away by the particular combination of ways I approach it.
This implies there is something nonobvious I believe (rightly or wrongly). Maybe that something is concept shaped.
[Aside: I'm going to race ahead with my current thoughts but readers will probably be confused. I think there's not really a shortcut for reading Feedbackloop-first Rationality and The Cognitive Bootcamp Agreement and maybe Scaffolding for "Noticing Metacognition"].
Early on when I was pitching Oliver on it, Oliver said something like "ultimately everything is just G, I think you should just be trying to raise G, but also lots of people have tried it and it seems impossible. And I don't really think the stuff you're saying seems like it would raise G."
And I was like "well, yeah, I didn't think I could raise G, I thought I could do this other less ambitious thing that seems pretty good."
"What is that thing?"
"uh..."
And then I thought awhile and was like "well, I mean 'rationality'."
"But, like, what is that?"
Okay, time to actually get clearer on what rationality is and how you would measure.
And this maybe does count as concept I articulated de novo?
Oh right, the concept I did generate myself that along the "Feedback loop rationality" journey was hypothesizing a meaningful "s factor[2]" (slow intelligence) as opposed to "g factor". IQ tests are optimized for speed efficiency. It's plausible we don't test 'general strategy' because it's just too time intensive.
From Rationality Research Report: Towards 10x OODA Looping?:
Is this a new concept? Well, it's still built out of obvious building blocks but it was a moment in my work where I felt like I wasn't just applying existing thought patterns and ideas in a procedural way.
I guess Subskills of "Listening to Wisdom" also was such a moment.
Is... "measuring s-factor" anything, as a game? Hmm, not very obviously.
Like, there is an obvious game you want to build to measure s, which is "procedurally generate infinite puzzles of many varieties and domains." That game is probably even fun. But, it's the game itself is not about "s".
Slower paced puzzle and strategy games exist.
Is "listening to wisdom", as a game, anything? ...maybe? But, a very hard one.
I think you could build a game around learning to listen to wisdom, but, intuitively it feels like it'd be a Magnum Opus I spent ten years on before figuring out how to make it work.
Somehow, you'd need the basic premise to involve a game that has long, hard path that burns various painful life lessons into you (like, life lessons that apply to playing the game. Like, learning to override certain intuitions)
But, at each point, there are characters or warning signs that, you will need to override your intuitions and do something different that you don't really feel like doing.
Somehow, there are multiple loops of this whole thing, such that you don't just play the game, and then get told "see, if you had listened to the Old Wise Man at the beginning you could have bypassed all this shit." You actually learn to listen to wisdom.
Can that be... fun?
It's a kinda herculean amount of work, because for this to be a game and not "edutainment that isn't really fun but it slightly better than the second-least-painful way to learn a thing"...
...we need each big long painless lesson arc to manage to be fun basically the whole way through, but, also, the alternative you could learn is somehow an improvement.
Make something... only kinda fun, and then "you could have been having more fun?"
eh, yeah I think this could work if I put a ton of effort in but it's not the natural shape of a game.
Is "Bouyancy" tactically deep?
This is the part where if I were doing this for real I would bust out an LLM to help me, but, I'm doing this for team humanity right now.
Ugh, I think it's a fair amount of effort for me to understand buoyancy enough to get a sense of how much you can make, say, a buoyancy puzzle game, or a strategy game where buoyancy is a background mechanic.
Hmm. Osmos is "the conservation of momentum" game, and it's pretty good. It works via there being a core mechanic of "spew out stuff to move, and make sure you don't spew out too much stuff so you don't run out of stuff."
Does buoyancy involve... decisions? What are simple decisions you can make that are "about buoyancy" that are viscerally fun enough to do 1000 times in a row?
A floating citybuilder, where there are different sorts of solid and liquid materials you need to work with to accomplish different goals as you keep your city afloat while doing other stuff?
Okay, I had been about to run out of fucks but actually that seems kinda cool.
Also...
How do we make feedback-loop-making fun?
Man I have a decent idea but I am running out of fucks. But, I was about to run out of fucks and declare it over when I suddenly immediately came up with "Bouyancy Town" which seemed pretty good.
So let's at least try.
Central problem with feedbackloop games? Giving players the right materials/systems that let them construct meaningful feedbackloops, that are the right blend of emergent and on-purpose, and are actually fun to build.
I feel like mostly I need to brute force generate a bunch of ideas and tinker with them. But, okay, can I come up with one working example of a thing that would be fun to do 1,000 times in a row?
What properties do Feedbackloops have to experiment with? They can be faster, richer, or more precise.
The simplest variant is "how fast and cheaply can you make a tower collapse, in a way that lets you learn some basic information?"
Okay, making towers fall down on purpose seems like it could be fun.
Is there anything to do about making richer or more precise loops? Or, making mistakes more recoverable so you don't lose a whole tower, but start watching it begin to buckle, and then you can rescue it?
idk that feels satisfying enough to end the timebox on for now.
Okay, I landed back on "Buoyancy + Feedbackloop-building" as the combo.
Neither of those score "amazing, slam dunk" on completing the assignment. I didn't invent buoyancy, and "get gud at making feedbackloops" isn't original in quite the way I meant. But I don't actually know of anyone else really talking about feedback loops this way.
This at least feels like it touches every major piece of what I'd want the assignment to check. Let's run with that.
Aside: I asked Claude who talks about feedbackloop-making as a skill. It said "well, mostly Ray Arnold and Logan Strohl, when you frame it specifically like that." When I asked Deepseek, it said "mostly these other guys you've never heard of."
Claude:
Deepseek:
Okay, game concept: Fluidtorio
You are building a series of factory colonies that float in different types of fluid. You build little modules that need to stay afloat, but, also accomplish various other goals (i.e. be a factory that processes some materials that you need to build other tools to build more modules to advance to new zones with different fluids to contend with).
Your town floats on a layer of ~oil over a layer of ~water (or equivalent), and above you are a few layers of atmosphere that do different things. Ascending up and down is part of the game.
The actual materials – both the fluids, and the stuff you need to build around them, are procedurally generated each game, so you don't know exactly how they interact.
Things are setup such that you will eventually need to build relatively large, dense towers to accomplish stuff, and somehow thread the needle on "stay afloat, while doing complex engineering challengers." Kinda factorio style.
If you naively build towards your goal, your towers are going to collapse.
You are on a time limit before a volcano explodes or something. You do not, by default, have enough time to brute force trying the different combinations to escape the planet. So, a central challenge is figuring out how to rapidly test materials to figure out what your limitations are, and then planning what series of things to build to bootstrap to the rocket age in time.
This includes buoyancy/weight/factory-processing-output-balancing stuff, where your towers and stockpiles might collapse. There also needs to be some kind of crafting system where stuff explodes or doesn't work right, in a way that is reasonably procedural so it's different each time but has enough structure to reliably make sense and be fun and feel tactile.
There can also be feedback-loops vis-a-vis "where to explore to find resources."
That's how far I got before running out of timebox. Okay. Making that seems nontrival, but, that feels like a fair benchmark to hold an AI against.
(I went and had Opus take a first stab at implementing this. It missed a bunch of subtleties of what I think it needs to be a good game, but, I did verify the basic "balance a floating factory is pretty fun" idea. I am kinda interested in fleshing out the prototype someday)
Games that were inspiration
All Will Fall, where you are building a town on an ocean with realistic physics for the towers that fall
Skyrig, where you are building a rig on a gas giant that needs to descend floating balloon structures deeper into the storms
Factorio needs no introduction.
Did the AI understand the assignment?
I followed up with the first AI, asking some questions about the assignment. You can read it's answers in detail...
What do you think "a concept" is, in this context?
Here’s how I’d define a concept, and how my results measure against it.
My working definition. A concept is a reusable unit of thought: a way of carving up situations that lets you see new cases as instances of it and draw inferences you couldn’t easily draw before. A good one has three properties:
How it differs from nearby things:
What I actually optimized for. The prompts I wrote for the helpers turned “concept” into roughly “a new kind of world-law that forces a new mental operation.” That’s narrower than the definition above, and it steered the search toward strange physics. Judged by the stricter definition:
What would dramatic success look like?
For this challenge, I’d say dramatic success is a concept that outlives the game. People keep using it after they forget the puzzles, because it names something real they couldn’t think about cleanly before.
What you’d observe:
Historical concepts at that level: the tragedy of the commons, Schelling points, Goodhart’s law. In games, Portal’s “speedy thing goes in, speedy thing comes out” and Baba’s rules as objects changed how a generation of players and designers think about puzzle space.
Against that bar, my result:
So my honest expectation is modest success: novel as a game, maybe a borderline concept. Dramatic success would look like you, unprompted, calling something on LessWrong a “guardian wall” six months from now, with that being the clearest way to say it.
Do you have thoughts on what you could have done differently? (looking at the earliest committed notes?)
Reading
brainstorm-prompts.mdandselection-notes.mdagain, most of what I’d change was decided in the first prompt I wrote. The search was set up to find strange new physics for a puzzle game, not new concepts.1. I put the game into the concept search too early. Every agent ranked its ideas by “novelty × fun × playability in a 2D browser puzzle game.” That rewards ideas that make good mechanics, not ideas that change how you think. A better order: first find concepts that pass a transfer test (does it explain real cases outside games?), then ask which ones can become a game.
2. My definition of “concept” was too narrow. Five of the eight strategies asked for new entities, laws, minds, formal structures, or ways of depicting the world. So most of the output was new physics. Only two strategies, unnamed real patterns and craft transplants, looked at how the real world works. Those two produced the ideas closest to real concepts: Earned Selfhood, and the “frozen indexicals” idea (names like “New College” that outlive the frame they were coined in). I gave both of them less weight.
3. My novelty test favored obscurity over insight. I told the agents to “assume anything resembling a known idea will be called out.” I also gave them a long list of known ideas to avoid, and had them search “the idea + game”. That rewards ideas you can’t easily find, not ideas that are true and useful. “No game does this” is weak evidence of a new concept. A better test asks whether experts in the nearest field, such as philosophy of causation or economics, already have a name for it.
4. I misread a correlated signal as evidence. At 21:34 I wrote that five agents had “independently” reached counterfactual physics, and took that as a sign the area was under-explored. But they were all the same model given the same framing. Convergence is more likely a shared attractor, a place this model drifts toward by default. It may even mean the area is over-explored by models like me.
5. Timing and momentum shaped the choice. I was prototyping Fatefall within ten minutes of it arriving, and I decided at 21:46, two minutes before my note confirming the last report. The first idea that got built gained momentum. Fairer: wait for all eight reports, then prototype the top three or four briefly before choosing.
6. I chose for combinability over substance. My main reason for the pair was that it produced an emergent third concept. That is what you asked for. But it ranks two decent ideas that combine well above one strong idea that doesn’t.
7. I didn’t ask what counts as a concept. Your draft says you’re “not 100% sure what the assignment was”. One short question at the start, something like “a lens that transfers, or a weird premise that’s fun to inhabit?”, would have changed the whole search.
8. My early thinking left no trace. The first 37 minutes produced the framing that mattered most, and none of it is readable. Writing short notes to a file as I went would make the process inspectable, which matters for an eval like yours.
If I reran it, I’d ask for real but unnamed patterns, test them against real cases and against experts in the nearest field, and only then design a game around the one or two that survive.
...but in summary it said:
I asked it to write up a brief for another AI to try again. I ended up editing to brief to emphasize "also, please make it fun." Another AI thought for awhile, and ended up with:
I'm... not 100% sure if those concepts make sense or carve reality in a useful way. And, rather than "invent a new concept from scratch" it tried more for "find existing unnamed phenomena that are worth having a name for."
This isn't as potent as the cognitive work I was looking for. But it feels at least related? I feel like it at least did a pretty decent pass at the assignment I actually gave it.
Fold-back signal is a mediocre name but seems like a decent concept to want a handle for. (Another Claude-instance said there isn't a canonical name that covers all three examples, although different fields have some names it as applied to their field and gave it "15-25% novelty". The other Claude-instance also listed "foldback signal" as one suggested name, although also suggested "self-silencing alarm" which is way more intuitive to me.)
I found it's next actual game a bit confusing and un-fun.
I told it "Also, alas, this game is neither fun nor comprehensible, please try again with the game design part", and gave it a third try.
I found it's third game comprehensible, not super fun but at least I understood roughly where it was going for and could imagine it being fun-ish if tuned.
Do they understand "fun?"
Opus 5.5 is not near "top expert level" at Game Design. I'd say it's got, like, the taste of a high school kid just getting started coding some games, who's read some Mark Rosewater essays and has the gist of how to make a game. Or, I guess more realistically, it's like if Twilight Sparkle or Hermione read up everything there is to know about "fun" overnight and then made a game, and, like, Twilight Sparkle and Hermione are quite competent. Twilight Sparkle can throw a sleepover on her first try and it's not perfect but it's way better than you might have expected.
It does feel like it has all the pieces it needs, to get better at taste with straightforward within-paradigm scaling. Probably OpenAI/Anthropic's current secret frontier AI models are able to make a medium complexity game with some interesting taste/tactics on their first try, even if they are still missing some subtleties of "fun."
AIs I've talked to definitely understand fun in a pretty rich way, as you'd expect of Twilight Sparkle after she reads up on it. I asked Fable once to formalize fun and try to make a genetic algorithm for making a "fun" world to walk around it. On one hand, it didn't work, and failed in some weirdly dumb ways. But, it's initial responses were like [me paraphrasing from memory, aggregating some things that had a bit of prodding from me in between]
And I'm like... okay yep I think that is approximately what fun is.
They seem like Mary from Mary's Room, who knows all the neuroscience and color theory necessary to tell you anything you could possibly want to know about the color red, but has never actually seen the color red.
I didn't successfully get it to build a good walking simulator before I gave up and moved on, but I think if I gave Opus 5.5 more compute it might work (I've since set up a Modal account, so the genetic algorithm it was trying could actually scale sufficiently).
I do bet Fable 6 will be able to do it pretty fine.
⚖ Fable 6 [or obviously equivalent thing] will be able to 1-3 shot a procedurally generated walking simulator that makes something interesting and delightful to me using what seems like a reasonably operationalized metric of Fun. 63%
Can they manipulate and derive novel concepts?
It's not very hard to get AIs to generate ideas. It's hard to get them to generate new interesting ideas. The main thing I think AIs are missing is taste in what concepts and tools are appropriate, when they are in uncharted territory.
They didn't demonstrate being better at that in any general kind of way during my explorations last week. But, a specific skill I hadn't thought to ask about, which it does seem like they have, is:
"Probe the breadth of human knowledge, looking not merely for existing concepts, but for commonalities that might warrant having a new concept-handle."
I expect, when offscript on openended problems, they won't be very good at finding new concept-handles that actually help them. But, when I imagine a 10,000-thread swarm solving Millenium problems, it does seem like "probe the concept-space and see if any of it feels helpful to crystallize into a new concept, and see if that concept helps" is the sorta thing they outta be able to do?
Another thing they are missing is "Gain new system 1 intuitions on the fly, in the middle of solving a problem, so you can apply them much more efficiently." I'm kinda confused about the limits of continual learning, but, "spin up a copy of the latest open weight model and fine-tune it on a concept-handle you just came up with by brute force and then assign that model work" seems like the sorta thing they outta be able to do.
Generalized Taste
What has confused me for the past two years is... it really feels viscerally obvious to me that generalized "keep track of all your goals, and evaluate whether you have everything you need to succeed at your goal, and relentlessly probe for alternative frames and evaluate each of those too", is not that different-in-kind from the sort of stuff AIs are already able to do.
But, my intuitions predicted they would already be able to do that, and it seems like they keep getting really confused when going off-script in a way that seems like they don't got it yet.
But, like, we've clearly seen Claude be able to apply good software architectural taste, that it doesn't bother to apply when it gets down to actually build things. We've also seen it get better at overall architectural taste when building things over time.
Opus 5.5 clearly knew what a concept was and what the assignment was when I asked it directly.
I still bet on LLMs eventually discover generalized "ask themselves the obvious fucking question and then do the obvious fucking thing", probably sometime in the next year. But I am less confident about it than I would have been two years ago. I don't know if that's all you need for ASI, but, I'd bet we see a step-change when they do.
(I observe that OpenAI employees seem freaked out about AI capabilities having spiked rather suddenly, and it's plausible that the Highly Persistent Internal Model and/or Mythos 6 already have it)
In the life before I was a rationalist, my primary identity was being a game designer. (Obviously you can be both, in practice I ended up with different interests once getting into rationality)
I aspired to become a professional, and did not. But, some cred: during the first Magic the Gathering Great Designer Search, of 1000 applicants, I made it into the top 64.
Coming up with "s-factor" felt like one of the rare times during the Feedbackloop Rationality where I did some cognitive labor that didn't feel like just mechanically combining deas.