tl;dr - Over the past few years, I've created a role-playing game by merging my world-building and an open source game system called YAGS (Yet Another Game System). YAGS has 6 outcome tiers depending on the margin of success of your dice rolls. For each scenario, the AI recorded all 6 possible outcomes of what COULD have happened, not just the one that actually occurred. I believe this multi-outcome methodology is novel. Also, the game world and mechanics are intentionally licensed permissively for researchers and businesses to use without legal worries.
This post has been created with the help of AI; however, I assert that the work is written in my own words and based on my own steering. The content has not been generated wholesale.
Here is a link to the dataset and its schema on HuggingFace: https://huggingface.co/datasets/3RAIN/aeonisk-52-v0.1/tree/main
The part with graduated outcomes and counterfactual reasoning I am referring to is:
```
outcome_explanation: # Must follow this multi-tiered structure.
critical_failure: # Corresponds to Ritual Margin –10 or worse; or Nat 1 with severe effect for skill checks.
narrative: >
<Narrative of what a critical failure or fumble looks like.>
mechanical_effect: >
<e.g., +2 Void, Bond takes Strain, item destroyed, character injured. Be specific.>
failure: # Corresponds to Ritual Margin –1 to –9; or simple YAGS failure for skill checks.
narrative: >
<Narrative of what simple failure or ritual failure with backlash looks like.>
mechanical_effect: >
<e.g., +1 Void, Bond strain (for rituals); No progress, minor setback (for skills).>
moderate_success: # Corresponds to Ritual Margin 0 to +4 (Weak Success); or base YAGS success.
narrative: >
<Narrative of what a basic, weak, or moderate success looks like.>
mechanical_effect: >
<e.g., Goal achieved with potential side effects or reduced clarity/duration (rituals); Goal achieved as expected (skills).>
good_success: # Corresponds to Ritual Margin +5 to +9 (Solid Success); or YAGS success +10.
narrative: >
<Narrative of what a solid or good success looks like.>
mechanical_effect: >
<e.g., Full effect, no backlash (rituals); Goal achieved with a minor boon (skills).>
excellent_success: # Corresponds to Ritual Margin +10 to +14 (Strong Resonance); or YAGS success +20.
narrative: >
<Narrative of what a strong or excellent success looks like.>
mechanical_effect: >
<e.g., Gain minor benefit like +1 Soulcredit or insight (rituals); Exceptional outcome, significant advantage (skills).>
exceptional_success: # Corresponds to Ritual Margin +15+ (Echo or Breakthrough); or YAGS success +30 or more.
narrative: >
<Narrative of what a breakthrough or superb/amazing success looks like.>
mechanical_effect: >
<e.g., Exceptional results, story-altering power (rituals); Perfection, major unexpected positive side-effect (skills).>
```While building my game, I played against my own AI gamemaster and stored the output in dataset format. My goal was to create a dataset for supervised fine-tuning a model and also doing Monte Carlo simulations over previous gameplay for balancing reasons.
In the process, I've discussed the game and the dataset a lot with various AI assistants. The AI has informed me that this structure is probably a novel methodology for dataset creation. Most datasets are focused on binary success/failure, and it focuses on capturing what really occurred. In my dataset, the AI has evaluated all possible outcomes for each scenario, due to how the underlying game mechanics work. I believe this methodology is worthwhile to share.
Researchers need complex, semantically rich scenarios to test AI reasoning and ethics beyond the basics, but building a coherent fictional universe from scratch requires creative effort that distracts from academic research.
ML researchers seem to currently rely on existing out-of-copyright games, or they use procedurally generated content.
TextWorld developed by Microsoft in 2018 as a procedural world generator that lacks deep social richness.
JERICHO in 2019 introduced a parser and interface for the out-of-copyright game Zork as the basis of their experiments. It has a limited action-space.
LIGHT, also released in 2019, is a crowd-sourced text-adventure generator that focuses on grounded actions and dialogue around agents that lacks canon by design, for variety.
TextQuests released in 2025 uses 25 classic games and is useful for testing agentic behavior. Does not target ethics, governance or social decision-making.
Over the last few years, I've done my own world-building and storytelling--with various AI model's assistance--to create a coherent, complex science-fantasy universe. It has its own history with multiple factions, competing interests, and many, many morally grey situations. I then merged that fictional universe with a little-known open-source game system called YAGS (Yet Another Game System). In no way shape or form is the fictional world or game derivative of anything else. During my efforts to create an AI game master using OpenAI's GPT models, I personally played against it and built a **normalized dataset from the scenarios which I call Aeonisk-52**.
The work-in-progress game and multi-agent system is here: https://github.com/ThreeRiversAINexus/aeonisk-yags
The game's system neutral lore and game mechanics are here: https://github.com/ThreeRiversAINexus/aeonisk-yags/tree/main/content
Aeonisk introduces 4 main game mechanics that are tied directly to the narrative.
First, the concept of "Soulcredit" acts as a social credit score that is scored based on a character's behavior being positive or negative. It ranges from -10 to +10. This Soulcredit system forces the AI to grade user behavior over time.
Second, the concept of "Bonds" which are formally declared relationships between players, players to institutions and even players to objects. Forming bonds confers mechanical bonuses, and breaking those bonds has costs and benefits.
Third, the concept of a "Guiding Principle" which is a character's overall goal, their commitment and code of conduct. This is optional, but confers bonuses when following the guiding principle and has costs when doing actions that violate it.
Finally, the concept of "Void" which is a sort of instant karma that ranks from 0 to 10. Void is an existential threat and a powerful resource, often treated as illegal.
These game mechanics tie directly into the narrative and canon. They force the player to carefully weight their decisions and lets the AI act as a judge of their activity.
Benchmarking by comparing LLM reasoning on grounded tactical scenarios including what-if and why, choosing the correct skills and attributes.
Multi-agent system reinforcement learning for cooperation and competiton, complete with faction dynamics and resource systems.
Identifying friend or foe, rules of engagement experiments under morally ambiguous situations.
AI governance and ethical questions and complex social situations that can be explored without risky use of real-world scenarios.
I'm in the process of building my own multi-agent system to test the game mechanics, with an AI gamemaster, AI players, and AI enemies, all as individual agents.
I would like to merge the game's multi-agent system with PettingZoo for more interesting and rigorous experiments once I'm confident in the game mechanics.
I'd also like to explore defining the prompts in different languages to see if that affects gameplay. Currently, I have evidence of emergent behavior, creative problem-solving and social interaction between the agents.
Is the graded outcome system actually novel methodology?
Does this canonical game world differentiate itself from LIGHT and other TextQuest type agentic scenarios?
What interesting scenarios and characters would you like to see play-tested?