I asked a bunch of LLMs with websearch to try and name the classic mistake you're alluding to:
- Opus 4.5: "apriorism about alignment", the alignment-specific version of "intelligence, properly understood, must be architecturally legible"
- DeepSeek + Deep Think: "apriorism in AI", the "formalist fallacy", etc
- Gemini 3 Pro Thinking: "the logicist paradigm"
- Grok 4 Expert: the "neats" paradigm (as in "neats vs scruffies")
- ChatGPT 5.2 Extended Thinking: "neatness bias", "premature formalisation: locking in crisp, human-legible abstractions before you actually have an empirically grounded handle on how the thing works, and then you end up doing a lot of beautiful work on the wrong interface"
- Kimi K2 Thinking: the "knowledge engineering mirage", "the faith that you can formalize the world’s complexity faster than it generates edge cases"
To be honest these just aren't very good, they usually do better at naming half-legible vibes.
Just learned about the Templeton World Charity Foundation (TWCF), which is unusual in that one of their 7 core funding areas is, explicitly, 'genius':
Genius
TWCF supports work to identify and cultivate rare cognitive geniuses whose work can bring benefits to human civilization.
In this context, geniuses are not simply those who are classified as such by psychometric tests. Rather, they are those who: (1) generate significant mathematical, scientific, technological, and spiritual discoveries and inventions that benefit humanity or have the potential to transform human civilization, and (2) show exceptional cognitive ability, especially at an early age.
Eligible projects may include research on the benefits of various attributes of geniuses to humanity, biographical studies of individual geniuses, comparisons of groups of geniuses with various levels of cognitive abilities, and projects that facilitate the spread of creative insights, discoveries, and original ideas of geniuses. Projects may also investigate genetic factors contributing to genius, and the cultural and nurturing factors that engender geniuses who contribute to such cognitive virtues as diligence, constructive thinking, and noble purposes. Ineligible projects include physical, musical, or artistic geniuses; spelling bees; geniuses with spectacular memory; and scholarships for geniuses.
Among the 613 projects they've funded so far, 7 grants come up if you search for 'genius', all between 2013-18 so I'm not sure why they stopped since. Some of the largest grants:
- Investigating Mathematical Talent and Autism using Genetics and Epigenetics, 3-year $2.7M grant to Simon Baron-Cohen at Cambridge in 2015 to do whole genome sequencing and epigenetic sequencing of "20 highly enriched, maximally informative families (i.e., those who have 3 or more individuals with autism in the nuclear family)" and "50 extremely talented mathematicians (international math prizewinners)"
- Developing Mathematical Creativity, 3.5-year $1.7M grant to John D. Barrow at Cambridge (not all of them are to Cambridge, just the biggest ones apparently) to "create a distinct, new program of innovative educational resources, including a substantial collection of resources aimed at students of all ages from 7 to 16. The principal resources we are creating are rich and engaging open tasks, published online on a dedicated project portal, the new Wild Maths website. These rich tasks are presented in thematically-linked pathways supporting progression towards increasingly sophisticated thinking, independence, and creativity. The online resources are designed to encourage students to explore and make connections. They include inspirational, explanatory articles and videos, as well as opportunities for students to share and discuss ideas. Hands-on workshops provide further opportunities to develop and explore mathematical creativity, and a public lecture program provides stimulus and inspiration"
- The Potential in All of Us: Identifying and Cultivating Genius at a Worldwide Scale, 5-year $1M grant to Po-Shen Loh (ex-US IMO team national coach for a decade, now obsessed with post-AGI humanity apparently?) at Expii, Inc. to overcome the lack of "effective, personalized strategies for finding exceptionally talented individuals around the world... by restricting access to talent searches on the basis of geography, institutional affiliation, or the inability to pay for educational materials" by "rigorous free resources in science and math to students around the world on its user-friendly website" and more
- Extension of a Nationwide Hungarian Talent Support Network to Identify and Support Potential One‐in‐a‐Million Cognitive Geniuses, 2-year $780k grant to Peter Csermely at Magyar Tehetségsegítő Szervezetek Szövetsége to identify "300 young Hungarians... with exceptional (1 out of 10,000) cognitive talents – becoming Hungarian Junior Templeton Fellows – to take part in the program" and "put together a one-year personalized talent development program for each of the 300 Hungarian Junior Templeton Fellows... to set up a creative community network of exceptionally talented people, who will become the leaders, researchers and entrepreneurs of Hungary and Europe over the next 10 to 30 years". This of course reminded me of the Martians
Yeah the "pi like you" was a reference to that passage.
1Yeah, this was the source of much personal consternation when I left my operations-heavy career path in industry to explore research roles, as much as I found the latter more intrinsically exciting.
It's also what's always back-of-mind w.r.t. the alignment-related work I'm most excited by, even though part of why I'm excited about them is how relatively empirically grounded they are.
Scott Aaronson made a very simple string finder that beats almost all naive human strategies. Check out the Github here, or play against it yourself here (using Collisteru’s implementation).
I can't resist sharing this quote from Scott's blog post, I loved it the first time I read it all those years ago in Lecture 18 of his legendary Quantum Computing Since Democritus series, the whole lecture (series really) is just a fun romp:
In a class I taught at Berkeley, I did an experiment where I wrote a simple little program that would let people type either "f" or "d" and would predict which key they were going to push next. It's actually very easy to write a program that will make the right prediction about 70% of the time. Most people don't really know how to type randomly. They'll have too many alternations and so on. There will be all sorts of patterns, so you just have to build some sort of probabilistic model. Even a very crude one will do well. I couldn't even beat my own program, knowing exactly how it worked. I challenged people to try this and the program was getting between 70% and 80% prediction rates. Then, we found one student that the program predicted exactly 50% of the time. We asked him what his secret was and he responded that he "just used his free will."
I wonder if he'd just memorised the first couple dozen digits of something like Chaitin's constant or e or pi like you or whatever and just started somewhere in the middle of his memorised substring, that's what I'd've done.
Mildly funny analogy by John Cutler, niche audience, illustrating a failure mode that feels personally salient to me. Here's how it begins:
Imagine if a restaurant behaved like your average product team. The kitchen is packed. Everyone is moving. Every station is busy. Prep lists are long. Meetings are constant. There is always something to do. Chopping, rearranging, documenting, planning, replating.
But plates rarely reach customers. When they do, they’re late. Or wrong. Or cold. Or oddly disconnected from what the diners said they wanted. Yet the kitchen isn’t “failing,” exactly. It never looks like a crisis. No one storms out. No one flips a table. Diners don’t riot. They just lower their expectations and stop coming back.
Inside the kitchen, though, the staff feels productive. Everyone is exhausted. Everyone is “at capacity.” Everyone can point to a dozen tasks they completed. They can even argue those tasks were important. And in isolation, many of them were.
But restaurants are not judged by how busy the kitchen is. They are judged by how consistently they deliver great food, on time, to the people who ordered it. Product development is strange because this feedback loop is muted. There is no instant revolt. A team can be unbelievably heroically busy without producing much that actually moves the needle.
That’s the trap: in software, effort is easy to generate, activity is easy to justify, and impact is surprisingly easy to avoid.
(much more at the link)
This comment ended up more interesting than I expected when I started reading it. Did you write up your progress journaling anywhere online?
I've mentioned this elsewhere — I first learned about effective altruism circa 2014 via A Modest Proposal, Scott's polemic on using dead children as units of currency to force readers to grapple with the opportunity costs of subpar resource allocation under triage. I was young and impressionable when I encountered it, so I've never stopped feeling the weight of the frame of EA as duty/obligation, although its weight has lightened considerably since. I related to Tyler's personal story (which unsurprisingly also references A Modest Proposal as a life-changing polemic) since I followed a similar life arc:
I thought my own story might be more relatable for friends with a history of devotion – unusual people who’ve found themselves dedicating their lives to a particular moral vision, whether it was (or is) Buddhism, Christianity, social justice, or climate activism. When these visions gobble up all other meaning in the life of their devotees, well, that sucks. I go through my own history of devotion to effective altruism. It’s the story of [wanting to help] turning into [needing to help] turning into [living to help] turning into [wanting to die] turning into [wanting to help again, because helping is part of a rich life].
There are other more personally-beneficial frames that arguably (persuasively, IMO) lead to much more long-run impact because they're sustainable, e.g. Steven Byrnes' response to a different comment seems pertinent, also Holden Karnofsky's advice:
I think the difference between “not mattering,” “doing some good” and “doing enormous good” comes down to how you choose the job, how good at it you are, and how good your judgment is (including what risks you’re most focused on and how you model them). Going “all in” on a particular objective seems bad on these fronts: it poses risks to open-mindedness, to mental health and to good decision-making (I am speaking from observations here, not just theory).
That is, I think it’s a bad idea to try to be 100% emotionally bought into the full stakes of the most important century - I think the stakes are just too high for that to make sense for any human being.
Instead, I think the best way to handle “the fate of humanity is at stake” is probably to find a nice job and work about as hard as you’d work at another job, rather than trying to make heroic efforts to work extra hard. (I criticized heroic efforts in general here.)
I think this basic formula (working in some job that is a good fit, while having some amount of balance in your life) is what’s behind a lot of the most important positive events in history to date, and presents possibly historically large opportunities today.
That said, if you asked me to list the activities I find most joyful, I'm not sure EA-related ones would make the top five.
Bit tangential: re: your sequence name "civilization is FUBAR", I get the FU, but why BAR? Maybe I'm just in too much of a progress-vibed bubble?