Phoenixes have always been my spirit animal, and I bought this tapestry long before reading Harry Potter and the Methods of Rationality, however, yes, this is also the group house in which HPMOR was completed, Godric’s Hollow, which if I’m not mistaken does give us the non-exclusive, non-copyrighted, non-commercial, completely unrelated to J. K. Rowling rights to use the term “Order of the Phoenix”
Originally written November, 2025. I still endorse maybe 50-70% of this memo[1].
0. Preamble
With this memo, on the top floor of the tallest tower of Godric's Hollow, I instantiate the real-life Order of the Phoenix, and begin the proceedings with our group cheer which carries the initials of this memo and is hereby imbued with the full force of our collective energy-wisdom-intentionality:
OOO-OOP-HOH!!!
Now that that’s out of the way…
This memo began as a response to an email from Dony which I found deeply energizing and epic. In the spirit of "Yes-And," I am responding in kind. I grouped together all Dony’s ideas that seemed similar, noticed they fell into a few main categories, and prioritized them in what I believe is very roughly the order of operations in which they should be enacted. I'll start each section with some of Dony's text and then add my commentary.
Table of Epicness
1. Prioritization: Comprehensively list best project ideas and carefully pick the #1 hinge-iest project for preventing x-risk, then take the #1 hinge-iest action on it every day
2. Feedback: Publish #1 hinge-iest idea OR ship #1 hinge-iest project feature every day to real people and source high-quality feedback
3. People: Meet numerous people, collaborate constantly on little projects; ultimately build a community around the project and find world-class core collaborators
4. AI: Continuously build new AI tools to accelerate our projects and ExoBrain components to bring out the best in ourselves and others
5. AI EA: Launch an offshoot of EA, "g/acc", that helps make AI itself go well, and utilizes AI to differentially do as much good as possible
1. Prioritization
Dony writes:
"Slow is smooth, smooth is fast. Progress looks like sitting at a laptop each day, vibecoding, vibewriting, vibing, vibe rating. I want an office with dozens of people tapping away."
"We have to learn to be highly decisive and cope with VUCA. Premature optimization is a tendency I have. I am a completionist in video games and want to collect every item and achievement. There's no way you can do that in the game of life. We have to let go our addiction to our future potential and learn to cut off that potential in the choice of an actuality."
“Most ideas suck and need to be pruned. Most things are not worth doing. The work requires doing boring shit day in and day out, consistently. There is an art of letting go of dead/bad/not-good-enough ideas that I don't know.”
“I think we should work on our own unique ideas and stick to them, even if they're not the best ideas (as long as they're robustly net positive in expectation). I expect there's no way for a human to figure out the best thing to do in principle. We can only satisfice on something. . . . I strongly believe in Diving In as Nate Soares describes it.”
Strong agree on the slow part. What does slow actually look like? To start with, I think it looks like slowing way down and listing out all the projects which have the greatest likelihood of saving the world or redirecting the trajectory of the future, our "hinge-iest" projects, then taking all the time needed to think about which is the most important and prioritizing that list. Maybe this will take weeks, maybe even a few months. This is worth it, because we want to work on the best project we can and really commit to it.
However, while I strong agree on the slow part, I'm strong disagreeing on the "there's no way to figure out the best thing" part. Obviously we can't know for sure, but I think we can get at least a little bit closer to the truth, and little increments could result in massive expected value, because the actual best thing may be many times better than the next best or third best thing. We can uncover real crucial considerations and avoid massive mistakes by realizing conceptual insights about what is important and why it's important. Plans are useless but planning is indispensable.
If you're spending 80,000 hours on your career, you should think deeply about it, try to understand what general and personal factors make for the most impactful career, generate a lot of ideas and carefully prioritize and evaluate them, write out your best ideas and get detailed feedback from trusted confidants and top experts, and run experiments to see which paths actually empirically work best. We don't have 80,000 hours but we probably do have a few years, and so I think it's worth committing to trying really hard to figure this out for at least a few weeks, maybe even months.
Another example: the "Charity Entrepreneurship" process really works. It generates a massive number of good ideas, then systematically eliminates the weaker ideas through progressively deeper dives, and by the end produces genuinely excellent plans which launch organizations like the Lead Elimination Project and the Shrimp Welfare Project. I think we should have a community-wide event where everyone in the EA movement follows a process isomorphic to this for determining their very best idea for making AI go well or effectively utilizing AI to super-charge their highest impact project.
On Dony's point about letting go, yes, most ideas need to be let go. We have a million ideas and most of them are kind of garbage. We have a couple really good ones but we can't even do a couple of them, we need to pick one and work on that one idea every single day, or at least most days. We can't be completionists.
This is why I say we need to slow down, think very carefully, and make prioritized lists of our top projects so that we can see the relative importance of our projects and get a strong sense that our top choice really is the best one according to our best judgment, so that we can move forward with confidence that cutting out all of the lower priority projects is the right decision.
Once we've picked the project, we need to commit to working on it for at least 90 minutes every day no matter what. To make it a habit, each day we could set an alarm for some specific time, or plan a routine where after some reliable trigger like directly after morning caffeine or after our first meal, sit down and:
Figure out the one most important thing we can do to concretely push the project forward that day
Set a timer and do that one thing for 90 minutes
Track the habit every day
It can always be longer than 90 minutes, but 90 minutes needs to be the minimum. It's doable every day, which builds momentum, and keeps us from being discouraged which could happen if we set a much higher goal and then don't reach it consistently.
Once we have prioritized and committed to our hinge-iest project, and are taking the most hinge-y actions on it for 90 minutes every day, the next thing is getting excellent feedback so that we know our project is actually going somewhere, not just perfectionizing and spinning around the echo chambers of our own minds endlessly.
2. Feedback
Dony writes:
"It is incredibly important to start with tiny scopes and do things that don't scale. Build prototypes and rapidly test them with others. You don't know what you don't know”
“Start tiny projects that can snowball and crystallize into something massive. Don't do linear things that can't combine exponentially.”
“If we build up the skill of quickly hacking on projects together and shipping in public this builds momentum and a portfolio of things we can do and external stakeholder buy-in”
"Publish, publish, publish, publish. We all seem to have a sensitivity to our ideas being put out there and criticized, torn to shreds. We need to have faith in ourselves and our safety and strength to put things out there and influence others."
Hell yes. We need rapid feedback loops where we produce the best thing we can produce in a single day, a 1-5 minute post of our single best, most hinge-y idea, or the single most important feature on our software project. We need to be consistently shpping in a single day, getting real people looking at it and sourcing high-quality feedback, rapidly invalidating what doesn't work, 80/20ing and building up a body of work, or getting our product progressively closer to something that is actually useful for saving the world and achieving amazing futures.
Literally every day though? If on exceptionally important stuff we occasionally take 3 or 4 days, fine, might be worth it for the most important stuff. And to be clear, I think there can be times when there needs to be deep work over a period of months or years on a large project, it’s just that right now I think we are all in the phase where we need rapid iteration and rapid feedback to keep us grounded and focused on real-world progress. The intense discipline and immense momentum that a very literal single day deadline forces is actually crucial for us right now. I think we may amaze ourselves with what we achieve.
The reason we need to be publishing and shipping our very best stuff every day first is that it's hard to have super deep meaningful powerful impactful hinge-y connections with people before we have anything to connect over. Once our projects are making meaningful progress every day, then we have something real to share with others, something we can ask for feedback on, something to collaborate with others on. Otherwise, we risk just making a bunch of aimless connections that don't go anywhere. Daily progress on incredibly hinge-y projects creates a powerful magnet around which a community can be built.
3. People
Dony writes:
"Getting the alliance of other people is a huge force multiplier. Each of our projects needs a team of advisors, interested users, and interested collaborators."
"The world wants to give us things if we give it what it wants. It requires building prototypes and showing it to others. . . . Expand luck surface area, explore serendipity and Get In The Van when feasible."
“We should cooperate with 1000s of other people, either altruists or other in-groupies, and unlock their aligned potential. Do things that scale.”
"We must profusely publish and Find The Others who can join the Fellowship. This requires being willing to have uncomfortable conversations and face rejection. . . . We want to get good at finding others with complementary talents and inviting them to our home or working with us."
Yes yes yes. People are our most valuable resource. They multiply us by many times, and we can multiply them as well.
I think one way of framing this is building communities around our projects. I think the most relevant and deep connections will be ones with people who share our core values and vision, and who we can meaningfully collaborate with on our projects in deep ways.
Quality here is far more important than quantity. It's easy to waste time with dozens of people who vaguely agree with us and are kind of pretty cool, but it would be much better to form deep partnerships with 1 to 3 people who are the most obsessed and excellent people in the world on our subject of interest. That said, we probably need to meet quite a lot of people to find those 1 to 3 people.
It could be really high leverage to go through a phase where once we have some serious momentum on our projects we find high leverage ways of meeting a few new people every day who share our passion, constantly doing little collaborations on daily pieces we are publishing and features we are shipping, seeing which people we really vibe with and who really accelerates our work, and then building teams of people who can all work together on super high leverage projects and start scaling up the work we’re doing.
One reason people need to come before AI is that right now fully autonomous agents are kind of meh, at least compared to super amazing agentic humans, and so it's moreso people who are getting things done in the world and making use of AI rather than the other way around. We can find a few core collaborators and then source a huge number of amazing committed altruists in a community around our projects, and make sure we are getting them all using AI super early. AI is going to be nonnegotiable within the next year or two for sure and so we need to get started on it ASAP. However, if all we did was get extremely good at AI but didn't have important projects or a supercharged network, I would be probably a bit more pessimistic than the other way around.
4. AI
Dony writes:
"We must deploy tons and tons and tons of AI for good against the forces of Mordor. I cannot emphasize, we consistently underestimate the amount of Dakka that can be gotten out of using AI agents in parallel to make intellectual progress. It's superlinear."
"We should constantly be upgrading our exobrains, each day trying out one new AI or other tooling to increase our i/o, the quality of our thoughts and ideas and conversations, putting out new better code faster and faster."
“I want to have a daily briefing I can wake up to, composed by an AI that knows my priorities in an up to date way. Something more custom and better than Pulse. I want to see a feed that's calm and gives me just the info I need.”
AI is the thing. It is weird that there are still some people who don't get this. Regardless, we have to put our Dakka where our mouths are and go kind of crazy on AI.
One really important thing is building our ExoBrains which essentially means we have our entire life context loaded up in a highly optimized manner so that AI can easily know exactly who we are, how we think, what's important to us, what our core values and priorities are, what our main projects are and what the key bottlenecks are, what our workflows look like, what our daily schedules look like, all the messy little personal details where we could be optimizing in a million ways that we might not even realize and the AI could figure out how to accelerate us 2X to 10X and maybe in a few years 100X+
I really do believe that it is becoming increasingly true that "context is all you need." Once AI is infinitely more capable than us, the bottleneck becomes how well the AI knows us and all the fine-grained details of what we actually want. But before that day comes, hopefully we can find ways of differentially accelerating Human-AI Symbiosis, building tools which massively differentially accelerate human wisdom, agency, and wellbeing, approximately in that order. So that the AI is bringing out the best in us at the same time that we are bringing out the best in the AI by continually upgrading the ExoBrain, making it more and more finely attuned and synergistic and symbiotic with us but also more tuned toward the good de dicto—toward what is fundamentally and ultimately good in the universe, or according to humanity’s extrapolated volition.
As with all things we need to start small. Create one little tool each day and see if it can do anything useful. We’ll have a lot of failures or mehs and occasional sparks of brilliance. Latch onto those sparks and accelerate them 10X. I think the ExoBrain will be modular with many tendrils accelerating different parts of ourselves.
Essentially, we should have various AI tools continually analyzing every aspect of our projects and our lives, and putting millions of tokens into figuring how it can help us optimize in little ways and big ways:
Moral and wisdom coaching: analyzing how we think, speak, and act in the world through the lens of various moral theories and wisdom teachings to help us see where we could be acting with greater courage, truthfulness, altruism, compassion, agency, scope-sensitivity, thoughtfulness, prudence, love, open-mindedness, equanimity, prioritization mindset, opportunity-cost awareness, embodiment, aliveness, joy, mindfulness, and meliorism (the world can be made better through effort); collaboratively helping us work on our lives, what we do, and who we are to become the best versions of ourselves.
Information curation and briefing: searching through thousands of tweets, news articles, machine learning papers, strategy briefs, etc. every day, picking out the parts most relevant to us, and presenting them to us in an easily consumable style that makes us feel appropriately existentially serious about the present and existentially melioristic about the future.
EA Forum and LessWrong mining: as Dony suggests, having our ExoBrain sift through ideas on the EA Forum and if there's one that's really good, alert us so we can help that poster bring that idea to fruition.
Finding people we should connect with: scanning our networks, the EA Forum, Twitter, conferences, and identifying the specific individuals who share our passions and where collaboration could be fruitful, then preparing a detailed brief on those individuals and a list of promising conversation topics based on shared interests.
Analyzing and improving our habits: thinking about how we could adopt new habits or drop bad habits, identifying which small changes would have the largest exponential compounding effects over time.
Strategic analysis of our plans: analyzing our plans from a million different angles, strategizing and re-strategizing, thinking of a million different things we could add or remove to improve our plans and every possible way our plans could go wrong or be net negative and ways of mitigating these.
AI tool scouting: searching out new AI tools we could be using for every aspect of our life and extensively reviewing these tools to see if they would actually be helpful or are all hype, so it gives us only the gold.
Mental health optimization: sifting through all of the best mental health research and finding the things it thinks will help us most, as we continually feed it transcripts through lifelogging and journaling; data on our values, dreams, goals, anxious and depressing thoughts, traumas, preferences, fears, and desires.
Relationship improvement: recording our conversations and having it help us dramatically improve our communication and relationship skills, identifying patterns and high leverage ways of shifting those patterns.
Problem-solving and prioritization: thinking through all of the main problems we are confronting in life and work and finding and prioritizing among a million ways of fixing them based on what's most efficient, effective, and fun, and based on everything it knows about us.
Opportunity cost analysis: analyzing the opportunity cost of all the possible things we could be doing that we are not doing so it can alert us if there's something we have no idea we are missing out on which would fulfill our values far better than whatever we are doing.
Values-aligned wealth generation: generating, analyzing, and prioritizing all possible ways of making us trillionaires which are aligned with our values, preferences, skills, and life circumstances.
In general, across all these areas, the AI should be generating and prioritizing between thousands of little experiments we could be trying to determine the very best ones we could try and see if they create genuine improvements in our lives.
At the same time, we will save all of the best examples of when the ExoBrain performs unusually well or when it makes harmful mistakes, and feed those back to it. We will constantly play with the architecture and tinker with the prompts and the settings and the workflows and the tooling. We’ll keep expanding with new experiments and then compressing to what is truly useful, gradually building more and more powerful ExoBrains that are increasingly good at helping us become our best selves.
We could open source these and cross pollinate insights among a community of ExoBrain enthusiasts whose explicit goal is to exponentially accelerate humanity toward better and better instantiations of Human-AI Symbiosis.
I think it’s essential that we share these AI tools with others, especially people who are trying to have a positive impact. I would love to see the EA movement becoming really good at leveraging increasingly advanced AI to maximize its positive impact, as well as shifting significant focus toward finding ways of making sure that the AI transition itself goes well.
5. AI EA
Dony writes:
"MacAskill is obviously right that EA needs to shift to deploying AI well. To me this has been obvious for years, unfortunately there is a miasma in that we don't have common knowledge amongst all the EAs that this is the case. Literally every individual could be thinking this but they don't have common knowledge."
"I like EA and think it should continue, it just needs morphed and we have the differential creativity to incubate a new culture to fork off from it."
I'm glad Dony brought MacAskill up. His post "Effective altruism in the age of AGI" was a big part of what triggered my initial feelings of hinge-y-ness which led to this whole exchange. A lot of people know this but don't know what to do with that information. The popularity of his post was stunning (and well-deserved) and goes to show that people are hungry for this.
I hereby state my intention to launch a Phoenix Fellowship focused on "AI for Good," A.K.A. "Good Accelerationism" or "g/acc," and especially accelerating the good de dicto[2]. It could be one of the first projects of the Order of the Phoenix (maybe we can call ourselves the "Fellowship of the Phoenix," there's a nice symmetry in having the Fellowship of the Phoenix launch the Phoenix Fellowship). (Update: current iteration of this is Flourishing Futures Fellowship, you can sign up for the email list here)
Here are some proposed core values of g/acc, the "New EA movement," which I think should include MacAskill's "New Enlightenment":
Keeps many EA values: scope sensitivity, truth-seeking and rationality, tools like ITN and expected value, care about moral circle expansion and moral uncertainty, disagreeableness and constructive criticism, career choice and research as important levers.
The good de dicto: what is good fundamentally, not just instrumentally or in the short-term, but truly, ultimately good from first principles if moral realism is true, or from something like humanity's coherent extrapolated volition if anti-realism is true.
Developing the "New Enlightenment": the Enlightenment developed good philosophy and institutions for the Modern Era; the New Enlightenment could develop good philosophy and institutions to navigate the Age of AGI.
Making AI itself go well: navigating the AI transition and intelligence explosion wisely, especially hinge-y neglected issues other than alignment. Hinge-y here means issues which could have a significant impact on long-term value, for example through persistent path dependence or lock-in.
Leveraging AI to do as much good as possible: this could look like AI applied to traditional EA cause areas, but I believe even more so it means leveraging AI to create a condition of “viatopia” in which humanity has the tools and institutions to guide itself toward near-best futures in the long-term.
Human-AI Symbiosis: AI bringing out the best in humans and humans bringing out the best in AI, especially building AI that differentially accelerates human wisdom, agency, and wellbeing, in that order.
Trajectory change and macrostrategy: deep understanding of how to create good long-term trajectories. May necessitate AI augmented macrostrategy.
Potentially explicitly longtermist: at least longtermist leaning. Maybe also futurist and transhumanist.
Achieving first principles understanding: of what goodness is and what consciousness is.
Targets systemic change: utilizing an understanding of systems theory, feedback loops, attractor states, and mechanism design to create good systems or steer systems in good directions.
Creating a new economic system: that maximizes public goods, especially good de dicto, and makes capitalism obsolete by out-competing and out-cooperating it.
Potentially designing a new global order: if desirable, might play an active role in designing a new global constitution for governing a post-AGI world.
Multiplayer mindset: Doing things that can multiply the effectiveness of the ecosystem as a whole
Building extremely good infrastructure: creating systems that scale and compound over time.
Fostering extremely good community culture: the way everyone interacts is a highly underrated force multiplier.
Recursion and navel-gazing: the community is obsessed with reflecting on itself and crowdsourcing collective strategy and ways of improving the community itself. We give a karma bonus to community posts, not a penalty.
Decentralization and exponential organization practices: books like Exponential Organizations and Exponential Organizations 2.0 are key texts.
Absurd levels of ambition and prioritization of maximization of impact: at the macro level (high-level decisions like career choice and project choice) and micro level (highest leverage next action to move the project forward).
Opportunity cost ethics: failing to do good you could have done is just as bad as if you had actively caused that thing not to happen.
Entrepreneurial and social entrepreneurial mindset: building things that generate massive value for the world while capturing some of that value and moving the world in a good direction. Win-win-win mindset.
Prioritized lists and processes for generating them extremely effectively: e.g., Charity Entrepreneurship-like processes and tools like ExoPriors.
Self-development and self-care: exponential habits (i.e. habits that feedback on themselves to generate increasingly more resources over time) and finding ways of systematically exponentially multiplying your own effectiveness over time.
High integrity as a core value: being all around good people and not one-sided or lopsided. Self-actualization, always aspiring to be our best selves.
Honesty: telling the whole truth and nothing but the truth, not being deceitful or misleading. Maybe also radical transparency, open data, lifelogging and sharing everything like Audrey Tang.
Healthy relationships, emotional intelligence: good communication skills, mental health, meditation, mindfulness, embodiment, aliveness
As Dony’s says, EA is great and should continue. I find the idea of forging a new offshoot culture fine-tuned for operating in the age of AGI invigorating and inspiring. Would love to hear what y’all’s ideas are for what the core values of a “New EA” should be, definitely room for debate and discussion here.
Publishing now because it suddenly seems to be mid-to-late hinge. The biggest update from a year ago is that I think people should imminently position themselves to help slow down AI, work on automating alignment, or get in position for strategies that leverage advanced AI capabilities.
Phoenix in my room at the time of writing
Phoenixes have always been my spirit animal, and I bought this tapestry long before reading Harry Potter and the Methods of Rationality, however, yes, this is also the group house in which HPMOR was completed, Godric’s Hollow, which if I’m not mistaken does give us the non-exclusive, non-copyrighted, non-commercial, completely unrelated to J. K. Rowling rights to use the term “Order of the Phoenix”
Originally written November, 2025. I still endorse maybe 50-70% of this memo[1].
0. Preamble
With this memo, on the top floor of the tallest tower of Godric's Hollow, I instantiate the real-life Order of the Phoenix, and begin the proceedings with our group cheer which carries the initials of this memo and is hereby imbued with the full force of our collective energy-wisdom-intentionality:
OOO-OOP-HOH!!!
Now that that’s out of the way…
This memo began as a response to an email from Dony which I found deeply energizing and epic. In the spirit of "Yes-And," I am responding in kind. I grouped together all Dony’s ideas that seemed similar, noticed they fell into a few main categories, and prioritized them in what I believe is very roughly the order of operations in which they should be enacted. I'll start each section with some of Dony's text and then add my commentary.
Table of Epicness
1. Prioritization: Comprehensively list best project ideas and carefully pick the #1 hinge-iest project for preventing x-risk, then take the #1 hinge-iest action on it every day
2. Feedback: Publish #1 hinge-iest idea OR ship #1 hinge-iest project feature every day to real people and source high-quality feedback
3. People: Meet numerous people, collaborate constantly on little projects; ultimately build a community around the project and find world-class core collaborators
4. AI: Continuously build new AI tools to accelerate our projects and ExoBrain components to bring out the best in ourselves and others
5. AI EA: Launch an offshoot of EA, "g/acc", that helps make AI itself go well, and utilizes AI to differentially do as much good as possible
1. Prioritization
Dony writes:
Strong agree on the slow part. What does slow actually look like? To start with, I think it looks like slowing way down and listing out all the projects which have the greatest likelihood of saving the world or redirecting the trajectory of the future, our "hinge-iest" projects, then taking all the time needed to think about which is the most important and prioritizing that list. Maybe this will take weeks, maybe even a few months. This is worth it, because we want to work on the best project we can and really commit to it.
However, while I strong agree on the slow part, I'm strong disagreeing on the "there's no way to figure out the best thing" part. Obviously we can't know for sure, but I think we can get at least a little bit closer to the truth, and little increments could result in massive expected value, because the actual best thing may be many times better than the next best or third best thing. We can uncover real crucial considerations and avoid massive mistakes by realizing conceptual insights about what is important and why it's important. Plans are useless but planning is indispensable.
If you're spending 80,000 hours on your career, you should think deeply about it, try to understand what general and personal factors make for the most impactful career, generate a lot of ideas and carefully prioritize and evaluate them, write out your best ideas and get detailed feedback from trusted confidants and top experts, and run experiments to see which paths actually empirically work best. We don't have 80,000 hours but we probably do have a few years, and so I think it's worth committing to trying really hard to figure this out for at least a few weeks, maybe even months.
Another example: the "Charity Entrepreneurship" process really works. It generates a massive number of good ideas, then systematically eliminates the weaker ideas through progressively deeper dives, and by the end produces genuinely excellent plans which launch organizations like the Lead Elimination Project and the Shrimp Welfare Project. I think we should have a community-wide event where everyone in the EA movement follows a process isomorphic to this for determining their very best idea for making AI go well or effectively utilizing AI to super-charge their highest impact project.
On Dony's point about letting go, yes, most ideas need to be let go. We have a million ideas and most of them are kind of garbage. We have a couple really good ones but we can't even do a couple of them, we need to pick one and work on that one idea every single day, or at least most days. We can't be completionists.
This is why I say we need to slow down, think very carefully, and make prioritized lists of our top projects so that we can see the relative importance of our projects and get a strong sense that our top choice really is the best one according to our best judgment, so that we can move forward with confidence that cutting out all of the lower priority projects is the right decision.
Once we've picked the project, we need to commit to working on it for at least 90 minutes every day no matter what. To make it a habit, each day we could set an alarm for some specific time, or plan a routine where after some reliable trigger like directly after morning caffeine or after our first meal, sit down and:
It can always be longer than 90 minutes, but 90 minutes needs to be the minimum. It's doable every day, which builds momentum, and keeps us from being discouraged which could happen if we set a much higher goal and then don't reach it consistently.
Once we have prioritized and committed to our hinge-iest project, and are taking the most hinge-y actions on it for 90 minutes every day, the next thing is getting excellent feedback so that we know our project is actually going somewhere, not just perfectionizing and spinning around the echo chambers of our own minds endlessly.
2. Feedback
Dony writes:
Hell yes. We need rapid feedback loops where we produce the best thing we can produce in a single day, a 1-5 minute post of our single best, most hinge-y idea, or the single most important feature on our software project. We need to be consistently shpping in a single day, getting real people looking at it and sourcing high-quality feedback, rapidly invalidating what doesn't work, 80/20ing and building up a body of work, or getting our product progressively closer to something that is actually useful for saving the world and achieving amazing futures.
Literally every day though? If on exceptionally important stuff we occasionally take 3 or 4 days, fine, might be worth it for the most important stuff. And to be clear, I think there can be times when there needs to be deep work over a period of months or years on a large project, it’s just that right now I think we are all in the phase where we need rapid iteration and rapid feedback to keep us grounded and focused on real-world progress. The intense discipline and immense momentum that a very literal single day deadline forces is actually crucial for us right now. I think we may amaze ourselves with what we achieve.
The reason we need to be publishing and shipping our very best stuff every day first is that it's hard to have super deep meaningful powerful impactful hinge-y connections with people before we have anything to connect over. Once our projects are making meaningful progress every day, then we have something real to share with others, something we can ask for feedback on, something to collaborate with others on. Otherwise, we risk just making a bunch of aimless connections that don't go anywhere. Daily progress on incredibly hinge-y projects creates a powerful magnet around which a community can be built.
3. People
Dony writes:
Yes yes yes. People are our most valuable resource. They multiply us by many times, and we can multiply them as well.
I think one way of framing this is building communities around our projects. I think the most relevant and deep connections will be ones with people who share our core values and vision, and who we can meaningfully collaborate with on our projects in deep ways.
Quality here is far more important than quantity. It's easy to waste time with dozens of people who vaguely agree with us and are kind of pretty cool, but it would be much better to form deep partnerships with 1 to 3 people who are the most obsessed and excellent people in the world on our subject of interest. That said, we probably need to meet quite a lot of people to find those 1 to 3 people.
It could be really high leverage to go through a phase where once we have some serious momentum on our projects we find high leverage ways of meeting a few new people every day who share our passion, constantly doing little collaborations on daily pieces we are publishing and features we are shipping, seeing which people we really vibe with and who really accelerates our work, and then building teams of people who can all work together on super high leverage projects and start scaling up the work we’re doing.
One reason people need to come before AI is that right now fully autonomous agents are kind of meh, at least compared to super amazing agentic humans, and so it's moreso people who are getting things done in the world and making use of AI rather than the other way around. We can find a few core collaborators and then source a huge number of amazing committed altruists in a community around our projects, and make sure we are getting them all using AI super early. AI is going to be nonnegotiable within the next year or two for sure and so we need to get started on it ASAP. However, if all we did was get extremely good at AI but didn't have important projects or a supercharged network, I would be probably a bit more pessimistic than the other way around.
4. AI
Dony writes:
AI is the thing. It is weird that there are still some people who don't get this. Regardless, we have to put our Dakka where our mouths are and go kind of crazy on AI.
One really important thing is building our ExoBrains which essentially means we have our entire life context loaded up in a highly optimized manner so that AI can easily know exactly who we are, how we think, what's important to us, what our core values and priorities are, what our main projects are and what the key bottlenecks are, what our workflows look like, what our daily schedules look like, all the messy little personal details where we could be optimizing in a million ways that we might not even realize and the AI could figure out how to accelerate us 2X to 10X and maybe in a few years 100X+
I really do believe that it is becoming increasingly true that "context is all you need." Once AI is infinitely more capable than us, the bottleneck becomes how well the AI knows us and all the fine-grained details of what we actually want. But before that day comes, hopefully we can find ways of differentially accelerating Human-AI Symbiosis, building tools which massively differentially accelerate human wisdom, agency, and wellbeing, approximately in that order. So that the AI is bringing out the best in us at the same time that we are bringing out the best in the AI by continually upgrading the ExoBrain, making it more and more finely attuned and synergistic and symbiotic with us but also more tuned toward the good de dicto—toward what is fundamentally and ultimately good in the universe, or according to humanity’s extrapolated volition.
As with all things we need to start small. Create one little tool each day and see if it can do anything useful. We’ll have a lot of failures or mehs and occasional sparks of brilliance. Latch onto those sparks and accelerate them 10X. I think the ExoBrain will be modular with many tendrils accelerating different parts of ourselves.
Essentially, we should have various AI tools continually analyzing every aspect of our projects and our lives, and putting millions of tokens into figuring how it can help us optimize in little ways and big ways:
In general, across all these areas, the AI should be generating and prioritizing between thousands of little experiments we could be trying to determine the very best ones we could try and see if they create genuine improvements in our lives.
At the same time, we will save all of the best examples of when the ExoBrain performs unusually well or when it makes harmful mistakes, and feed those back to it. We will constantly play with the architecture and tinker with the prompts and the settings and the workflows and the tooling. We’ll keep expanding with new experiments and then compressing to what is truly useful, gradually building more and more powerful ExoBrains that are increasingly good at helping us become our best selves.
We could open source these and cross pollinate insights among a community of ExoBrain enthusiasts whose explicit goal is to exponentially accelerate humanity toward better and better instantiations of Human-AI Symbiosis.
I think it’s essential that we share these AI tools with others, especially people who are trying to have a positive impact. I would love to see the EA movement becoming really good at leveraging increasingly advanced AI to maximize its positive impact, as well as shifting significant focus toward finding ways of making sure that the AI transition itself goes well.
5. AI EA
Dony writes:
I'm glad Dony brought MacAskill up. His post "Effective altruism in the age of AGI" was a big part of what triggered my initial feelings of hinge-y-ness which led to this whole exchange. A lot of people know this but don't know what to do with that information. The popularity of his post was stunning (and well-deserved) and goes to show that people are hungry for this.
I hereby state my intention to launch a Phoenix Fellowship focused on "AI for Good," A.K.A. "Good Accelerationism" or "g/acc," and especially accelerating the good de dicto[2]. It could be one of the first projects of the Order of the Phoenix (maybe we can call ourselves the "Fellowship of the Phoenix," there's a nice symmetry in having the Fellowship of the Phoenix launch the Phoenix Fellowship). (Update: current iteration of this is Flourishing Futures Fellowship, you can sign up for the email list here)
Here are some proposed core values of g/acc, the "New EA movement," which I think should include MacAskill's "New Enlightenment":
As Dony’s says, EA is great and should continue. I find the idea of forging a new offshoot culture fine-tuned for operating in the age of AGI invigorating and inspiring. Would love to hear what y’all’s ideas are for what the core values of a “New EA” should be, definitely room for debate and discussion here.
Publishing now because it suddenly seems to be mid-to-late hinge. The biggest update from a year ago is that I think people should imminently position themselves to help slow down AI, work on automating alignment, or get in position for strategies that leverage advanced AI capabilities.
What is essentially or ultimately good in the final analysis, vs. some particular thing that seems good in the moment, as discussed by Will MacAskill.