Abram has done such an amazing job in making the agenda accessible, I really enjoyed reading this post. I keep procastinating but I hope to have my own version of "what I understand to be sahil's agenda" out soon.
Until then I would love to jump on a video call with anyone who is confused but still wants to engage, we can talk about the agenda, maybe read through one of the posts in the live theory sequence. I have spend a lot of time around Sahil and think I have a decent grasp of his ideas.
We are raising funding for the Residency program happening in India! So if anyone wants to support us please check that project out.
You don't save the world by doing inward-looking stuff at a monastery
My model of MAPLE is that this is exactly the crux. The theory of impact that they have is based on developing a potent enough ideology (a "meme") that then spreads among powerful actors in the real world affecting their actions.
Currently they are in the design phase and are not looking to scale but eventually this "design a trustworthy religion" will become "offer the religion to cyborgs around the world". Cyborgs are those humans who closely integrate AI, tech into their workflows, building a sort of exocortex.
If we don't live in the FOOM world, then imagine how gradual disempowerment will play out. We will see increasing inequality in competence, productivity, capability as these models get more powerful. Also these cyborgs will be even more isolated, confused, lonely and due to suffering be keen to explore alternative religions to make sense of the world and decide on what motivations to pursue as meaningful, moral, etc.
It is these cyborgs that will first see consensus reality break down and traditional frameworks of meaning making will fail to help them properly orient to the world. Already we are seeing this with chatbots that are able to provide us "intimacy", making us feel seen better than our friends. When algorithms can optimise our quantified self better than our own intuition, won't more people abdicate their power to these systems?
The future of our species will depend on how empathetic, kind, compassionate, wise these cyborgs are. So I believe MAPLE is trying to design a lens to see the world, a perspective that resonates with them, that is true and ensures we continue to believe in truth being more than just predictive power, the ability to manipulate and control the external world, rather it is about living in harmony with other sentient beings.
They have an online course here which is likely more accurate than my interpretation. But during my stay with them as an AI fellow I got to interact with the students there and ask questions about why they believed the key was to cultivate the mind, how mind was chief.
Religions or ideologies are powerful because they form the framework by which we coordinate at scale. The current AI Alignment problem is not just a technical problem but a socio-technical problem, we are dealing with actors taking selfish actions, falling into race dynamics and feeling helpless due to the incentive structures around us.
In this video Soryu talks about how impactful religions have been in determining the flow of history. Towards the end he emphasises how people who deem themselves secular are in fact religious themselves. We have humanistic religions that hold human connection to be sacred and fundamental to meaning, scientific religions that hold reductionist, materialistic perspectives to be an axiomatic truth.
These systems make factual claims about what Is true about the world and then try to act innocent when normative claims (Ought) emerge (interdependently) as a consequence of believing in the factual claims. This relates to what herschel was saying about the culture being confused. If we believe that the basic ontology of reality is matter then concepts like love, joy, friendship will be reduced to transactional details, we can have allies that we cooperate with but we stop believing in anything valuable which cannot be reified. We seem to confused the map with the territory and believe that the representation can fully capture the referent.
I find all this very interesting and uncorrelated with other approaches to solve the alignment problem. I'd love to see more empirical falsifiable tests that could be designed to investigate the sort of moral realism adjacent (maybe?) implications of the claim that there is no hume's gap.
I highly recommend people watch Connor talk about his interpretation of this post.
He talks about how Eliezer is a person who managed to access many anti memes that slid right off our heads.
What is an anti meme you might ask?
Anti meme
By their very nature they resist being known or integrated into your world model. You struggle to remember them. Just like how memes are sticky, and go viral anti memes are slippery and struggle to gain traction.
They could be extraordinarily boring. They could be facts about yourself that your ego protects you from really grasping. Facts about human nature that cause cognitive dissonance because it contradicts existing beliefs you hold dear.
Insights that are anti memes are hard to communicate. You need to use fables, narratives, embody the idea into a story and convey the vibe. You use metaphors in the story.
Jokes or humor is a great way to communicate anti memes, especially ones that are socially awkward and outside people's overton window.
---
Then Connor gives the example of this post - Death with Dignity as an example of an anti meme. Most of those reading the post seem to completely miss the actual point even when it was clearly spelled out.
maybe it's somewhat easier on account of how we have more introspective access to a working mind than we have to the low-level physical fields;
We have a biased access which makes things tricker because we weren't selected for our introspection skills to be high fidelity and having a correspondence with reality. Rather it's about the utility to survival.
It doesn't have to be the result of explicit metaphysical beliefs...it could be the result of vague guesswork, and analogical thinking.
Yeah I could be wrong but my claim is implicit metaphysical beliefs have a big role here.
defining "agentic" as "possessing spooky metaphysical free will" rather than "not passive". It's perfectly possibly to build an agent-in-the-sense-of-active out of mechanical parts.
I was just noting that people who are aware of the internal workings of AI will have to acutely face cognitive dissonance if they admit it can have "spooky" agency. They can't compartmentalize it the way others can.
"topics about which philosophy is still concerned because we don't or can't get information that would enable us to have sufficient certainty of answers to allow those topics to transition into science".
I think that is quite close. I mean the implicit assumptions behind all these discussions, which are unquestioned. Moral realism, Computationalism, Empiricism, and Reductionism all come to mind. These topics cannot be tested or falsified with the scientific method.
but there's not really anything here that seems like an argument that would convince anyone who didn't already agree
I thought it would be best to try even if I am not confident it will make any impact on people reading it. My attempt is, like you rightly said, trying to get AI safety researchers to take philosophy more seriously. Most people see it as a past time that they can enjoy for intrinsic pleasure. In my opinion there is a lot of utility if we practiced going more meta until we could see the underpinnings of both the problem of x risk and the solution.
Some of the utility comes from being able to communicate it to more diverse people at higher fidelity. The rest comes from empowering existing researchers to maybe make a breakthrough in alignment itself.
A lot of these objects like values, and goals seem to exist strongly in our ontology. I would like to see people try and question these things, consider other possibilities.
This exchange between Connor and Joscha seems to be an example where Connor clearly is irritated at the question because it is trying to use philosophy to question if we should even both saving humanity, is humans bad by our own standards. I can understand how he feels completely. But notice how Joscha seems to seriously think the philosophy of what values we have and how they are justified are very important.
In this community it seems to taken as fact that the direction we align the AI towards is something to be considered after figuring out how to set the direction in anyway whatsoever. We have decoupled these two things. I would like to question these assumptions, and because I am not smart enough maybe others can also try. This needs us to unsee the boundaries we are so used to and be very careful which ones we put down.
In particular, they might unlearn it in narrow contexts related to their immediate work, but then get confused and fail to unlearn it in general, resulting in them getting confused about things like agency and free will.
Yeah, I was hoping to draw attention to this problem with my post. I love the embedded agency comic series. Yeah, the cartesian boundary is one of such boundaries which most of us have but again if we want to think about alignment honestly, I think it is worthwhile to train to unsee that too.
I will check out your book. I hope to also maybe write something that can help people grok monoism and other philosophical ideas they might want to consider in its entirety.
1Aren't non-academics and non-experts the majority,
I was talking about people who had not grokked materialism which is the majority. The people who are not aware of the technical details model AI as this black box, therefore, seem to be more open to considering that it might be agentic but that is them just deferring to an outside view that sounds convincing rather than building their own model.
so maybe people there, who are working on AI and machine learning, more often have a religious or spiritual concept of human nature, compared to their counterparts in the secularized West?
Most people I talked to were from India and it is possible there is a pattern there. But I see similar arguments come up even in the people in the west. When people say "it is just statistics", they seem to be pointing to the idea that deterministic processes can never be agentic.
I am not trying to bring consciousness into the discussion necessarily but I think there is value in helping people make their existing philosophical beliefs more explicit so that they can see it to the natural conclusion.
Thanks for the constructive critism. I thought about it and I guess I need to increase the legibility of what I wrote.
I will add a TLDR and update the post soon.
I am so happy to read this post, knowing Jake is working on this makes me glad. For sure I will be nudging more people I know to apply. I hope Coefficient gets really high quality applicants in this round.
In fact whenever I am asking for help to pivot towards contributing to the field I use the OP's technical AIS RFP as a resource to get people to think about what kind of experiments they would want to do or upskill to be able to do. so "But grantmakers don’t just evaluate proposals; we elicit them." is really powerful and true.
This definitely makes the role highly impactful and the main update I had from this is that it is more fun/motivating than you would have imagined. Thanks for sharing.