Probably because Carl Jung’s theories are mostly unfalsifiable woo…?

However insightful Jung was or wasn't, his ideas are not precise. Meaning, none of us knows how to reduce them to math, or how to use them in practice to apply them to non-human systems. Almost all the work needed would be in that step, and it isn't clear why starting from Jung would help more than starting from any other thinker who pondered the workings of the mind in that past several thousand years. 

As an aside, if anyone did know how to do any of this, I think I'd much rather live in a world where an AI was aligned to think of itself as being in Rawles' original position than one who thought of itself as an individuated Jungian archetype. I think both would be disastrous, but at least the former might be so indecisive it fails to do too much bad stuff.

CHAT GPT 4's answer to the question:

AI alignment research has predominantly focused on the technical and algorithmic aspects of aligning AI systems with human values and intentions, as well as ensuring AI systems are robust and safe. There are a few reasons why AI alignment research may not have extensively drawn on Carl Jung's theories of archetypes, individuation, and the collective unconscious:

1. Different domains: AI alignment research is primarily a technical field, focused on computer science, mathematics, and engineering. Carl Jung's theories are grounded in psychology and the human experience, which may not directly intersect with the core focus of AI alignment.
2. Complexity of human psychology: While Jung's theories provide valuable insights into the human psyche, they don't offer a simple, straightforward way to translate these insights into AI alignment algorithms. Human psychology is complex and nuanced, and integrating these concepts into AI systems may be challenging.
3. Lack of empirical evidence: Although Jung's theories are influential and thought-provoking, they have been criticized for a lack of empirical evidence and testability. AI alignment research often demands rigorous, evidence-based methods, which may not align well with the more theoretical and speculative nature of Jung's work.
4. Scalability: AI alignment research aims to create AI systems that can scale to handle increasingly complex tasks and environments. Incorporating Jung's theories into AI alignment may introduce additional complexity that is difficult to scale or generalize across different domains and tasks.

However, this doesn't mean that Carl Jung's theories cannot contribute to AI alignment research in any way. There may be value in exploring how these theories can inform our understanding of human values and behavior, which can be relevant when designing AI systems that must interact with and understand humans. Additionally, studying the collective unconscious and archetypes could inspire new approaches to AI alignment that account for the cultural and historical context in which AI systems are developed and deployed.

Follow-up question:

But Jungian archetypes is not enough as a bridge to the thought patterns that resides in our psyche? I believe a model can be trained with enough archetypal data...

CHAT GPT 4's answer:

It is true that Jungian archetypes might offer some insights into human thought patterns, and it is possible that a model could be trained with archetypal data to better understand human psychology. However, there are still some challenges to consider:

• Representing archetypes: Archetypes are abstract concepts that can manifest in various ways across different cultures and individuals. Finding a suitable representation of archetypes in a form that can be used as input for an AI model may be difficult.
• Incompleteness: Jung's theories and archetypes may not cover the entire spectrum of human thought patterns and behavior. While they can provide valuable insights, relying solely on Jungian archetypes may limit the AI's understanding of the nuances and complexities of human psychology.
• Subjectivity and interpretation: The interpretation of archetypes and their relevance to specific situations can be highly subjective. This subjectivity may pose challenges in training an AI model that can accurately and consistently identify and apply archetypes in a meaningful way.
• Integration with existing AI alignment approaches: AI alignment research already includes various methods and approaches to understand and align AI systems with human values. Integrating Jungian archetypes with these existing approaches may require significant effort and rethinking of the underlying principles and techniques.

That being said, exploring the potential of Jungian archetypes in AI alignment research could lead to interesting insights and novel approaches. It may be worthwhile to investigate how these archetypes can inform our understanding of human values and behavior, as well as how they can be incorporated into AI systems in a meaningful way. This could involve interdisciplinary research, combining expertise from psychology, computer science, and other fields to develop innovative solutions to AI alignment challenges.