This is a special post for quick takes by agulaya24. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I've found that through what ended up being one of the most rigorous processes of research, review, and writing that I have ever undertaken, I have realized new perspectives and a deeper understanding of what I am interested in doing with my life. There is a tendency to think this feeling of clarity is temporary, fleeting. Sometimes, having a dream means trying to prove it wrong for your own sanity, to verify you are working on something that still aligns. How many trials and tribulations must it go through until it has earned its place as part of your being?
----
I am interested in founding a multidisciplinary research organization that focuses on human-AI alignment, specifically how do we ensure AI has an accurate understanding of humans as a collective and as individuals. AI is very quickly moving from a tool someone uses to an agent that acts on an individual's behalf.
This transition is currently happening in pockets, but over the next 5 years, as adoption spreads, I believe verifying AIs are acting in its user's best interests will become the primary focus. Currently, evaluating this behavioral alignment in an AI, is opaque and exists privately within leading frontier model labs. There is currently no transparent or verifiable access to an AI's current understanding of an individual user. Convention suggests only frontier providers would have access to that kind of data; a human-aligned perspective would suggest the users should have ownership and transparency over an AI's understanding or interpretation of them. A user-verified AI representation allowed to act on their behalf, owned by the individual, verified for all external parties.
In terms of the cheapest possible test, we would need to prove the organization's core thesis. An interpretive layer built from a living person's or persons' data can better align an AI's behavior with an individual user or users. Further, if that layer is better aligned, can it be used against the interests of an individual user or users. A human-backed extension of the prototype benchmark from Beyond Recall (Arxiv 2605.28969), could be used to measure an AI's behavioral alignment with its user.
A human-backed study would validate the organization's core principles: that AI alignment can be measured and that proof of that alignment should be verified and owned by the individual.