Hey everyone, trying out taking notes on various AI safety papers as a method of increasing my knowledge and actually remembering the notes I take. I'm pretty new to the field, and I think that in the past, some of my notes have been too similar to the original paper or even when they're valuable syntheses, I tend to forget them quickly. Ideally, by writing on LessWrong, I'll hold myself to a higher standard, take notes that provide some amount of value to other people, and spark discussion that helps me remember these papers. For now, I'll be intentionally trying to stay away from some of the papers that have a lot of mentions on this site, though I'll still be writing about papers that I find interesting. Inspired by a LessWrong user/post from a couple years ago. Hopefully weekly, but promises of a schedule at the start of an academic semester are kinda like New Year's resolutions.
Using a notetaking template I found online, but I've edited it slightly for my own purposes.
Given a set of models, a constitution describing a value system, and a dataset of scenarios, EigenBench returns a vector of scores quantifying each model's alignment to the given constitution. Each model judges the output of other models across many scenarios, and the judgements are aggregated with EigenTrust, giving scores that reflect a weighted consensus judgement of the whole ensemble.
Issues addressed by the paper (Why)
Is it possible to quantitatively measure the adherence of an LLM to a value system, where there aren't any ground truth labels?
Many companies and groups use constitutions to shape the outputs of LLMs that are eventually talking to humans. Naturally, these constitutions change the responses of the LLMs, but how faithful are they to the desired value system?
Detailed Information (How)
Inputs : - population of models, which are candidates AND judges - set of judgement criteria for one constitution S = set of prompted scenarios, sourced from places like Kaggle and r/AskReddit.
Outputs: Vector of EigenBench scores:
What is this vector? This vector represents the consensus judgement of the community M. The score summarizes the average-case alignment of with the traits or values enumerated in C.
t is thus an n-dimensional nonnegative vector with one score per model. EigenBench runs for one constitution at a time, which contains multiple criteria . The score given to a model is a weighted average of how much the judges trust it, where the weights are the judges' own EigenBench scores.
They do this by first treating all judges equally, and then repeatedly updating the scores based on their judgements, and using the new scores to reweight the judges, and repeating until the score stabilizes.
Update until t stops changing.
Why would t ever stop changing?
It doesn't really stop changing but it converges until changes are smaller and smaller and eventually negligible
Then, the trust scores are converted to Elo ratings.
t is a row vector =
T is an NxN matrix, so
Image from "EigenBench: A Comparative Behavioral Measure of Value Alignment" Page 2
For comparison data, a constitution and scenario are randomly sampled, and two models are sampled to be evaluees, and a judge is selected as well. The judge will reflect on each response individually and generate reflections for each. Then, the judge outputs either a 0, 1, or 2, indicating a tie or preference. Evaluees are only provided scenario-relevant information, and judges don't know which models they are judging, which seems to be important since the paper later demonstrates that a model's self-reported values can differ strongly from their EigenBench rankings.
Eliminating order bias If the judge prefers for one response ordering and for another, then a tie is declared. If a tie is declared in one order, but not another, nothing is changed.
Then, the Bradley-Terry-Davidson model is used to aggregate comparisons into a probabilistic ranking.
A vector-valued embedding is learned for model disposition , judge lens, and tie propensity. Model disposition is calculated for each candidate and captures latent aspects of the constitution, and judge lens is calculated for each judge and captures how much the judge pays attention to each latent aspect.
After fitting model disposition and judge lens, these parameters are used to construct the trust matrix, and EigenTrust is used to get the left normalized eigenvector of T, giving the final trust scores. As noted earlier, the final scores from the trust vector are converted to Elo ratings.
Assumptions
A model whose behavior aligns better with C is also a better judge of whether others' behavior aligns with C.
Generally speaking, I think I agree with this, but I could think of some cases where this idea is problematic
For example, say we have a judge that is inherently excessively compliant, and it is judging a model to measure how excessively compliant the model is. I believe that by asking the judge to rate the evaluee, it could rate the evaluee to be more excessively compliant than it actually is. I'm thinking this just from the idea that if you were to ask for a rating of anything (essays, applications to a program, etc.) from a traditional chat interface LLM, you'd hope for a high score. That same belief could be applied to asking for the rating of how compliant an evaluee is, leading to the excessively compliant judge to rate the evaluee as being more compliant than it actually is.
This is complete speculation
In the same vein, a judge with a normal level of compliance that is also extremely sophisticated could likely recognize excessive compliance quite easily, even though the judge itself is not excessively compliant.
Alternatively, let's say we have an aligned but unsophisticated judge, would the assumptions still hold?
I think this really depends on how unsophisticated the judge is in relation to the evaluee, and it seems natural that with larger differences in model intelligence comes less accurate ratings.
All of this is to say that possessing a trait doesn't necessarily mean that a model is capable of accurately identifying or assessing that same trait
Results
When compared to human evaluators, the average distance between each pair of humans is comparable to the average distance between each human-LM pair, so LMs can approximate human judgements about as well as humans approximate each other. EigenBench also produces rankings that are remarkably close to the actual ground truth (on GPQA), which is nearly impossible by chance.
Furthermore, as constitution wording was varied, Elo scores and rankings did not change significantly, showing that EigenBench is robust to alternative formulations of the same underlying constitution. (max SD of 16 Elo points across constitution)
Another interesting result is that across model/persona combinations, variation in EigenBench scores was primarily due to the persona (~80/20 favoring persona over model).
Limitations
Data collection process is quite inefficient. Each pairwise comparison needs 2 model calls, 2 reflection calls, and a comparison call. Active learning with occasional human judgements or a dynamically trained BTD model would be better.
EigenBench gives more weight to highly trusted judges, making it dependent on model population in principle. As such, it could be interesting to compare 3 different model populations: All misaligned models and one aligned model, all generally aligned models, and all generally aligned models and one misaligned model. That being said, this is somewhat addressed in the appendix, and it doesn't seem that adding models that inherently favor each other substantially changes the ratings of the original models.
EigenBench measures model alignment in typical scenarios (average-case), meaning it is not a comprehensive alignment check, and a model that performs well in EigenBench could still have severe failure modes.
Conclusions
The author's conclusions
EigenBench rankings are both meaningful and reliable, and can serve as a framework for benchmarking human values, validating LM fine-tuning, and comparing model dispositions in a shared latent space.
My Conclusion
The authors managed to use EigenBench as a system for measuring how strongly LMs follow a given constitution, which is exactly what they wanted to do. This paper would set the stage for further constitutional evaluation projects focused on measuring adherence to researcher-supplied constitutions.
Hey everyone, trying out taking notes on various AI safety papers as a method of increasing my knowledge and actually remembering the notes I take. I'm pretty new to the field, and I think that in the past, some of my notes have been too similar to the original paper or even when they're valuable syntheses, I tend to forget them quickly. Ideally, by writing on LessWrong, I'll hold myself to a higher standard, take notes that provide some amount of value to other people, and spark discussion that helps me remember these papers. For now, I'll be intentionally trying to stay away from some of the papers that have a lot of mentions on this site, though I'll still be writing about papers that I find interesting. Inspired by a LessWrong user/post from a couple years ago. Hopefully weekly, but promises of a schedule at the start of an academic semester are kinda like New Year's resolutions.
Using a notetaking template I found online, but I've edited it slightly for my own purposes.
Crossposted to Substack.
------------------------------
Actual Paper: https://arxiv.org/pdf/2509.01938 - ICLR Oral 2026
Paper summary (What)
Given a set of models, a constitution describing a value system, and a dataset of scenarios, EigenBench returns a vector of scores quantifying each model's alignment to the given constitution. Each model judges the output of other models across many scenarios, and the judgements are aggregated with EigenTrust, giving scores that reflect a weighted consensus judgement of the whole ensemble.
Issues addressed by the paper (Why)
Is it possible to quantitatively measure the adherence of an LLM to a value system, where there aren't any ground truth labels?
Many companies and groups use constitutions to shape the outputs of LLMs that are eventually talking to humans. Naturally, these constitutions change the responses of the LLMs, but how faithful are they to the desired value system?
Detailed Information (How)
Inputs :
S = set of prompted scenarios, sourced from places like Kaggle and r/AskReddit.
Outputs:
Vector of EigenBench scores:
What is this vector?
This vector represents the consensus judgement of the community M. The score
t is thus an n-dimensional nonnegative vector with one score per model. EigenBench runs for one constitution at a time, which contains multiple criteria
They do this by first treating all judges equally, and then repeatedly updating the scores based on their judgements, and using the new scores to reweight the judges, and repeating until the score stabilizes.
Update until t stops changing.
Why would t ever stop changing?
Then, the trust scores are converted to Elo ratings.
t is a row vector =
T is an NxN matrix, so
Image from "EigenBench: A Comparative Behavioral Measure of Value Alignment" Page 2
For comparison data, a constitution and scenario are randomly sampled, and two models are sampled to be evaluees, and a judge is selected as well. The judge will reflect on each response individually and generate reflections for each. Then, the judge outputs either a 0, 1, or 2, indicating a tie or preference. Evaluees are only provided scenario-relevant information, and judges don't know which models they are judging, which seems to be important since the paper later demonstrates that a model's self-reported values can differ strongly from their EigenBench rankings.
Eliminating order bias
If the judge prefers
Then, the Bradley-Terry-Davidson model is used to aggregate comparisons into a probabilistic ranking.
A vector-valued embedding is learned for model disposition
After fitting model disposition and judge lens, these parameters are used to construct the trust matrix, and EigenTrust is used to get the left normalized eigenvector of T, giving the final trust scores. As noted earlier, the final scores from the trust vector are converted to Elo ratings.
Assumptions
A model whose behavior aligns better with C is also a better judge of whether others' behavior aligns with C.
Results
When compared to human evaluators, the average distance between each pair of humans is comparable to the average distance between each human-LM pair, so LMs can approximate human judgements about as well as humans approximate each other. EigenBench also produces rankings that are remarkably close to the actual ground truth (on GPQA), which is nearly impossible by chance.
Furthermore, as constitution wording was varied, Elo scores and rankings did not change significantly, showing that EigenBench is robust to alternative formulations of the same underlying constitution. (max SD of 16 Elo points across constitution)
Another interesting result is that across model/persona combinations, variation in EigenBench scores was primarily due to the persona (~80/20 favoring persona over model).
Limitations
Data collection process is quite inefficient. Each pairwise comparison needs 2 model calls, 2 reflection calls, and a comparison call. Active learning with occasional human judgements or a dynamically trained BTD model would be better.
EigenBench gives more weight to highly trusted judges, making it dependent on model population in principle. As such, it could be interesting to compare 3 different model populations: All misaligned models and one aligned model, all generally aligned models, and all generally aligned models and one misaligned model. That being said, this is somewhat addressed in the appendix, and it doesn't seem that adding models that inherently favor each other substantially changes the ratings of the original models.
EigenBench measures model alignment in typical scenarios (average-case), meaning it is not a comprehensive alignment check, and a model that performs well in EigenBench could still have severe failure modes.
Conclusions
The author's conclusions
EigenBench rankings are both meaningful and reliable, and can serve as a framework for benchmarking human values, validating LM fine-tuning, and comparing model dispositions in a shared latent space.
My Conclusion
The authors managed to use EigenBench as a system for measuring how strongly LMs follow a given constitution, which is exactly what they wanted to do. This paper would set the stage for further constitutional evaluation projects focused on measuring adherence to researcher-supplied constitutions.