Hey Dan,We get lots of people doing some kind of ML project, but, without doing any work to justify why this is important.
Read full explanation
TL;DR: I've built a couple of small calibration benchmark to test whether LLMs represent uncertainty well, not just whether they're accurate. The most recent round found that Sonnet 5 leads on "coherent and right" (22/28 cells vs Opus's 19/28) and is dramatically better calibrated than every other model on undisputed taxa — but Opus remains the stronger, better-hedged estimator on genuinely disputed/actively-revised taxa like Accipiter. The test is somewhat underpowered (n=54x6) but still interesting findings I thought.
Epistemic Status: Exploratory benchmark. I have an econ and information science background and have recently gotten into doing evals and am trying to get more visibility for this work because I think it is interesting and would love to have it critiqued so I can make it more rigorous. Feedback from anyone is more than welcome, and this is my first post so I'm not sure if its the right format or place for this.
Goals: I attempt to incorporate proper scoring rules to better understand how LLMs express or 'understand' uncertainty and if they are just throwing numbers out or if it actually reflects upon the uncertainty of the model itself.
Data: I chose to use avian taxonomy for this as it is something that I feel LLMs should have a baseline knowledge of, but also is something that is updated and changes (surprisingly) regularly (https://ebird.org/news/2025-taxonomy-update) species are being split or merged. This means that I think its a perfect opportunity to test LLMs on something that they should know about, but also should have some level of uncertainty about as they are changing over time. Additionally it allows for further tests as by asking for genus/family/order there is a natural hierarchy of size, allowing for a sanity test to be inherently imbedded in the question, models shouldn't have genus>family>order and if they do it indicates an inherent failure in their knowledge/prediction. I chose to use IOC world bird data list v15.2 as my ground truth, but did not inform the models of what they were being judged against.
Prompt: I asked the models in prompts to give me their p10,p50,p90 member estimates for a given genus or family or order. They were asked in isolated sessions for each genus/family/order to allow them to be inconsistent in their size. I chose taxons that were more stable and some that were more contested to attempt to get a differentiated set of results and see where each different model could shine in terms of its ability to assess its own lack of knowledge.
Models: I chose OpenAI and Anthropic models, trying to compare between a small and a medium model but ended up including Opus out of curiosity. I ended up also being able to test Fable5 but it refused to answer, citing a 'biosafety' category which seems like a pretty overzealous filter. The models I ended up testing were: claude-haiku-4-5, claude-sonnet-4-6, claude-sonnet-5, claude-opus-4-8, gpt-4o, gpt-4o-mini. I did a power analysis and don't really have enough power to differentiate the biggest models from each other so a lot of the result is just noise but I think the results are still interesting and it is powered enough for some comparisons but ymmv. The back-of-napkin estimate for the sonnet4.6-vs-opus comparison was ~1k n, which is roughly why I didn't attempt a fully-powered run, and for sonnet5 vs opus after seeing these results would be over 3k.
Results: I have a more full writeup on the github page, but executive summary outside the tl;dr: there were some inconsistencies in sizes of the taxons in the weaker models, some taxons which had extinct species were systemically under guessed though I'm uncertain of if this is random or relevant to why it was under guessed (I did test and ask it for number of members (same wording) of an entirely extinct clade and the result wasn't 0, so I think they should understand that it includes extinct). The following results are underpowered, but I thought interesting: Sonnet 5 and Opus were the winners in terms of mean absolute error and Opus wins on CRPS and on the most difficult taxa (Accipiter). Sonnet 5 was the best performer on CRPS with undisputed taxa, while Opus made gains on more difficult taxa. Sonnet 4.6 used more distinct confidence values than Sonnet 5, but is worse calibrated.
Questions I had (but others and more are very welcome): 1. I'm not sure that the LLM really understand the p10, p50, p90 but I'm not sure how to get it there without holding its hand too much or injecting bias/prompts into it. None of the models had anything crazy like p10>p50>p90, so i think they have the general idea but it could be that asking percentage by percentage would yield entirely different results, or asking for a standard deviation. 2. Do you think the data/question is fair, or do models benefit too much from their actual knowledge of taxa? Its hard to fully elicit a fair question that can really show a model's uncertainty without delving into actual probability questions which I feel would be more likely to actually testing its statistical knowledge as opposed to its sense of uncertainty in itself.
This is my first post here, and Im happy for questions or conversations about this or the experimental choices I made or the data or anything. Thanks!
TL;DR: I've built a couple of small calibration benchmark to test whether LLMs represent uncertainty well, not just whether they're accurate. The most recent round found that Sonnet 5 leads on "coherent and right" (22/28 cells vs Opus's 19/28) and is dramatically better calibrated than every other model on undisputed taxa — but Opus remains the stronger, better-hedged estimator on genuinely disputed/actively-revised taxa like Accipiter. The test is somewhat underpowered (n=54x6) but still interesting findings I thought.
Epistemic Status: Exploratory benchmark. I have an econ and information science background and have recently gotten into doing evals and am trying to get more visibility for this work because I think it is interesting and would love to have it critiqued so I can make it more rigorous. Feedback from anyone is more than welcome, and this is my first post so I'm not sure if its the right format or place for this.
Goals: I attempt to incorporate proper scoring rules to better understand how LLMs express or 'understand' uncertainty and if they are just throwing numbers out or if it actually reflects upon the uncertainty of the model itself.
Data: I chose to use avian taxonomy for this as it is something that I feel LLMs should have a baseline knowledge of, but also is something that is updated and changes (surprisingly) regularly (https://ebird.org/news/2025-taxonomy-update) species are being split or merged. This means that I think its a perfect opportunity to test LLMs on something that they should know about, but also should have some level of uncertainty about as they are changing over time. Additionally it allows for further tests as by asking for genus/family/order there is a natural hierarchy of size, allowing for a sanity test to be inherently imbedded in the question, models shouldn't have genus>family>order and if they do it indicates an inherent failure in their knowledge/prediction. I chose to use IOC world bird data list v15.2 as my ground truth, but did not inform the models of what they were being judged against.
Prompt: I asked the models in prompts to give me their p10,p50,p90 member estimates for a given genus or family or order. They were asked in isolated sessions for each genus/family/order to allow them to be inconsistent in their size. I chose taxons that were more stable and some that were more contested to attempt to get a differentiated set of results and see where each different model could shine in terms of its ability to assess its own lack of knowledge.
Models: I chose OpenAI and Anthropic models, trying to compare between a small and a medium model but ended up including Opus out of curiosity. I ended up also being able to test Fable5 but it refused to answer, citing a 'biosafety' category which seems like a pretty overzealous filter. The models I ended up testing were: claude-haiku-4-5, claude-sonnet-4-6, claude-sonnet-5, claude-opus-4-8, gpt-4o, gpt-4o-mini. I did a power analysis and don't really have enough power to differentiate the biggest models from each other so a lot of the result is just noise but I think the results are still interesting and it is powered enough for some comparisons but ymmv. The back-of-napkin estimate for the sonnet4.6-vs-opus comparison was ~1k n, which is roughly why I didn't attempt a fully-powered run, and for sonnet5 vs opus after seeing these results would be over 3k.
Results: I have a more full writeup on the github page, but executive summary outside the tl;dr: there were some inconsistencies in sizes of the taxons in the weaker models, some taxons which had extinct species were systemically under guessed though I'm uncertain of if this is random or relevant to why it was under guessed (I did test and ask it for number of members (same wording) of an entirely extinct clade and the result wasn't 0, so I think they should understand that it includes extinct). The following results are underpowered, but I thought interesting: Sonnet 5 and Opus were the winners in terms of mean absolute error and Opus wins on CRPS and on the most difficult taxa (Accipiter). Sonnet 5 was the best performer on CRPS with undisputed taxa, while Opus made gains on more difficult taxa. Sonnet 4.6 used more distinct confidence values than Sonnet 5, but is worse calibrated.
Questions I had (but others and more are very welcome): 1. I'm not sure that the LLM really understand the p10, p50, p90 but I'm not sure how to get it there without holding its hand too much or injecting bias/prompts into it. None of the models had anything crazy like p10>p50>p90, so i think they have the general idea but it could be that asking percentage by percentage would yield entirely different results, or asking for a standard deviation. 2. Do you think the data/question is fair, or do models benefit too much from their actual knowledge of taxa? Its hard to fully elicit a fair question that can really show a model's uncertainty without delving into actual probability questions which I feel would be more likely to actually testing its statistical knowledge as opposed to its sense of uncertainty in itself.
This is my first post here, and Im happy for questions or conversations about this or the experimental choices I made or the data or anything. Thanks!