> This work was supported through the MARS (Mentorship for Alignment Research Students) program at the Cambridge AI Safety Hub (caish.org/mars). Why We Built inspect_wandb Evaluating frontier AI models can be messy; each benchmark has its own quirks, scripts, and formats, and even after a successful run, the results usually...

Sep 20, 20253

Feedback request: `eval-crypt` a simple utility to mitigate eval contamination.

Credits: side-project with Qi Guo, Daniel Polatajko, mentored by Justin Olive (MARS 3.0). PROBLEM The code and data that make up LLM evaluations may be included in LLM training corpora if they are publicly hosted on the internet in plain text. This reduces the eval's ability to assess generalisation of...

Aug 2, 20259