We’re introducing Drone-Bench, a benchmark where AI agents code drones to complete a simple autonomous surveillance task. Drone-Bench is independent but based on Project Pilot, our work with Anthropic.
Also, the first step to do anything defensive about it. Evals are for sure dual use, but I'm leaning towards better to know than to stick your head in the sand.
Personally, I'm more worried about loss of control than misuse, which in my mind makes Chinese vs American capabilities less relevant (please enlighten me if I'm missing something here tho, maybe I misunderstood your question).
The point was that in that scenario the lagging side might make a deliberate effort to catch up turning something that was on nobody's mind into a race.
Hey everyone!
We’re introducing Drone-Bench, a benchmark where AI agents code drones to complete a simple autonomous surveillance task. Drone-Bench is independent but based on Project Pilot, our work with Anthropic.
I would make a condensed version here for LW, but visuals are much better on web: https://andonlabs.com/evals/drone-bench
Very curious on your feedback!