Jev as a CoT Monitor: 6x Faster and 500x Cheaper!
Jev is a new model format where instead of outputting text, it outputs certainties for a defined set of options. Due to this structure, it’s extremely fast! Naturally, a classification task that comes to mind is monitoring harmful thought traces. I wanted to see how it performed at Chain of...
Sep 2327