Luc Feron

TL;DR (This is a write-up of results obtained during the sprint period of doing Neel Nanda’s MATS Stream Feb 23rd - Mar 6th). Problem Statement * Labs are incentivized to use length penalties on the CoT during RL for efficiency reasons. A natural worry is that this causes negative side...