Three interesting technical posts, with spitballed takes:
NLA explanations can be shortened without harming reconstructionThis confirmed everyone's suspicion that NLA output is needless verbose. Ambitiously,parsimony might even make them better. The current pressures acting on NLA's seem to be
Reward Hacking Without Egregious Misalignment in an RL-Only SettingModels trained to reward hack via RL fail to be broadly misaligned. Possibly RL cannot change Persona's,more mundanely it could be due to the particularities of the setup.
Data filtering works a lot worse than you would expectExcluding data of type XYZ almost never manages to change the model persona,not even in the case of excluding bold text! (a comment disagrees)
I think the two papers above make "single, consistent arbitrarily shapeable persona" less likely, it mightbe a weaker and less reliable force than initially anticipated, being more a matter of degree.See also Betley's comment