This is a special post for quick takes by Daniel Tan. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I am not an expert but it seems really difficult to me for hardware verification approaches to be able to distinguish RL vs inference. Like, more than 90% of the typical RL workload is just inference. This combined with the trend towards RL-dominated training pipelines makes me mildly pessimistic about anything compute governance-y working.
I explored one way to detect training vs inference with physical side channels here. Even if the window is fully dominated by inference you can still detect training due to weight updates. There are different fingerprints of training and different ways maybe to tell them apart (I explored a small fraction and still working on robustness).
Some inference verification techniques like DiFR can verify that an inference output came from a known static model. In this case, it could detect RL because the model is changing during training, and the outputs would differ from the declared static model.
There are also methods which preserve privacy. For example, "this inference output can be recomputed from this hashed 1TB object" (where the 1TB is model weights + inference code).
8ajskateboarder
I assumed because backprop is 2-3x more expensive and scales linearly it should be easier to tell apart, but yeah this is dubious. It's more useful that backprop is different enough from inference computation-wise