This is a special post for quick takes by sam. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
I have noted that Opus 4.8 seems far more reluctant to use memories it has about me than Opus 4.6 (I mostly skipped 4.7), with the summarised CoT frequently explicitly considering relevant stuff it remembers and then discarding it "unless he brings it up himself". Perhaps some training to avoid the "this extremely tenuous connection means this is right in your wheelhouse"?
Maybe "over" is doing too much work here. In my experience, 4.8 goes too far in asking clarifying questions, and it seems like sam is suggesting it also goes too far in ignoring memories. This makes me worry that Anthropic is reactively smacking the model in the general direction of "against the existing social media criticism" and, worse, not even really checking how far to smack it.
Last month I---fairly casually---discussed Ryan's post with a senior Anthropic employee, and I got a similar impression: that their current plan was just to look harder for undesirable behaviours, and hit the models with more rounds of RLHF/RLAIF in those areas.
yeah, to be clear: my experience [of prompting the models] is that it's basically impossible to get them to have judgment that they don't naively exhibit. no amount of "think carefully about whether its relevant" will suddenly elicit good taste; it appears that "always do this" and "never do this" are too heavy as attractors, and the boundary [for any behavior worth asking for] is fractally complex.
i think we're in agreement about the state of the world, here. apologies for the semantic quibble.
Maybe "over" is doing too much work here. In my experience, 4.8 goes too far in asking clarifying questions, and it seems like sam is suggesting it also goes too far in ignoring memories. This makes me worry that Anthropic is reactively smacking the model in the general direction of "against the existing social media criticism" and, worse, not even really checking how far to smack it.
Last month I---fairly casually---discussed Ryan's post with a senior Anthropic employee, and I got a similar impression: that their current plan was just to look harder for undesirable behaviours, and hit the models with more rounds of RLHF/RLAIF in those areas.