x

LESSWRONG
LW

alexandraabbas — LessWrong

alexandraabbas

26140

Posts

Sorted by New

Wikitag Contributions

Comments

Sorted by

No wikitag contributions to display.

Latent Adversarial Training (LAT) Improves the Representation of Refusal

alexandraabbas1y40

Yes! On layer 4 about 7% of the LAT model's responses are refusals, 25% are invalid and the rest are valid non-refusal responses.

1

Latent Adversarial Training (LAT) Improves the Representation of Refusal

alexandraabbas1y30

Initially we did some keyword matching to detect refusal but that proved to be unreliable so we reviewed all completions manually.

1

Latent Adversarial Training (LAT) Improves the Representation of Refusal

alexandraabbas1y20

Yes, we did. Indeed, the invalid response rate when steering on layers 2 and 3 was 100%. So as you said steering at these layers breaks the model.

Reducing sycophancy and improving honesty via activation steering

alexandraabbas2y10

"[...] This is because there would be no general direction towards a truth-based belief domain or away from using human modeling in output generation."

What do you mean by "human modeling in output generation"?

21Latent Adversarial Training (LAT) Improves the Representation of Refusal

1y

6