cuchullain

There have been several discussions about the importance of adversarial robustness for scalable oversight. I’d like to point out that adversarial robustness is also important under a different threat model: catastrophic misuse. For a brief summary of the argument: 1. Misuse could lead to catastrophe. AI-assisted cyberattacks, political persuasion, and...

> Recent research in mechanistic interpretability has attempted to reverse-engineer Transformer models by carefully inspecting network weights and activations. However, these approaches require considerable manual effort and still fall short of providing complete, faithful descriptions of the underlying algorithms. In this work, we introduce a procedure for training Transformers that...

Summary Forecasts of explosive growth from advanced AI often assume that AI will fully automate human labor. But there are a variety of compelling objections to this assumption. For example, regulation could prevent the automation of some tasks, or consumers could prefer human labor in others. Skeptics of explosive growth...

Anthropic, DeepMind and Google Brain are all working on strategies to train language models on their own outputs. For a brief summary of the work so far: 1. Red Teaming Language Models with Language Models (Perez et al., 2022). One model prompts another, seeking to expose undesirable generations. A third...

What are the restrictions? Broadly speaking, the policy restricts the sale of GPUs and related technology to Chinese companies. The specifics remain unclear because (a) the Department of Commerce hasn't released its official press report yet and (b) licenses to sell compute to Chinese companies will be approved on a...

See Arxiv and Twitter. From the abstract: > We argue that neural network loss landscapes contain (nearly) a single basin, after accounting for all possible permutation symmetries of hidden units. We introduce three algorithms to permute the units of one model to bring them into alignment with units of a...