LESSWRONGTags
LW

Transformers

EditHistorySubscribe

Help improve this page

EditHistorySubscribe

Help improve this page

Contributors

Posts tagged Transformers

5

37Striking Implications for Learning Theory, Interpretability — and Safety?

8mo

4

3

126How LLMs are and are not myopic

1y

14

2

219Modern Transformers are AGI, and Human-Level

6mo

90

2

86Google's PaLM-E: An Embodied Multimodal Language Model

2y

7

2

75Residual stream norms grow exponentially over the forward pass

StefanHex, TurnTrout

1y

24

2

62Tracr: Compiled Transformers as a Laboratory for Interpretability | DeepMind

2y

12

2

56Concrete Steps to Get Started in Transformer Mechanistic Interpretability

2y

7

2

53How fast can we perform a forward pass?

2y

9

2

33AGI will be made of heterogeneous components, Transformer and Selective SSM blocks will be among them

9mo

9

2

27How Do Induction Heads Actually Work in Transformers With Finite Capacity?

1y

0

1

401Transformers Represent Belief State Geometry in their Residual Stream

4mo

100

1

88An Analogy for Understanding Transformers

CallumMcDougall

1y

6

1

77Attention SAEs Scale to GPT-2 Small

Connor Kissane, robertzk, Arthur Conmy, Neel Nanda

7mo

4

1

70Adam Optimizer Causes Privileged Basis in Transformer LM Residual Stream

Diego Caples, rrenaud

5d

7

1

55Skepticism About DeepMind's "Grandmaster-Level" Chess Without Search

Arjun Panickssery

7mo

13