LESSWRONGTags
LW

Transformers

EditHistorySubscribe

Help improve this page

EditHistorySubscribe

Help improve this page

Contributors

Posts tagged Transformers

3

110How LLMs are and are not myopic

4mo

10

2

86Google's PaLM-E: An Embodied Multimodal Language Model

9mo

7

2

71Residual stream norms grow exponentially over the forward pass

StefanHex, TurnTrout

7mo

24

2

62Tracr: Compiled Transformers as a Laboratory for Interpretability | DeepMind

10mo

12

2

54Concrete Steps to Get Started in Transformer Mechanistic Interpretability

1y

7

2

53How fast can we perform a forward pass?

1y

9

2

23How Do Induction Heads Actually Work in Transformers With Finite Capacity?

8mo

0

1

78An Analogy for Understanding Transformers

CallumMcDougall

6mo

5

1

44Brief Notes on Transformers

1y

3

1

44Searching for Modularity in Large Language Models

NickyP, Stephen Fowler

1y

3

1

42Building a transformer from scratch - AI safety up-skilling challenge

Marius Hobbhahn

1y

1

1

42GPT-2's positional embedding matrix is a helix

4mo

18

1

32New Tool: the Residual Stream Viewer

2mo

7

1

29We Need To Know About Continual Learning

7mo

14

1

26The positional embedding matrix and previous-token heads: how do they actually work?

4mo

4