Glitch Tokens

Edited by Raemon, CronoDAS last updated 18th Apr 2023

Glitch Tokens are tokens in a language model that cause anomalous output, such as SolidGoldMagikarp.

Posts tagged Glitch Tokens

10

687SolidGoldMagikarp (plus, prompt generation)

Ω

Jessica Rumbelow, mwatkins

3y

Ω

208

10

192The ‘ petertodd’ phenomenon

mwatkins

3y

50

6

92SolidGoldMagikarp III: Glitch token archaeology

Ω

mwatkins, Jessica Rumbelow

3y

Ω

36

3

141Anomalous tokens reveal the original identities of Instruct models

Ω

janus, jdp

3y

Ω

16

3

114SolidGoldMagikarp II: technical details and more recent findings

Ω

mwatkins, Jessica Rumbelow

3y

Ω

45

3

109' petertodd'’s last stand: The final days of open GPT-3 research

mwatkins

2y

16

2

115Mapping the semantic void: Strange goings-on in GPT embedding spaces

mwatkins

2y

31

2

85The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited

mwatkins

2y

29

2

71SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4

Ω

AdamYedidia

3y

Ω

18

2

40What's up with all the non-Mormons? Weirdly specific universalities across LLMs

mwatkins

2y

13

2

38Glitch Token Catalog - (Almost) a Full Clear

Lao Mein

1y

3

2

37A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA)

Lao Mein

1y

8

2

34Linear encoding of character-level information in GPT-J token embeddings

mwatkins, Joseph Bloom

2y

4

2

21Nokens: A potential method of investigating glitch tokens

Hoagy

3y

0

2

12Exploring the petertodd / Leilan duality in GPT-2 and GPT-J

mwatkins

1y

1

LESSWRONG
LW

LESSWRONG
LW

Glitch Tokens