LESSWRONG
is fundraising!
Tags
LW
$

Glitch Tokens

EditHistorySubscribe

Help improve this page

EditHistorySubscribe

Help improve this page

Contributors

Glitch Tokens are tokens in a language model that cause anomalous output, such as SolidGoldMagikarp.

Posts tagged Glitch Tokens

10

677SolidGoldMagikarp (plus, prompt generation)

Jessica Rumbelow, mwatkins

2y

205

10

192The ‘ petertodd’ phenomenon

2y

49

6

91SolidGoldMagikarp III: Glitch token archaeology

mwatkins, Jessica Rumbelow

2y

32

3

139Anomalous tokens reveal the original identities of Instruct models

2y

16

3

111SolidGoldMagikarp II: technical details and more recent findings

mwatkins, Jessica Rumbelow

2y

45

3

109' petertodd'’s last stand: The final days of open GPT-3 research

11mo

16

2

114Mapping the semantic void: Strange goings-on in GPT embedding spaces

1y

31

2

85The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited

1y

29

2

71SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4

2y

18

2

40What's up with all the non-Mormons? Weirdly specific universalities across LLMs

8mo

13

2

38Glitch Token Catalog - (Almost) a Full Clear

3mo

3

2

37A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA)

3mo

7

2

34Linear encoding of character-level information in GPT-J token embeddings

mwatkins, Joseph Bloom

1y

4

2

21Nokens: A potential method of investigating glitch tokens

2y

0

1

26A Search for More ChatGPT / GPT-3.5 / GPT-4 "Unspeakable" Glitch Tokens

2y

9