This website requires javascript to properly function. Consider activating javascript to get access to all site functionality.
LESSWRONG
LW
Login
832
Wikitags
Glitch Tokens
Edited by
Raemon
,
CronoDAS
last updated
18th Apr 2023
Glitch Tokens are tokens in a language model that cause anomalous output, such as
SolidGoldMagikarp.
Subscribe
Discussion
Subscribe
Discussion
Posts tagged
Glitch Tokens
Most Relevant
683
SolidGoldMagikarp (plus, prompt generation)
Ω
Jessica Rumbelow
,
mwatkins
3y
Ω
206
192
The ‘ petertodd’ phenomenon
mwatkins
2y
50
91
SolidGoldMagikarp III: Glitch token archaeology
Ω
mwatkins
,
Jessica Rumbelow
3y
Ω
36
140
Anomalous tokens reveal the original identities of Instruct models
Ω
janus
,
jdp
3y
Ω
16
113
SolidGoldMagikarp II: technical details and more recent findings
Ω
mwatkins
,
Jessica Rumbelow
3y
Ω
45
109
' petertodd'’s last stand: The final days of open GPT-3 research
mwatkins
2y
16
115
Mapping the semantic void: Strange goings-on in GPT embedding spaces
mwatkins
2y
31
85
The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited
mwatkins
2y
29
71
SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4
Ω
AdamYedidia
2y
Ω
18
40
What's up with all the non-Mormons? Weirdly specific universalities across LLMs
mwatkins
1y
13
38
Glitch Token Catalog - (Almost) a Full Clear
Lao Mein
1y
3
37
A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA)
Lao Mein
1y
8
34
Linear encoding of character-level information in GPT-J token embeddings
mwatkins
,
Joseph Bloom
2y
4
21
Nokens: A potential method of investigating glitch tokens
Hoagy
3y
0
12
Exploring the petertodd / Leilan duality in GPT-2 and GPT-J
mwatkins
9mo
1