This website requires javascript to properly function. Consider activating javascript to get access to all site functionality.
LESSWRONG
Tags
LW
Login
Glitch Tokens
Edit
History
Subscribe
Discussion
(0)
Help improve this page
Edit
History
Subscribe
Discussion
(0)
Help improve this page
Glitch Tokens
Random Tag
Contributors
6
CronoDAS
6
Raemon
Glitch Tokens are tokens in a language model that cause anomalous output, such as
SolidGoldMagikarp.
Posts tagged
Glitch Tokens
Most Relevant
10
671
SolidGoldMagikarp (plus, prompt generation)
Ω
Jessica Rumbelow
,
mwatkins
2y
Ω
205
10
189
The ‘ petertodd’ phenomenon
mwatkins
1y
50
6
91
SolidGoldMagikarp III: Glitch token archaeology
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
32
3
137
Anomalous tokens reveal the original identities of Instruct models
Ω
janus
,
jdp
2y
Ω
16
3
111
SolidGoldMagikarp II: technical details and more recent findings
Ω
mwatkins
,
Jessica Rumbelow
2y
Ω
45
3
108
' petertodd'’s last stand: The final days of open GPT-3 research
mwatkins
8mo
16
2
114
Mapping the semantic void: Strange goings-on in GPT embedding spaces
mwatkins
10mo
31
2
85
The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited
mwatkins
1y
29
2
71
SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4
Ω
AdamYedidia
1y
Ω
18
2
40
What's up with all the non-Mormons? Weirdly specific universalities across LLMs
mwatkins
6mo
13
2
37
Glitch Token Catalog - (Almost) a Full Clear
Lao Mein
15d
3
2
37
A New Class of Glitch Tokens - BPE Subtoken Artifacts (BSA)
Lao Mein
16d
7
2
34
Linear encoding of character-level information in GPT-J token embeddings
mwatkins
,
Joseph Bloom
11mo
4
2
21
Nokens: A potential method of investigating glitch tokens
Hoagy
2y
0
1
25
A Search for More ChatGPT / GPT-3.5 / GPT-4 "Unspeakable" Glitch Tokens
Martin Fell
1y
9