This website requires javascript to properly function. Consider activating javascript to get access to all site functionality.
LESSWRONG
Tags
LW
Login
Glitch Tokens
Edit
History
Subscribe
Discussion
(0)
Help improve this page
Edit
History
Subscribe
Discussion
(0)
Help improve this page
Glitch Tokens
Random Tag
Contributors
6
CronoDAS
6
Raemon
Glitch Tokens are tokens in a language model that cause anomalous output, such as
SolidGoldMagikarp.
Posts tagged
Glitch Tokens
Most Relevant
10
669
SolidGoldMagikarp (plus, prompt generation)
Ω
Jessica Rumbelow
,
mwatkins
1y
Ω
205
10
180
The ‘ petertodd’ phenomenon
mwatkins
1y
50
6
90
SolidGoldMagikarp III: Glitch token archaeology
Ω
mwatkins
,
Jessica Rumbelow
1y
Ω
30
3
137
Anomalous tokens reveal the original identities of Instruct models
Ω
janus
,
jdp
1y
Ω
16
3
109
SolidGoldMagikarp II: technical details and more recent findings
Ω
mwatkins
,
Jessica Rumbelow
1y
Ω
45
3
108
' petertodd'’s last stand: The final days of open GPT-3 research
mwatkins
4mo
16
2
114
Mapping the semantic void: Strange goings-on in GPT embedding spaces
mwatkins
5mo
31
2
85
The "spelling miracle": GPT-3 spelling abilities and glitch tokens revisited
mwatkins
10mo
29
2
71
SmartyHeaderCode: anomalous tokens for GPT3.5 and GPT-4
Ω
AdamYedidia
1y
Ω
18
2
39
What's up with all the non-Mormons? Weirdly specific universalities across LLMs
mwatkins
1mo
13
2
34
Linear encoding of character-level information in GPT-J token embeddings
mwatkins
,
Joseph Bloom
6mo
4
2
20
Nokens: A potential method of investigating glitch tokens
Hoagy
1y
0
1
23
A Search for More ChatGPT / GPT-3.5 / GPT-4 "Unspeakable" Glitch Tokens
Martin Fell
1y
9
1
5
An examination of GPT-2's boring yet effective glitch
MiguelDev
1mo
3
1
4
(redacted) Anomalous tokens might disproportionately affect complex language tasks
nikola
10mo
0