LESSWRONG
LW

158
Wikitags

Lottery Ticket Hypothesis

Edited by Multicore last updated 31st May 2021

The Lottery Ticket Hypothesis claims that neural networks used in machine learning get most of their performance from sub-networks that are already present at initialization that approximate the final policy ("winning tickets"). The training process would, under this model, work by increasing weight on the lottery ticket sub-network and reducing weight on the rest of the network.

The hypothesis was proposed in a paper by Jonathan Frankle and Micheal Carbin of MIT CSAIL.

Subscribe
Discussion
Subscribe
Discussion
Posts tagged Lottery Ticket Hypothesis
374A Mechanistic Interpretability Analysis of Grokking
Ω
Neel Nanda, Tom Lieberum
3y
Ω
48
151Understanding “Deep Double Descent”
Ω
evhub
6y
Ω
51
84Gradations of Inner Alignment Obstacles
Ω
abramdemski
4y
Ω
22
73Updating the Lottery Ticket Hypothesis
Ω
johnswentworth
4y
Ω
41
58Exploring the Lottery Ticket Hypothesis
Rauno Arike
2y
3
50Understanding the Lottery Ticket Hypothesis
Ω
Alex Flint
4y
Ω
9
25What happens to variance as neural network training is scaled? What does it imply about "lottery tickets"?
QΩ
abramdemski, evhub
5y
QΩ
4
14Does the lottery ticket hypothesis suggest the scaling hypothesis?
QΩ
Daniel Kokotajlo
5y
QΩ
17
75Why Neural Networks Generalise, and Why They Are (Kind of) Bayesian
Ω
Joar Skalse
5y
Ω
59
Add Posts