LESSWRONG
LW

Wikitags

Quantilization

Edited by plex, Mateusz Bagiński, et al. last updated 2nd Dec 2024

A Quantilizer is a proposed AI design that aims to reduce the harms from Goodhart's law and specification gaming by selecting reasonably effective actions from a distribution of human-like actions, rather than maximizing over actions. It is more of a theoretical tool for exploring ways around these problems than a practical buildable design.

See also

  • Quantilizers: AI That Doesn't Try Too Hard by Rob Miles
  • Arbital page on Quantilizers
  • Quantilizers: A Safer Alternative to Maximizers for Limited Optimization by Jessica Taylor (original paper)
Subscribe
1
Subscribe
1
Discussion0
Discussion0
Posts tagged Quantilization
33Quantilizers maximize expected utility subject to a conservative cost constraint
Ω
jessicata
10y
Ω
3
26Another view of quantilizers: avoiding Goodhart's Law
Ω
jessicata
10y
Ω
2
65When to use quantilization
Ω
RyanCarey
7y
Ω
5
14Quantilal control for finite MDPs
Ω
Vanessa Kosoy
7y
Ω
0
9Computing an exact quantilal policy
Ω
Vanessa Kosoy
7y
Ω
0
119Soft optimization makes the value target bigger
Ω
Jeremy Gillen
3y
Ω
20
24Quantilizers and Generative Models
Ω
Adam Jermyn
3y
Ω
5
25Why don't quantilizers also cut off the upper end of the distribution?
QΩ
Alex_Altair, Jeremy Gillen
2y
QΩ
2
44[Aspiration-based designs] 1. Informal introduction
Ω
B Jacobs, Jobst Heitzig, Simon Fischer, Simon Dima
1y
Ω
4
37The murderous shortcut: a toy model of instrumental convergence
Ω
Thomas Kwa
11mo
Ω
0
20Hedonic Loops and Taming RL
Ω
beren
2y
Ω
14
11Quantilizer ≡ Optimizer with a Bounded Amount of Output
Ω
itaibn0
4y
Ω
4
47How to safely use an optimizer
Ω
Simon Fischer
1y
Ω
21
38AISC team report: Soft-optimization, Bayes and Goodhart
Simon Fischer, benjaminko, jazcarretao, DFNaiff, Jeremy Gillen
2y
2
30Recursive Quantilizers II
Ω
abramdemski
5y
Ω
15
Load More (15/18)
Add Posts