LESSWRONG
LW

Gradient Hacking

Edited by Multicore last updated 27th Aug 2022

Gradient Hacking describes a scenario where a mesa-optimizer in an AI system acts in a way that intentionally manipulates the way that gradient descent updates it, likely to preserve its own mesa-objective in future iterations of the AI.

See also: Inner Alignment

1

1

Posts tagged Gradient Hacking

107Gradient hacking

6y

39

170Gradient hacking is extremely difficult

3y

22

56Gradient Filtering

3y

16

39Challenge: construct a Gradient Hacker

Thomas Larsen, Thomas Kwa

2y

10

15Some real examples of gradient hacking

4y

8

68How does Gradient Descent Interact with Goodhart?

Scott Garrabrant, evhub

7y

19

60Gradient Hacker Design Principles From Biology

3y

13

41Understanding Gradient Hacking

4y

5

39Towards Deconfusing Gradient Hacking

4y

3

38Gradient hacking: definitions and examples

3y

2

33Thoughts on gradient hacking

4y

11

16Approaches to gradient hacking

4y

8

107Programmatic backdoors: DNNs can use SGD to run arbitrary stateful computation

Fabien Roger, Buck

2y

3

55Meta learning to gradient hack

4y

11

48Goodbye, Shoggoth: The Stage, its Animatronics, & the Puppeteer – a New Metaphor

2y

8

Load More (15/25)

Add Posts