LESSWRONGAlignment Newsletter
LW

Alignment Newsletter

I publish the Alignment Newsletter, a weekly publication with recent content relevant to AI alignment. See here for more details. Quick links: email signup form, RSS feed, spreadsheet of all summaries.

12The Alignment Newsletter #1: 04/09/18Ω

5y

3

8The Alignment Newsletter #2: 04/16/18Ω

5y

0

9The Alignment Newsletter #3: 04/23/18Ω

5y

0

8The Alignment Newsletter #4: 04/30/18Ω

5y

0

8The Alignment Newsletter #5: 05/07/18Ω

5y

0

8The Alignment Newsletter #6: 05/14/18Ω

5y

0

8The Alignment Newsletter #7: 05/21/18Ω

5y

0

8The Alignment Newsletter #8: 05/28/18Ω

5y

0

8The Alignment Newsletter #9: 06/04/18Ω

5y

0

16The Alignment Newsletter #10: 06/11/18Ω

5y

0

8The Alignment Newsletter #11: 06/18/18Ω

5y

0

15The Alignment Newsletter #12: 06/25/18Ω

5y

0

70Alignment Newsletter #13: 07/02/18Ω

5y

12

14Alignment Newsletter #14Ω

5y

0

42Alignment Newsletter #15: 07/16/18Ω

5y

0

42Alignment Newsletter #16: 07/23/18Ω

5y

0

32Alignment Newsletter #17Ω

5y

0

17Alignment Newsletter #18Ω

5y

0

18Alignment Newsletter #19Ω

5y

0

12Alignment Newsletter #20Ω

5y

2

25Alignment Newsletter #21Ω

5y

0

18Alignment Newsletter #22Ω

5y

0

16Alignment Newsletter #23Ω

5y

0

10Alignment Newsletter #24Ω

5y

6

18Alignment Newsletter #25Ω

5y

3

13Alignment Newsletter #26Ω

4y

0

16Alignment Newsletter #27Ω

4y

0

11Alignment Newsletter #28Ω

4y

0

15Alignment Newsletter #29Ω

4y

0

29Alignment Newsletter #30Ω

4y

2

17Alignment Newsletter #31Ω

4y

0

18Alignment Newsletter #32Ω

4y

0

23Alignment Newsletter #33Ω

4y

0

24Alignment Newsletter #34Ω

4y

0

15Alignment Newsletter #35Ω

4y

0

21Alignment Newsletter #36Ω

4y

0

25Alignment Newsletter #37Ω

4y

4

9Alignment Newsletter #38Ω

4y

0

32Alignment Newsletter #39Ω

4y

2

21Alignment Newsletter #40Ω

4y

2

22Alignment Newsletter #41Ω

4y

6

20Alignment Newsletter #42Ω

4y

1

14Alignment Newsletter #43Ω

4y

2

18Alignment Newsletter #44Ω

4y

0

25Alignment Newsletter #45Ω

4y

2

12Alignment Newsletter #46Ω

4y

0

18Alignment Newsletter #47Ω

4y

0

29Alignment Newsletter #48Ω

4y

14

23Alignment Newsletter #49Ω

4y

1

15Alignment Newsletter #50Ω

4y

2

25Alignment Newsletter #51Ω

4y

2

19Alignment Newsletter #52Ω

4y

1

94Alignment Newsletter One Year RetrospectiveΩ

4y

31

20Alignment Newsletter #53Ω

4y

0

20[AN #54] Boxing a finite-horizon AI system to keep it unambitiousΩ

4y

0

17[AN #55] Regulatory markets and international standards as a means of ensuring beneficial AIΩ

4y

2

21[AN #56] Should ML researchers stop running experiments before making hypotheses?Ω

4y

8

26[AN #57] Why we should focus on robustness in AI safety, and the analogous problems in programmingΩ

4y

15

54[AN #58] Mesa optimization: what it is, and why we should careΩ

4y

9

43[AN #59] How arguments for AI risk have changed over timeΩ

4y

4

23[AN #60] A new AI challenge: Minecraft agents that assist human players in creative modeΩ

4y

6

12[AN #61] AI policy and governance, from two people in the field

4y

2

27[AN #62] Are adversarial examples caused by real but imperceptible features?Ω

4y

10

21[AN #63] How architecture search, meta learning, and environment design could lead to general intelligenceΩ

4y

12

11[AN #64]: Using Deep RL and Reward Uncertainty to Incentivize Preference LearningΩ

4y

8

11[AN #65]: Learning useful skills by watching humans “play”Ω

4y

0

12[AN #66]: Decomposing robustness into capability robustness and alignment robustnessΩ

3y

1

17[AN #67]: Creating environments in which to study inner alignment failuresΩ

3y

0

17[AN #68]: The attainable utility theory of impactΩ

3y

0

60[AN #69] Stuart Russell's new book on why we need to replace the standard model of AIΩ

3y

12

16[AN #70]: Agents that help humans who are still learning about their own preferencesΩ

3y

0

12[AN #71]: Avoiding reward tampering through current-RF optimizationΩ

3y

0

26[AN #72]: Alignment, robustness, methodology, and system building as research priorities for AI safetyΩ

3y

4

11[AN #73]: Detecting catastrophic failures by learning how agents tend to breakΩ

3y

0

19[AN #74]: Separating beneficial AI into competence, alignment, and coping with impactsΩ

3y

0

38[AN #75]: Solving Atari and Go with learned game models, and thoughts from a MIRI employeeΩ

3y

1

14[AN #76]: How dataset size affects robustness, and benchmarking safe exploration by measuring constraint violationsΩ

3y

6

21[AN #77]: Double descent: a unification of statistical theory and modern ML practiceΩ

3y

4

26[AN #78] Formalizing power and instrumental convergence, and the end-of-year AI safety charity comparisonΩ

3y

10

13[AN #79]: Recursive reward modeling as an alignment technique integrated with deep RLΩ

3y

0

35[AN #80]: Why AI risk might be solved without additional intervention from longtermistsΩ

3y

94

31[AN #81]: Universality as a potential solution to conceptual difficulties in intent alignmentΩ

3y

4

19[AN #82]: How OpenAI Five distributed their training computationΩ

3y

0

15[AN #83]: Sample-efficient deep learning with ReMixMatchΩ

3y

4

23[AN #84] Reviewing AI alignment work in 2018-19Ω

3y

0

14[AN #85]: The normative questions we should be asking for AI alignment, and a surprisingly good chatbotΩ

3y

2

14[AN #86]: Improving debate and factored cognition through human experimentsΩ

3y

0

28[AN #87]: What might happen as deep learning scales even further?Ω

3y

0

18[AN #88]: How the principal-agent literature relates to AI riskΩ

3y

0

16[AN #89]: A unifying formalism for preference learning algorithmsΩ

3y

0

11[AN #90]: How search landscapes can contain self-reinforcing feedback loopsΩ

3y

6

15[AN #91]: Concepts, implementations, problems, and a benchmark for impact measurementΩ

3y

10

18[AN #92]: Learning good representations with contrastive predictive codingΩ

3y

1

24[AN #93]: The Precipice we’re standing at, and how we can back away from itΩ

3y

0

11[AN #94]: AI alignment as translation between humans and machinesΩ

3y

0

20[AN #95]: A framework for thinking about how to make AI go wellΩ

3y

2

17[AN #96]: Buck and I discuss/argue about AI AlignmentΩ

3y

4

15[AN #97]: Are there historical examples of large, robust discontinuities?Ω

3y

0

22[AN #98]: Understanding neural net training by seeing which gradients were helpfulΩ

3y

3

29[AN #99]: Doubling times for the efficiency of AI algorithmsΩ

3y

0

33[AN #100]: What might go wrong if you learn a reward function while actingΩ

3y

2

15[AN #101]: Why we should rigorously measure and forecast AI progressΩ

3y

0

38[AN #102]: Meta learning by GPT-3, and a list of full proposals for AI alignmentΩ

3y

6

29[AN #103]: ARCHES: an agenda for existential safety, and combining natural language with deep RLΩ

3y

1

19[AN #104]: The perils of inaccessible information, and what we can learn about AI alignment from COVIDΩ

3y

5

24[AN #105]: The economic trajectory of humanity, and what we might mean by optimizationΩ

3y

3

14[AN #106]: Evaluating generalization ability of learned reward modelsΩ

3y

2

13[AN #107]: The convergent instrumental subgoals of goal-directed agentsΩ

3y

1

19[AN #108]: Why we should scrutinize arguments for AI riskΩ

3y

6

17[AN #109]: Teaching neural nets to generalize the way humans wouldΩ

3y

3

13[AN #110]: Learning features from human feedback to enable reward learningΩ

3y

2

23[AN #111]: The Circuits hypotheses for deep learningΩ

3y

0

25[AN #112]: Engineering a Safer WorldΩ

3y

2

23[AN #113]: Checking the ethical intuitions of large language modelsΩ

3y

0

21[AN #114]: Theory-inspired safety solutions for powerful Bayesian RL agentsΩ

3y

3

19[AN #115]: AI safety research problems in the AI-GA frameworkΩ

3y

16

21[AN #116]: How to make explanations of neurons compositionalΩ

3y

2

27[AN #117]: How neural nets would fare under the TEVV frameworkΩ

3y

0

15[AN #118]: Risks, solutions, and prioritization in a world with many AI systemsΩ

3y

6

11[AN #119]: AI safety when agents are shaped by environments, not rewardsΩ

2y

0

13[AN #120]: Tracing the intellectual roots of AI and AI alignmentΩ

2y

4

27[AN #121]: Forecasting transformative AI timelines using biological anchorsΩ

2y

5

28[AN #122]: Arguing for AGI-driven existential risk from first principlesΩ

2y

0

20[AN #123]: Inferring what is valuable in order to align recommender systemsΩ

2y

1

13[AN #124]: Provably safe exploration through shieldingΩ

2y

0

25[AN #125]: Neural network scaling laws across multiple modalitiesΩ

2y

7

24[AN #126]: Avoiding wireheading by decoupling action feedback from action effectsΩ

2y

1

46[AN #127]: Rethinking agency: Cartesian frames as a formalization of ways to carve up the world into an agent and its environmentΩ

2y

0

16[AN #128]: Prioritizing research on AI existential safety based on its application to governance demandsΩ

2y

2

14[AN #129]: Explaining double descent by measuring bias and varianceΩ

2y

1

8[AN #130]: A new AI x-risk podcast, and reviews of the fieldΩ

2y

0

13[AN #131]: Formalizing the argument of ignored attributes in a utility functionΩ

2y

4

19[AN #132]: Complex and subtly incorrect arguments as an obstacle to debateΩ

2y

1

14[AN #133]: Building machines that can cooperate (with humans, institutions, or other machines)Ω

2y

0

13[AN #134]: Underspecification as a cause of fragility to distribution shiftΩ

2y

0

33[AN #135]: Five properties of goal-directed systemsΩ

2y

0

21[AN #136]: How well will GPT-N perform on downstream tasks?Ω

2y

2

18[AN #137]: Quantifying the benefits of pretraining on downstream task performanceΩ

2y

0

12[AN #138]: Why AI governance should find problems rather than just solving themΩ

2y

0

26[AN #139]: How the simplicity of reality explains the success of neural netsΩ

2y

6

45[AN #140]: Theoretical models that predict scaling lawsΩ

2y

0

27[AN #141]: The case for practicing alignment work on GPT-3 and other large modelsΩ

2y

4

34[AN #142]: The quest to understand a network well enough to reimplement it by handΩ

2y

4

13[AN #143]: How to make embedded agents that reason probabilistically about their environmentsΩ

2y

3

19[AN #144]: How language models can also be finetuned for non-language tasksΩ

2y

0

55Alignment Newsletter Three Year RetrospectiveΩ

2y

0

19[AN #145]: Our three year anniversary!Ω

2y

0

23[AN #146]: Plausible stories of how we might fail to avert an existential catastropheΩ

2y

1

14[AN #147]: An overview of the interpretability landscapeΩ

2y

2

24[AN #148]: Analyzing generalization across more axes than just accuracy or lossΩ

2y

5

19[AN #149]: The newsletter's editorial policyΩ

2y

3

15[AN #150]: The subtypes of Cooperative AI researchΩ

2y

0

19[AN #151]: How sparsity in the final layer makes a neural net debuggableΩ

2y

0

22[AN #152]: How we’ve overestimated few-shot learning capabilitiesΩ

2y

6

25[AN #153]: Experiments that demonstrate failures of objective robustnessΩ

2y

1

12[AN #154]: What economic growth theory has to say about transformative AIΩ

2y

0

21[AN #155]: A Minecraft benchmark for algorithms that learn without reward functionsΩ

2y

5

44[AN #156]: The scaling hypothesis: a plan for building AGIΩ

2y

20

28[AN #157]: Measuring misalignment in the technology underlying CopilotΩ

2y

18

20[AN #158]: Should we be optimistic about generalization?Ω

2y

0

18[AN #159]: Building agents that know how to experiment, by training on procedurally generated gamesΩ

2y

4

28[AN #160]: Building AIs that learn and think like peopleΩ

2y

6

15[AN #161]: Creating generalizable reward functions for multiple tasks by learning a model of functional similarityΩ

2y

0

25[AN #162]: Foundation models: a paradigm shift within AIΩ

2y

0

41[AN #163]: Using finite factored sets for causal and temporal inferenceΩ

2y

0

13[AN #164]: How well can language models write code?Ω

2y

7

23[AN #165]: When large models are more likely to lieΩ

2y

0

52[AN #166]: Is it crazy to claim we're in the most important century?Ω

1y

5

19[AN #167]: Concrete ML safety problems and their relevance to x-riskΩ

1y

4

15[AN #168]: Four technical topics for which Open Phil is soliciting grant proposalsΩ

1y

0

33[AN #169]: Collaborating with humans without human dataΩ

1y

0

21[AN #170]: Analyzing the argument for risk from power-seeking AIΩ

1y

1

32[AN #171]: Disagreements between alignment "optimists" and "pessimists"Ω

1y

1

54[AN #172] Sorry for the long hiatus!Ω

9mo

0

37[AN #173] Recent language model results from DeepMindΩ

8mo

9