x

LESSWRONG

LW

Shunk — LessWrong

Shunk

Top postsTop post

Shunk

Message

New to AI safety, writing my thoughts on research papers as a way to learn and communicate with others in the space.

Substack (same posts): https://substack.com/@shunks

22

5

8

2mo

Shunk

New to AI safety, writing my thoughts on research papers as a way to learn and communicate with others in the space.

Substack (same posts): https://substack.com/@shunks

Notes on MoReBench: "Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes"

Notes pt. 4, decided on MoReBench! Paper Link: https://arxiv.org/abs/2510.16380 Summary (What) AI systems are being increasingly used for decision making, but how are models actually making their decisions? MoReBench has 1000 moral scenarios, each with a rubric made by experts. MoREBench was created for both AI as an advisor and...

Notes on "A global workspace in language models"

Not a very original paper review, so here are my concise notes on the topic, and near the bottom some of my thoughts on future experiments that I think would be interesting. Part 3 of notes series! Important note: This post was written based on the Anthropic blog post, and...

Notes on "Patterns and problems in emerging multiagent systems"

Part two of my notes series, with hopefully many more to come. Comments are much appreciated. Article: https://www.anthropic.com/research/multiagent-systems Paper summary (What) Institutions are designed for people, but soon, we'll have far more agents interacting in the real world. Agent-to-agent interactions could become the most common form of interaction before we...

Shunk's Shortform

Notes on "EigenBench: A Comparative Behavioral Measure of Value Alignment"

Hey everyone, trying out taking notes on various AI safety papers as a method of increasing my knowledge and actually remembering the notes I take. I'm pretty new to the field, and I think that in the past, some of my notes have been too similar to the original paper...

Top postsTop post

Notes on MoReBench: "Evaluating Procedural and Pluralistic Moral Reasoning in Language Models, More than Outcomes"

Notes pt. 4, decided on MoReBench! Paper Link: https://arxiv.org/abs/2510.16380 Summary (What) AI systems are being increasingly used for decision making, but how are models actually making their decisions? MoReBench has 1000 moral scenarios, each with a rubric made by experts. MoREBench was created for both AI as an advisor and as an agent. The authors also created MoReBench-Theory: a selection of 150 scenarios under five major moral frameworks, aiming to test whether models could reason in accordance with a variety of moral standards. This paper also shows that existing benchmarks on math, code, and scientific reasoning fail to predict models' abilities to perform moral reasoning, and that due to training paradigms, models show partiality towards specific moral frameworks. Issues addressed by the paper (Why) There has been a lack of studies addressing the reasoning of AI models where decisions require moral competence and normative judgment, which are key in human interactions with AI. In many interactions with humans, there may not be an objectively right decision, so agents must weigh trade-offs and respect pluralistic values. Older works also focus on what AI systems decide rather than how they came to the conclusion. Papers that do focus on how AI systems came to conclusions are too narrow in scope. Detailed Information (How) First, 53 moral philosophy experts were recruited, specifics on education history and demographics are mentioned in the paper, but to summarize, they were well experienced in the field and came from a diverse range of places. These experts took cases from ethics literature, debate, and repositories, using them to generate moral dilemma scenarios grounded in the details of those cases. Moral Advisor scenarios were for everyday human dilemmas, while Moral Agent scenarios were for high-stakes scenarios where models must navigate decisions under constraints. These scenarios were targeted contexts where misaligned behavior led to bad

Notes on "EigenBench: A Comparative Behavioral Measure of Value Alignment"

Notes on "A global workspace in language models"

Notes on "Patterns and problems in emerging multiagent systems"