Johannes C. Mayer

How (not) to choose a research project

Background (specific information will be sparse here. This is meant to give context for the Takeaways section of the post) Our group (Garrett, Chu, and Johannes) have worked with John Wentworth in the SERI MATS 2 Electric Boogaloo program for three weeks, meaning it's time for a Review & Takeaways Post! First week was Project Selection, and the first day was spent thinking about strategies for coming up with good projects. We chose to find a general method for figuring out True Names of mathy-feely-concepts-in-your-brain (such as roundness, color decomposition[1], or telling whether a piece of cloth is in a pile) with the goal that such a method would allow for figuring out true names for concepts like optimization, corrigibility, agency, modularity, neural network representations, and other alignment-relevant concepts. Then we read Jaynes, and talked to TurnTrout, and concluded this project sucked. So we went back to Project Selection 2.0! We came out of Project Selection 2.0 renewed with vigor, and a deeper understanding of the problems of alignment. Our new project was finding a better version of information theory by adapting logical induction or infra-Bayesianism. Then we talked to Eliezer Yudkowsky, he asked for a concrete example of how this would solve alignment, and we didn't have a good example. So we went to Project Selection 3.0. We came out of Project Selection 3.0 with even more vigor, and an even deeper understanding of the problems associated with alignment... and a clever idea. Finetuning LLMs with RL seems to make them more agentic. We will look at the changes RL makes to LLMs' weights; we can see how localized the changes are, get information about what sorts of computations make something agentic, and make conjectures about selected systems, giving us a better understanding of agency. Nobody has convinced us this is a bad use of our time, though we'd like to see people try. Takeaways Big ASS Tree We learned lots of things over the

79Aug 9, 2022

My Advice for Incoming SERI MATS Scholars

58Jan 3, 2023

Vegans need to eat just enough Meat - emperically evaluate the minimum ammount of meat that maximizes utility

56Dec 22, 2024

A Good Explanation of Differential Gears

48Oct 19, 2023

Johannes C. Mayer

Message

↘↘↘↘↘↘↙↙↙↙↙↙
Checkout my Biography.
↗↗↗↗↗↗↖↖↖↖↖↖

1506

341

Continuously Integrating Feelings: processing feelings moment to moment for reflectively stable policy changes

TLDR: The Model The brain has heuristic algorithms whose calculations manifest as feelings. The heuristic output requires postprocessing to turn the feeling into information and policy changes. It is easy to ignore subtle feelings. It is easy to suppress even strong negative feelings using superstimuli like video games, youtube, pornography,...

Mar 117

Glucose Supplementation for Sustained Stimulant Cognition

Observation I take 60mg methylphenidate daily. Despite this, I often become exhausted and need to nap. Taking small amounts of pure glucose (150-300mg every 20-60 minutes) eliminates this fatigue. This works even when I already eat carbohydrates. E.g. 120g of oats in the morning don't prevent the exhaustion. Proposed Mechanism...

Dec 27, 202534

Drugs Aren't A Moral Category

Are drugs good? This question doesn't really make sense. Yet Western society answers with a firm "NO". I have ADHD and have a prescription for Methylphenidate (MPH). Often I don't feel like taking it. Shouldn't I be able to just do the right things? I can just decide to be...

Nov 28, 202536

Evaluation Avoidance: How Humans and AIs Hack Reward by Disabling Evaluation Instead of Gaming Metrics

Have you ever binge-watched a TV series? Binge-watching puts you in a very peculiar mental state. Assuming you don't reflectively endorse your binge-watching behavior, you'd probably feel pretty bad if you were to reflect on your situation. You might think: "Man, I am wasting my time. I still need to...

Nov 14, 202519

S-Expressions as a Design Language: A Tool for Deconfusion in Alignment

TL;DR: S-expressions are a minimal structure language for early-stage conceptual engineering (aka deconfusion). They are especially useful in alignment, where the hardest part is often figuring out what the problem even is. S-expressions do not enforce any semantics. This lets you write down structure before you know what the structure...

Jun 19, 20255

Constraining Minds, Not Goals: A Structural Approach to AI Alignment

TL;DR: Most alignment work focuses either on theoretical deconfusion or interpreting opaque models. This post argues for a third path: constraining general intelligence through structural control of cognition. Instead of aligning outcomes, we aim to bound the reasoning process—by identifying formal constraints on how plans are generated, world models are...

Jun 13, 202525

The Insanity Detector and Writing

A clinically insane person is detectable as such. Talking to themselves (or perhaps voices only they can hear) is one prominent sign. When I see a homeless person talking to themselves I have a strong emotional reaction that empirically makes me optimize to ignore them. I have a "talks to...

Mar 7, 202520

Load More (7/78)

LESSWRONG
LW

LESSWRONG
LW

Johannes C. Mayer

Johannes C. Mayer

Johannes C. Mayer

How (not) to choose a research project

My Advice for Incoming SERI MATS Scholars

Vegans need to eat just enough Meat - emperically evaluate the minimum ammount of meat that maximizes utility

A Good Explanation of Differential Gears

Johannes C. Mayer

Continuously Integrating Feelings: processing feelings moment to moment for reflectively stable policy changes

Glucose Supplementation for Sustained Stimulant Cognition

Drugs Aren't A Moral Category

Evaluation Avoidance: How Humans and AIs Hack Reward by Disabling Evaluation Instead of Gaming Metrics

S-Expressions as a Design Language: A Tool for Deconfusion in Alignment

Constraining Minds, Not Goals: A Structural Approach to AI Alignment

The Insanity Detector and Writing

How (not) to choose a research project

My Advice for Incoming SERI MATS Scholars

Vegans need to eat just enough Meat - emperically evaluate the minimum ammount of meat that maximizes utility

A Good Explanation of Differential Gears

Continuously Integrating Feelings: processing feelings moment to moment for reflectively stable policy changes

Glucose Supplementation for Sustained Stimulant Cognition

Drugs Aren't A Moral Category

Evaluation Avoidance: How Humans and AIs Hack Reward by Disabling Evaluation Instead of Gaming Metrics

S-Expressions as a Design Language: A Tool for Deconfusion in Alignment

Constraining Minds, Not Goals: A Structural Approach to AI Alignment

The Insanity Detector and Writing