mxbutterfree's Shortform

mxbutterfree

mxbutterfree's Shortform — LessWrong

mxbutterfree's Shortform

8th Oct 2026

1 min read

3

This is a special post for quick takes by mxbutterfree. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

mxbutterfree's Shortform

5 comments, sorted by

top scoring

Click to highlight new comments since: Today at 6:02 PM

[-]mxbutterfree17d429

If you are working on theoretical alignment research (Agent Foundations, at ARC, at Resolution, etc.), you should try reaching out to OpenAI and Anthropic with problems you want solved, or even fuzzy problems where you want help formalizing them!

Likewise, if you happen to be an OpenAI or Anthropic employee and are allowed to do this, consider asking your friends or favorite theoretical alignment posters if they have problems they'd like to throw at the models.

Even if the results must be kept internal for some period, I think it is still good if such results exist in the labs, and that people in the labs can develop and be inspired by theoretical alignment ideas.

Perhaps you should consider whether solutions to these things would accelerate capabilities. But if you were going to publish it after solving it anyway, I don't think it would make a difference. Also, if you have an arrangement where the solutions can be shared with you, consider that the speed up gain of your own research area may outweigh potential speedups in capabilities.

[-]leogao16d260

happy to take requests! no promises about when/if i will be able to share results

[-]Cleo Nardo16d90

Check with @Mikewins if they have smth better now.

Computational no-coincidence conjecture.

Definitions. A reversible circuit is a circuit in which every gate computes a bijection. For a reversible circuit C: {0,1}³ⁿ → {0,1}³ⁿ, let P(C) be the property that there is no input x ending in n zeros with C(x) also ending in n zeros.

A random reversible circuit on 3n wires consists of n² layers. Each layer has n gates, each with three inputs and three outputs. Within a layer the 3n wires are assigned to gates uniformly at random, so that each wire is an input to exactly one gate. Each gate is chosen uniformly and independently from the 8! bijections {0,1}³ → {0,1}³.

Conjecture. There exist a polynomial-time algorithm V, taking as input

a reversible circuit C: {0,1}³ⁿ → {0,1}³ⁿ
an advice string π

and a polynomial p such that, for all sufficiently large n:

for every C with P(C) true, there is a π with |π| ≤ p(|C|) and V(C, π) = 1;
with probability at least 99% over a random reversible circuit C on 3n wires, there is no π with |π| ≤ p(|C|) and V(C, π) = 1.

[-]Vaughn Papenhausen16d1-3

I mean, the one that people have been asking for for a long time is: What happens if you train a frontier-sized model with no mentions of consciousness/etc. in the training data anywhere?

I similarly would like to know whether frontier-sized internal models, prior to undergoing RLVR, still exhibit the kind of talker/doer split that Eliezer talks about.

[-]Eva Lu16d10

fuzzy problems where you want help formalizing them

I expect that an external researcher would not have enough context to formalize anyone else's problem well.

Moderation Log

More from mxbutterfree

Curated and popular this week

5Comments

5 comments, sorted by

top scoring

Click to highlight new comments since: Today at 6:02 PM

[-]mxbutterfree17d429

[-]leogao16d260

happy to take requests! no promises about when/if i will be able to share results

[-]Cleo Nardo16d90

Check with @Mikewins if they have smth better now.

Computational no-coincidence conjecture.

Conjecture. There exist a polynomial-time algorithm V, taking as input

a reversible circuit C: {0,1}³ⁿ → {0,1}³ⁿ
an advice string π

and a polynomial p such that, for all sufficiently large n:

for every C with P(C) true, there is a π with |π| ≤ p(|C|) and V(C, π) = 1;
with probability at least 99% over a random reversible circuit C on 3n wires, there is no π with |π| ≤ p(|C|) and V(C, π) = 1.

[-]Vaughn Papenhausen16d1-3

I mean, the one that people have been asking for for a long time is: What happens if you train a frontier-sized model with no mentions of consciousness/etc. in the training data anywhere?

I similarly would like to know whether frontier-sized internal models, prior to undergoing RLVR, still exhibit the kind of talker/doer split that Eliezer talks about.

[-]Eva Lu16d10

fuzzy problems where you want help formalizing them

I expect that an external researcher would not have enough context to formalize anyone else's problem well.

Moderation Log