Quick Takes

I've felt for a little while now that steganography, in the sense of text that looks normal and monitorable to humans but actually allows LLMs to do general-purpose hidden reasoning, is a less likely failure mode of CoT monitoring than switching to recurrent latent reasoning (neuralese) or obvious and visible linguistic drift. I think I have a new and fairly crisp argument for this: Using general-purpose steganographic encoding and decoding itself requires a lot of serial reasoning. For example, one way you can do steganography is by partitioning the set of all tokens into disjoint groups, with synonyms placed in different sets, and every time you have multiple words to choose from with the same meaning, you encode some additional hidden meaning based on which set the word you output belongs to. To reliably output the right token to encode meaning by such a scheme, you probably need to think pretty carefully about how to follow some rules about which tokens are in which set and what you are trying to encode. Similarly, extracting the hidden message from the text requires carefully pulling out the hidden bits and then piecing together what they mean. So either the serial reasoning required for encoding and decoding should be easily monitorable, or there is some non-steganographic reason why that serial reasoning is unmonitorable. Either way, steganography isn't the first reason why CoT monitoring fails. I would guess this is true of all general-purpose steganographic encoding and decoding schemes because it seems hard to hide arbitrary messages in normal-looking text without thinking hard about what to write, but I'm not super confident about that.

AI Interpretability idea: Train an LLM with a "split-brain" architecture. That is, the data in the intermediate layers is siloed into two groups with only a small amount of interaction between the groups. This could be enforced in the linear layers by forcing the cross interaction to be a low-rank matrix, or by adding to the regularizer some matrix norm of the cross interaction matrices. I'm sure this can be generalized from MLP to transformer architecture. Next, apply standard interpretability tools to see how it uses its two sides to represent concepts. Hopefully you'll find that there are some concepts and tasks that are more focused on one side than on another. Next, try to elicit the LLM to give verbal descriptions of which concept is represented in which side. Easy mode is to explicitly tell it how it is designed, give some examples of specific concepts primarily encoded on specific sides, even fine-tune train it with those examples, and ask it to predict in which sides are concepts you haven't trained/told it about before. Hard mode is not to tell it anything about how it was designed but just give it general prompts about trying to introspect try to get it to figure this out from scratch. Maybe hand it off to some of the LLM whisperers without telling it how it was created. Finally, if you succeed in hard mode, give the same prompting to a regular LLM and pay attention to what it says. The idea is that I'm trying to elicit the LLM to introspect in a way that actually gives us information about the features and concepts it uses to generate text, rather than just tell the story of the humanoid character it is playacting as. I want to create a scenario where there is a non-obvious ground truth that we have access to that it could plausibly have access to and be able to share.

Marcus Olang's I'm Kenyan. I Don't Write Like ChatGPT. ChatGPT Writes Like Me is interesting because my pre-university education mirrored his due to shared British colonial history, so it brought back a lot of memories: [...] So, story time: I went to the top government boarding school in my country by average GPA in our equivalent of the O-Levels / GCSE. My school, which only had 4th and 5th form students (ages 16 and 17 respectively), was a benchmaxxing factory; the school average GPA was effectively the overriding cost function; all 60ish govt boarding schools were ranked nationally by this, and for the trial exams (but not the real final nationwide exam) we knew not only every school's rank nationally but every student's rank nationally across all boarding schools, all 8,000+ of us. Roughly half the teachers in my school were Excellent Teachers, a literal translation of an official designation given to something like the top 1% of teachers nationwide in one of the standardised subjects as evaluated by (if I recall correctly) student average GPA on our GCSE equivalent in their subject of specialisation. Every student in my school was a straight-A student in the Lower Certificate of Education (LCE), another nationwide standardised exam taken by all 3rd form students. The results spoke for themselves: something like 1-2% of the half-million-odd students nationwide sitting the GCSE equivalent got straight As back in my day while over 60% of us did; ~0.1% got straight A+ while 3-5% of us did, so ~30x base rates. The top students got full-ride scholarships sponsored by either the government, the central bank, or government-linked O&G companies to study abroad in the UK, Australia, North America etc; this was life-changing for us. A lot of my peers came from rural areas and didn't really have good English. They did have great short-term memory, tremendous work ethic and pain tolerance, and a really intense desire to get those life-changing scholarships, because pov

what's the strongest argument for why i shouldn't auto-ignore any acausal arguments that involve hypothetical entities extremely far away (or which only exist in other Everett branches or whatever) such that we will never interact with them causally at all? a razor i have is things which are entirely epiphenomenal should be ignored because they are unfalsifiable. in particular, this seems consistent even if you accept one-boxing and paying in the counterfactual mugging. the key question is what kinds of evidence you accept as evidence of the existence of an acausal connection. in these hypotheticals, we simply declare by assumption that Omega is truthful and capable of predicting you. in reality, we would either arrive at such a belief by empirical observation (Omega has a strong track record), or pure theoretical deduction. all of the epiphenomenal acausal theories depend on pure deduction. that empirical observation has to be conveyed to us causally. it seems reasonable to draw a line and say we simply don't trust pure deduction to be able to convince us of an acausal link. also, with both the counterfactual mugging and Newcomb, even though in each instance you can't prove the other branch could have happened, in the long run those who do the right thing (pay, one box) will win. whereas with purely epiphenomenal acausal theories you will literally never find out any difference whatsoever, because the branching point happened long ago.

Here are some of my top candidates for big pushes to do right now on technical AI safety (low effort notes): * Much better model organisms / misalignment analogies: * Doing a wider set of pessimized training runs * Good candidate for lots of AI labor automation? Like maybe good to try to set up pipelines for building these envs. * Demonstrating risks from fitness-seekers/reward-seekers empirically * Even on current models with better tests, see here * Demonstrating various types of memetic spread of misalignment? * Actually do control * Build pipelines for red-teaming monitors and the agent itself. For the agent red-teaming, I'd put particular focus on checking whether it continues malign trajectories. * Scaffold integrated control features and other non-monitoring runtime control measures * Human response and auditing * Improving async and sync monitoring * Agent security features * Surveilling for rogue internal deployments (as in, building after-the-fact detection methods for rogue deployments) * Preparing for handoff and elicitation * Get AIs generically better at conceptual work * Have a plan for the evals we ultimately need to see if handoff/deference would go well and start iterating on earlier versions * These presumably will involve a bunch of manual scoring, so we'll need to build a process for it. * Analyze AI biases and epistemics and improve across many domains * Build the anti-slop/anti-mundane-misalignment coalition via doing ratings of AIs and applying some pressure to improve on these ratings. This could focus on a variety of related issues. * The hope is basically that there might be widespread interest in removing/redacting mundane misalignment and other non-misalignment behavioral problems that reduce productivity and large parts of this seem differentially good. So, if we could make this a salient metric, AI companies might improve this. A lot of the difficulty would be in measuring

Data from AI coding-agent sessions that you run on your computer, by default, not only are not anonymous, they contain frequent repetitions of your username. This is because commands and log messages sometimes use absolute paths, and those paths will be descended from your home directory, so the transcripts are littered with references to paths like /home/<username>/projects/<projectname>. Training pipelines should probably strip this out, but I'm not aware of any saying that they do so (and I had an agent look for statements to that effect and couldn't find any). This means that if you use an AI coding assistant, have your transcripts incorporated into a training run, and use the model from that training run in ways that also mention the same username, the model may be primed with much more information about you, your projects and your past coding agent interactions than you expect. There are a few things that have an extremely strong statistical imprint, in your sessions: your choice of programming language, projects, and tab size, for example. If this is happening, you would expect agents to quickly learn associations between usernames and programming languages, and might bias their new-project setup towards the language they associate with the current user. That would be harmless, and a bit useful. There are plausible scenarios where this would be pathological, however. It might produce user- or group-specific quirks, making it hard for people to collectively reason about what models are like. If training data contains a mix of users using smarter models and users using dumber models, the resulting model might perform differently depending on which group your username was in in the training data. A small but significant fraction of transcripts I see online show users being angry and abusive towards their AIs. I'm not sure what will happen when those users try out next-gen models that remember more than expected; I don't expect naive game theory or human psyc

The Mythos checkpoint that the AISI ran their cyber evals on was actually an earlier version of the model. The one released with project Glasswing is much stronger.