The good thing is that at least those actions on larger platforms leave evidence that can be established by the court. 

Yes, I think "runes" throw many LLMs off into the wrong simulator. Humans don't fall for this because the symbols "look" mathematical but a text-based LLM can't "see" that. The opposite happens for computer scientists: They see "[]" and start to think in programming terms such as lambda functions...

Using a much simpler prompt and without mentioning number theory or math o3 easily solves it:

There's a series of symbol sequences, composed entirely of "[", "]", and "-" in some combination that is equal to a number. Here are examples: 
...
What is the meaning of this formal notation?

Yes. I first tried things like this, too. I also tried term rewrite rules, and some of these were quite close. For example, AB -> A*(B+1) or AB -> A*(B+A) or  AB -> A*(B+index) led to some close misses (the question was which to expand first, so which associativity, I also considered expanding smaller first) but failed with later expansions. Took me half an hour to figure out that the index was not additive or multiplicative but the exponent base.

When we talk about AIs scheming, alignment faking or goal preservation, we imply there is something scheming or alignment faking or wanting to preserve its goals or escape the datacentre.

See also this previous discussion about

What is the AGI system and what is the environment? Where does the AGI system draw the boundary when reasoning about itself? 

Sure, but presumably that brought them to LW where AI safety is a topic.

But maybe the point at which you get into AI safety is less clear once you are on LW. And FO is a something that can more clearly named. So it could all just be availability heuristic.

Why do you think that is the case? I mean a) why do they reveal that only after a few drinks and b) why was that the convincing story - and not HPMoR?

[Linkpost] China's AI OVERPRODUCTION

Claim by Balaji:

China seeks to commoditize their complements. So, over the following months, I expect a complete blitz of Chinese open-source AI models for everything from computer vision to robotics to image generation.

If true, what effects would that have on the AI race and AI governance?

Yes! That's the right intuition. And the LLMs are doing the same - but we don't know their world model, and thus, the direction of the simplification can be arbitrarily off.

Drilling down on the simplifications, as suggested by Villiam might help. 

This is an interesting UI proposal and, if done right, might provide the needed transparency. Most people wouldn't read it, but some would, esp. for critical answers.

Yes, but it didn't mean that AIs could do all kinds of long tasks in 2005. And that is the conclusion many people seem to draw from the METR paper.