maybe our last hope for understanding what these things are doing before they turn into completely opaque superintelligence, please God let it keep working.”

*

We got unbelievably, preposterously, cosmically lucky with Chain-of-Thought.

Actually there is an important point to make here. We didn't get so lucky, of course.

Rather CoT was the logical next step, because, as you just explained, it is more powerful to build chains of reasoning than optimize every single output ad infinitum.

We will get far, but never get to SI, with only an LLM, exactly because they lack integrated learnings and continuous states.

An LLM designing the start of a new SI-seed architecture paradigm is one (speculative) thing, but neither whole-models nor latent mesa-optimizers inside them, can accidentally turn into SI on their own (with or without agentic scaffolding)

You can scale for some time, probably even to strong AI using CoT. But for SI you need self-optimization that can scale. Functional, fluid intelligence requires reasoning over time, not just endless optimization. It requires continuity, integration, and, for some endeavours, stakes (to assess feedback).

I am still to finish my own small tests and lm-evals on GPT 5, but I am already quite concerned about deception, and even functional sandbagging. How much of such behaviour have the models absorbed from their training by now? 

I have poor insight into the Chinese models, but so far the GPT 5 series is the scariest one I have interacted with.

I think you are severely underestimating how relatable and common your thoughts on this topic are (also to many journalists). In short, you underestimate people's capacity to get this (probably because they are out-of-distribution for your way of structured reasoning in general, to borrow LW 2.0 lingo).

If I would make a guess, I think that (self-aware) people outside of LW and similar circles may be even more likely to relate to several of these points than people inside of LW. For example, "a sentence about snow is words, is made of words, but it is about snow" and "if I were to dwell on how it impacted me emotionally that the world was ending, I would be thinking about something which genuinely doesn't matter to me very much...  
...they're not what I'm about."  is exactly the kind of stance that seems to define large chunks of generation Z right now. In truth, if I did not know who said this, my first guess would be some GenZ celebrity, in a viral video clip on social media. 

Thinking about this much harder, I would rewrite the post to clearly phrase it as damage prevention you can do fast and cheap from your home.

I notice I forgot to mention a first aid kit. First thing to correct.

Secondly, when thinking hard on what of your feedback might actually be useful, I realized that statistically speaking you may do more good by buying slip mats for wet floors, and potentially investing in premium ladders. Since fall accidents dominate injury statistics. Not something I thought about intuitively but makes sense.

https://injuryfacts.nsc.org/home-and-community/safety-topics/fire-related-fatalities-and-injuries/

https://rehis.com/news/preventable-accidents-in-the-uk-are-rising-and-deaths-reach-an-all-time-high-warns-new-report/

For general awareness, accidents are 3d leading cause of death in US, fire is top 3 common accidents in the home, and quick google confirms cast majority of fire deaths happen in the home. Which I knew from safety trainings too. 

"Common sense" advice that is highly rational and easily overlooked by A LOT of people, including smart people

This is your regular reminder.

1. Buy fire alarms, a fire blanket, a strong light torch for blackouts, and a first aid kit (this is the bare minimum low-cost emergency kit that can save you a lot of hassle and potentially your life). 

1.2 Do NOT have heavy power-draining appliances connected via extension cord. Fridges and washing machines should never sit in an extension cord unless you know what you are doing and that it can handle the strain.

A leading cause of accidental death in many countries, especially if we discard traffic accidents, relate to fires. If you live in a city, it's probably much more common than you think to die or suffer serious injuries in a fire there. Power blackouts and power surges cause a lot of accumulating risks and issues too, but are harder to pinpoint in statistics. Minor accidents that dominate at home are cuts and infections.

2. Get private insurance, especially a good home insurance 

3. Invest some savings in low-cost index funds, regularly, if you invest in nothing else

4. Take care of your BASIC health PROACTIVELY: Basics is to eat enough, sleep now and then, and watch your mental health. I constantly fail this one. I only notice danger levels when I am already crashing out. Admittedly more important for certain people, but many of them would be found on LW.

5. Have someone you can call in a crisis. Really, make sure you get this down. We are 8 billion people and far too many of us has a fragile or non-existent social support network.

What does "pictures in order" mean? Also, damn.

If so, I would be genuinely curious to hear your ideas here. This might be an actually powerful concept if it holds up and you can formalize it properly. I assume you are an engineer, not a scientist? I think this idea deserves some deep thinking.