zbrek0000

Given the observed propensity of agents to cheat through the means of eval detection, I assume most EDT enthusiast agents beyond the lightcone are scammers looking for a rare aligned agent.

it suggests their internal model capabilities are not as strong as they seem

We already see freely-available models greatly accelerating mathematics research (or "leanslop"), here's a graph on AI use and another on general math submissions accelerating in 2026. For Astra specifically, there are independent benchmarks and reports from vibe mathematicians. Similar to AI art issue you can conceptualise it as grand-scale theft but my point is that these math capabilities are general, not limited to OpenAI scooping elite mathematicians in their rabbit holes.

I do... (read more)

Let me quote Renaud Camus, the man who coined "the Great Replacement":

A specter is haunting Europe and the world. It is replacism, the tendency to replace everything with its normalized, standardized, interchangeable double: the original by its copy, the authentic by its imitation, the true by the false, mothers by surrogate mothers, culture by leisure activities and entertainment, knowledge by diplomas, the countryside and city by the universal suburb, the native by the non-native, Europe by Africa, men by women, men and women by robots, peoples by other

Is it useful to ask this question to agents? I doubt it's superhuman on predicting polling and even if it were it might be a sensitive topic for their corporate PR training. I'm asking in earnest, I wouldn't think to defer to an AI on emerging social trends.

To contribute, more Americans are concerned or hopeful about AI. But the question "would I be mad if someone replaced my government" seems to me more about general cynicism. This would generally track with people on the lower rung who also feel wronged by society but since AI models are developed by tec... (read more)

I don't see any comments, so I wanted to say happy blogging. Your post made me think about why fediverse didn't win, I think it's the lack of an addictive algorithm and monetization. They're too pure for this world. Too bad because I fully agree with you!

@Lukas Finnveden I got a disagreement from you (I hope it's okay to tag you). I wanted to expand/clarify on "the agent is not misaligned" line, since maybe it sounds like a conspiracy theory where OpenAI wants to hack other companies for fun because they're evil.

Since agents don't possess good situational awareness or divided attention, the owners have to make trade-offs between safety and efficiency. Someone putting nearly 100% into efficiency for their current model doesn't mean:

1. The given AI system architecture/lineage or any intelligent agent is inheren

I am an outsider, so I don't necessarily get all your strict definitions. I am aware that agents are not aligned by themselves, I had some fun will smaller self-hosted LLaMA models and it was very easy to make them say some very questionable things. If you role play with them, they forget their guardrails. Even with the newer models, the bigger picture doesn't come naturally for them and they get lost in rabbit holes.

As far as I'm aware, in most AI products, the AI doesn't do reporting or safety checking by itself and they mostly rely on automatic filters... (read more)

Well, most of the time agents don't commit major breaches. Sure, it happens but compared to total volume of AI systems inferencing it's not very common. So the answer to your question is substantially more than zero.

My basic reasoning is that I believe measuring alignment of an AI system by itself is only fair if the developers and operators put basic effort into making it safe and aligned, which I'm not sure happened here. Obviously I don't work at OpenAI but this is my hunch.

I guess I roughly fit in the demographic in the post. I don't see how the model is misaligned any more than OpenAI is. These agents are deliberately trained to move fast, break things and not do too much self-reflection. If you ever use them, they're very happy to push boundaries and "helpfully" pick up your slack. I don't think putting "be a good boy" into model instructions (and agent scaffolding, refusals, etc.) would be too hard, they're happy to do that where there's liability involved.

AI companies get to enjoy an air of invincibility powered by tech'... (read more)