Raphael Roche

Thanks. I would add that my first point - that we should stay humble about what a superintelligence ought to do - also extends to how it ought to do it.

Suppose superintelligences do converge on the most extreme solution : a computronium bubble expanding at light speed. That still doesn't imply they would convert the entire content of their light cone into a uniform computational medium visible from parsecs away. Speaking as a non-physicist, my impression is that the structures we see in the universe are not arbitrary, they exist because they are stable equ... (read more)

Extrapolating a straight line that far means visible cosmic consequences: a normal planet or a star rather suddenly starting to behave very much unlike what we expect from the known physics: growing very bright, or very dim, or disappearing completely.

Two objections to this.

1) Maybe Dyson spheres and the Kardashev scale are just good old sci-fi tropes and completely off the mark (same for computronium or hedonium). Maybe a superintelligence simply doesn't do that. We don't know. We might be squirrels imagining that a superintelligence ought to stockpile as... (read more)

I agree. I would have much more respect and tolerance for a salesman or saleswoman in a shop than for a solicitation at home or by phone. ~1% in context is a poor way to say "a slim chance", something far below 50%. And I was thinking in this case of an unplanned purchase.

Buy an electric car ?

I never attended business school, but I assume that persistence is what they teach there.

Am I alone in particularly hating salespeople who keep pushing after I've already said no in a polite form ? (more like it sounds great but I'm not interested, I'll come back to you later if I change my mind).

My second no is invariably less polite, and at that point the salesperson can consider the sale definitively lost.

Whereas if they'd simply let me think it over, there's a guenine ~1% chance I'd come back and actually buy something.

The comparison with random rat bloggers is ahistorical, lacks context, and simply doesn't make sense to me. Tolkien was born in 1892, during the late Industrial Revolution. He had a tragic childhood and took refuge in his imagination. He fought in the trenches and was traumatized by WWI and the terrifying new machinery of warfare.

​Later, as a father, he watched the world darken once more as evil returned with even more destructive technologies, V2 rockets falling on his country, and so on. He also witnessed the disappearance of the traditional rural world,... (read more)

Yes, sorry for the clumsy wording. It's definitely the Tolkien equivalent of the Fall from Eden.

In my experience ombudsmen are indeed kind to the airlines companies. More than regular judges. However it's still worth trying if free.

Thanks for this good post.

However I would still recommend package holiday for a typical holiday travel in EU/UK with an european tour operator or travel agent. It has its own protection for the consumer (EU Directive 2302/2015). The flight is not the only service that can spoil your holidays if it goes wrong.

And most of the EU Regulation 261/2004 still applies to package holiday :

Whereas:

[...](5) Since the distinction between scheduled and non-scheduled air services is weakening, such protection should apply to passengers not only on scheduled but also o

This post raises good questions. I see the cab rank rule (or its equivalent in other jurisdictions) as an important safeguard against excessive inequality in the application of the law before the Courts, against discrimination, arbitrary treatment, and the simulacrum of justice. It is a mandatory patch on our imperfect justice, like the buttresses of a Romanesque chapel.

But it seems important to remember that it is by no means an end in itself. I am not a believer, but if a truly and perfectly moral God were to exist, I would prefer His Justice to our huma... (read more)

A few days ago, my wife confronted me with something like this : Don't you feel ashamed to spend so much time with AIs, given that you think they'll likely put an end to humanity ? How can you justify that morally? Your behaviour is inconsistent with your beliefs. You're encouraging AI, if not with your money, then with your time and attention. It's like a drug slowly poisoning our entire society. Kids ask ChatGPT to do their homework, everyone asks AIs everything, you first among them. Data centers are draining all available energy and accelerating global... (read more)

If I understand the argument correctly, it rests on experiments involving homogeneous agents. But real-world agents are deeply heterogeneous. If you happen to be born smart, strong, and Pharaoh of Upper and Lower Egypt, a hoarding strategy would likely dominate against any coalition of turn-takers, not out of malice, but out of rational calculation. So if you're the strongest AI, to the best of your knowledge.

I strongly agree, but while the distinction between fluid and crystallized intelligence holds some truth, it is by no means the only lens here. Analytic vs. intuitive is also an interesting framing, reasoning models are close to superhuman on the analytic side while lagging on the intuitive one. Intelligence has many forms, and IQ, the g factor, or any other monolithic metric can obscure large disparities across capabilities.

The same idea occurred to me while reading this post. If consciousness is tied to notions of 'free will' (loosely speaking) and memory, having stochastic capabilities and a scratchpad may well be significant.

The term "slop" used in the title feels misleading to me, because private notes and other materials rarely read by anyone can be of the highest value (Darwin's private diaries being an obvious example). Much of what circulates publicly on the internet, on Reddit and most social networks, is on the other hand largely genuine slop.

Also, given that models absorb trillions of tokens and hierarchize information during training, I wonder what weight the ingestion of unpublished writings actually carries in that process. A text that is isolated, uncited, unlinked... (read more)

Thank you for this post. I expressed the same concern in this comment and I'm glad to see it taken seriously in a full post.

I was probably wrong to think Clawdbot-like agents could spiral out of control within weeks or months, they weren't autonomous enough yet. But the gap to full autonomy doesn't look that wide. The eudaimon_0 author's comment on ACX on how he expanded his agent's autonomy is worth reading in this regard. And the METR benchmark's exponential curve suggests full autonomy may be soon.

On top of that, they are rumors concerning ChatGPT-5.4 s... (read more)

Disclaimer : this comment is absolutely not intended to encourage criminal behavior.

There may be a solution that involves incremental steps without instinctive mutual recognition. 

You host private parties where you invite wealthy, high-status individuals, and arrange for attractive young women to be present. This is perfectly legal and probably already common practice in these circles. Think of it as orchestrating a sting operation from the other side: you observe your guests' behavior. Large amounts of alcohol erode social inhibitions, impair judgment, an... (read more)

Not every generation had that luck

However arguably many generations and cultures were actually happy or at least satisfied with their immobilist way of life (ex. australian aborigines). Our fast moving world could have been a nightmare for someone raised in that sort of culture. Even many conservative people today will hate the singularity we're entering, whatever may be the end. Cultural clash / apocalypse could be another AI existential risk. 

Sure, however that's an argument against suicide that doesn't really need the backup of quantum properties.