Amalthea

His secret sauce is presumably being an Anthropic employee with infinite Claude access.

I think this kind of thing would've been more appropriate in the 2000s or so, when the issue at stake was somewhat less that the powerful people are just going to shove their preferences down everyone's throat, norms be damned, and there was still more civility going on.

What's missing in the discourse is capitalists making the case that it's none of your business whether you like it or not.

I think this is really poorly phrased, and could reasonably be read as tone-deaf and outright contemptuous.

I wonder if that's upstream of why Claude likes the "secret third thing" so much.

A lot of people I respect don't believe in "godlike" AI at all, so I am confused there.

Can you name 5? Who also have clearly thought about the topic in a serious way? (E.g. not Tyler Cowen)

Or is the issue with "godlike", but they'd be fine with "wildly superhuman".

It seemed to me that OpenAI have been somewhat intentionally fostering this (starting with providing early access to famous mathematicians and shopping any results around on twitter back when the models weren't good enough to one shot significant stuff)

My filter here is that Cowen isn't trying to be serious based on his past communications and outright dismissal of certain positions. If you take 10 random economists, I'd expect that maybe 9/10 or so would be happy to properly engage on hypothetical scenarios.

It's also a simple issue of sampling from the most extreme end of a distribution (extremely opinionated + high visibility) rather than from the distribution itself.

Please don't overly index on Tyler Cowen. He's been clearly not trying very hard to be correct on things in this area. There's probably better examples of economists who have bad takes and are more serious about it.

I don't see how "nontrivial to enforce a pause" follows (beyond it requiring some amount of international coordination at some point which is unrelated to your point).
It was my impression that the continued progress in e.g. Moore's law requires a comparable steady increase in investment and R&D spending, so if the government does a large scale intervention on the inputs, this could effectively halt progress in many ways that matter. (Some forms of algorithmic progress seem indeed hard to halt without aggressive enforcement)

Last time I mentored (for SPAR), I had to turn down a lot of folks who were more accomplished and likely to succeed (e.g. professors) so that I could accept those I felt would have more long-term potential to contribute to safety.

Wouldn't professors be more likely to only apply if they genuinely care about safety, while younger people (overall) might be still figuring things out and happy for any opportunity to advance their career?

Clearly virtue isn't the thing that actually optimizes for success with people. (Unless you specifically want to be successful with people who really care about virtue and are good at judging it)

This wouldn't have solved the problem if people hadn't stopped having children.

I used to think about this all the time.

I remember reading some quote of an observant contemporary of Malthus, who predicted that there may be hope in the most well-off + educated people not choosing to reproduce as much.

That aside, I still think the current situation is extremely surprising, and that unless you had a model that takes the effect of educated people having less children into account in an highly calibrated way, predicting overpopulation was the right call for much of the time. There might have been more evidence in the 1950s though. (And in any case, that doesn't mean the panic around it was justified)

Tsimerman may be more AGI-pilled than Gowers, it's probably just that the latter has a more approachable communication style. E.g. https://arxiv.org/abs/2507.09369

If I understand Hendryck's logic here, then caring 1/1000 as much about a random stranger as about yourself, means you care several million times more about all random strangers combined than about yourself, which you don't seem to be saying?

Aren't you dividing twice there, since you:
1) single out a stranger (thus dividing the amount you care about the average stranger by their number)
2) then apply Hendricks central number to that stranger (where now you should be applying the pooled number, since you're already ignoring all the other group members)

So I think this in fact pretty close to your intuition if interpreted correctly (you say 1e-3, Hendricks says 1e-2).

higher margins rightfully means higher market cap. if your company is barely scraping by, youre not producing as much value.

This should be capturing rather than producing. (Arguably Meta produces negative value).

Wow, kudos - I genuinely wasn't sure what was going on with the essay since the tone was indeed almost Borges-style (or perhaps a more grounded Bjartur).

I'll just say that this doesn't read like Linch.