Harry Esau

To be honest, I have no idea what would make sense in the context of a Chinese lab doing this. I know very little about China's legal system, but China is a one party authoritarian state, and I feel like that's bound to complicate things.

The international diplomacy part of responding to AI, if we end up properly responding to it, has always seemed very opaque and hard to reason about, at least from my perspective. I suppose my primary hope is that Congress gets far enough in their understanding of the situation such that this is relevant at all.

I certainly... (read more)

I saw a screenshot of this comment, and I was going to reply about the ex post facto prohibition, but 1a3orn beat me to it. That said, I have more detailed opinions about this, from back when Holly initially said this, so I've summarized some of them here.

My worry about Holly Elmore calling for Nuremberg trials for AI developers is less about what she directly said and more about her tone/vibes and the way she seems to conceptualize it. From the blog post where she argues for it:

But let’s imagine that we do have a “successful” warning shot that rallies the

It's interesting to me that you think mind uploading is impossible but brain emulation could be possible. I was using those words to refer to the same thing! I assume what you think here is that moving a mind from a biological to digital substrate is impossible but copying one is not? To be honest, I'm confused about how consciousness works and don't really have much of a solid opinion about this.

Anyway, I agree that we need a system which protects existing biological life if we're going to make lots of digital minds which we ought to grant rights. We also... (read more)

Out of curiosity, would you be against mind uploading/whole brain emulation, if it were possible? By "machine", do you mean nonhuman artifical intelligences or do you mean any form of mind running on a computer?

I don't necessarily disagree with you, but that's not my read of what the Pro-Human Declaration is saying. "No AI Personhood" is in the "Human Agency and Liberty" section, next to stuff like "AI should not be allowed to exploit data about the mental or emotional states of users" and "AI systems should be designed to empower, rather than enfeeble their users". In context, I would not consider their position on AI personhood to be rooted in x-risk concerns. The first two points of the declaration are "Human Control Is Non-Negotiable" and "Meaningful Human Co... (read more)

I agree in thinking this might slow OpenAI's frontier development, but I don't think it'll move overall timelines by years. OpenAI is currently building out datacenters, so I would expect a delay of at most one year (although this could change who reaches capabilities breakthroughs first).

I just don't see why an OpenAI slowdown would effect overall industry timelines that substantially. It might reduce pressure on Anthropic to ship, but I don't expect it to stall their internal development much.

It doesn't bother me that Epoch took money from OpenAI. It doesn't bother me that OpenAI has access to the FrontierMath solutions.

What does bother me is Epoch concealing this information. I certainly assumed FrontierMath was a private eval. Clearly there are people who would not have worked on this if they'd known OpenAI would have access to the dataset. I'm really not sure why Epoch or OpenAI think misleading people about this is beneficial to them—this information coming out now, like this, just means people won't trust Epoch in the future. Was the data ... (read more)