Roman Malov's Shortform

Roman Malov

This is a special post for quick takes by Roman Malov. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.

I feel so exhausted by the double hype.

On the one side, there are people who are saying that AI is going to be a machine god, backing up their claims with various exponential graphs. On the other side, there are people who are claiming that this is all bullshit, that they're stochastic parrots and slop generators, that nothing revolutionary has happened, and that technology stagnated 2 years ago.

And I feel like, both groups are onto something? There are obviously a lot of limitations to current AI systems, and big tech can overblow their capabilities. But also, the speed of advancement is insane, and you can see where the trajectory is going.

But my point is more about how difficult it is to form an adequate picture of reality from people's opinions, because both groups often ignore obvious facts. Like, can we at least agree on something?

For example, some people live in a reality where a year ago an internal OpenAI model won gold on the IMO. Other people believe that's somehow "hype" and "fake".

Some people believe Mythos is a revolution in cybersecurity capabilities. Other people replicated the same vulnerability findings with open-source models and find Anthropic's noise around Project Glasswing annoying.^[1]

I'm just so tired of us being incapable of agreeing on basic facts.

^{^}
In both of those cases I actually do not know who is correct. Maybe I haven't invested enough time in this, but that is kinda the point of my complaint.

I do currently believe that I can't do Project Glasswing. (more beliefs: I could discover vulnerabilities better with an open model than without AI. The false positive rate is too high for full automation. The argument that a harness can verify full exploits proves too much, namely that a harness can power RSI and we're all already dead.) Do you think I am overconfident?

iirc when I previously encountered a replication, a comment convinced me that the model was told where to look. If your point is that I should gather evidence as would convince all my frien... (read more)

1Nate Showell5mo

Both groups are over-extrapolating from small pieces of a spiky capability distribution, but one is focusing too much on the peaks and the other on the troughs. Underestimating the spikiness of the distribution is the mistake both sides are making.

1icely5mo

This is a topic that I also happened to notice independently, and this article is still an incomplete one in my eyes in an unpublished article. I definitely see the trajectory and find AI will not go away especially with how obviously unoptimized the harnesses are. I also do not think it is actually wrong to say Mythos was half marketing, as such a supposedly all-powerful model seemingly should be able to do RSI things while it seemingly has not. Also the open source findings were done with more narrow looking at the files, where they "isolated the relevant code". But there is also an aspect of having way more compute funding available too. To steelman not being impressed at math findings it would be that it is heavily confirmer reliant, which is (seemingly) not available for more nebulous goals with more unclear success conditions. It's also incredibly obvious that AI being a broad topic + serious + online = any person is vulnerable to be picked apart by hecklers that can trick onlookers due to Obligated to Respond style reasons.

Roman Malov's Shortform

6

Roman Malov's Shortform

6

I feel so exhausted by the double hype.