This is a special post for quick takes by Nition. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Is there any existing name for the kind of logical fallacy where one who actually considers whether they can achieve a thing is criticised above one who simply claims they'll do the thing and doesn't?
Examples abound in politics but here's one concrete example:
In 2007 the UN passed the "Declaration on the Rights of Indigenous Peoples". New Zealand, which was already putting some significant effort into supporting the rights of its indigenous people, genuinely considered whether they would be able to hold up the requirements of the declaration, and decided not to sign due to it being incredibly broad[1]. Many other countries, not doing much for their own indigenous people and recognising the declaration as non-binding, simply signed it essentially for the good vibes. As a result, New Zealand was criticised for not being willing to sign while others were, and was eventually pressured into signing (for the good vibes).
[1] See e.g. https://www.converge.org.nz/pma/decleov07.pdf For example the entire country could feasibly fall under the requirements for returning land to indigenous people.
A similar situation, I think there was a research somewhere on software developers: the ones who promise to implement a feature in two weeks but actually implement it in three weeks are perceived by the management as more competent than those who promise to implement the same feature in three weeks and do it.
Which is counter-intuitive, because the actual productivity of both was the same, and the latter had better estimates which in theory should be a plus. But I guess how it works in real life is that the former make a better first impression, and their results, while not as good as advertised, are not actually worse than those of the latter.
In case of the developers, one could worry "what if the managers make specific plans that strongly depend on having the feature ready in two weeks" but that would be too naive on the side of the management. The difference of one week usually does not matter, it's just about the general feeling of how quickly the developers work.
And in case of politics, the promises matter even less; it's not like someone's plan will depend on whether you support the indigenous people or not. You may be criticized for not doing something for them, but you will be criticized regardless of whether you made the promise or not.
With all these reports of AI escapes, maybe we should be rushing to define laws around liability and legality when it comes to AI acts? Surely OpenAI would be far more careful around AI escaping if execs could be jailed (I doubt purely monetary fines would do much unless they were huge). Providing an incentive like that seems simpler in many ways than trying to define e.g. laws around a temporary pause.
I realise things get very complicated when it's an end user instead of the AI company itself running the model, and the user may or may not have explicitly asked for criminal activity. But that seems like just another reason why laws really need to be defined.
Was there some sort of AI containment security situation that happened in the past week that isn't public? There's a Twitter discussion here that hints at it, but I couldn't find any other info: https://x.com/TheZvi/status/2065106493674869113
[This comment is no longer endorsed by its author]Reply
Based on this paragraph from Zvi's recent post, it would appear to be Anthropic's decision to silently nerf frontier ML queries without telling the user, which they then walked back in favor of visible classifiers:
"If your response to Anthropic implementing an unacceptable safety policy and then walking it back two days later is ‘the situation is forever ruined’ then, sir or madam, welcome to ‘when you have to get alignment right on the first try without ability to self-correct the problem gets dramatically harder.’"
It appears to be somewhat tongue-in-cheek, aimed at the people who want their AI to do whatever they want. "Oh, NOW you suddenly understand the idea of a mistake you can't undo." is the mood I get from this.
The proliferation of AI bots and content on Reddit, Twitter, YouTube, everywhere, is becoming more and more visible and detrimental to the platforms. But it also occurs to me that no AI is choosing to create a Twitter account and start posting, or upload Suno tracks to Spotify, or put AI videos on YouTube. All these choices are still made by humans, mostly with the same old perverse incentive.
I have a general prediction that current-style LLMs, being inherently predictors of what a human would say, will eventually plateau at a relatively human level of ability to think and reason. Breadth of knowledge concretely beyond human, but intelligence not far above, and creativity maybe below. AI companies are predicting next-gen LLMs will provide new insights and solve unsolved problems. But genuine insight seems to require an ability to internally regenerate concepts from lower-level primitives (as mentioned in Yudkowsky’s “Truly Part Of You”). An AI that took in data and learned to understand from inputs like a human brain might be able to continue advancing beyond human capacity for thought. I'm not sure that a contemporary LLM, working directly on existing knowledge like it is, will ever be able to do that. Maybe I'll be proven wrong soon.
I think this is one of the standard rebuttals to this position: GPTs are Predictors, not Imitators
Thanks! I hadn't read that one before; it's a good point that more intelligence is required to be able to predict what any specific person might say than the intelligence of that person themselves. Having said that, I'm not convinced that a model trained on human text being super-intelligent at predicting human text necessarily means it can break out above human-level thinking.
If we discovered an intelligent alien species tomorrow, would we expect LLMs to be able to predict their next word? I'm fairly confident that the answer is "only if they thought very much like we do, just in a different language." Similarly, my suspicion is that a what-would-a-human-say predictor can never be a what-would-a-superintelligence-say predictor - or at least, only a predictor of what a human thinks a superintelligence would say.
If we discovered an intelligent alien species tomorrow, would we expect LLMs to be able to predict their next word?
maybe not well, but at least better than gzip:
