This is a special post for quick takes by Nition. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Nition's Shortform
6Nition
10Viliam
1Nition
1Nition
4Raemon
1Nition
3the gears to ascension
Is there any existing name for the kind of logical fallacy where one who actually considers whether they can achieve a thing is criticised above one who simply claims they'll do the thing and doesn't?
Examples abound in politics but here's one concrete example:
In 2007 the UN passed the "Declaration on the Rights of Indigenous Peoples". New Zealand, which was already putting some significant effort into supporting the rights of its indigenous people, genuinely considered whether they would be able to hold up the requirements of the declaration, and decided not to sign due to it being incredibly broad[1]. Many other countries, not doing much for their own indigenous people and recognising the declaration as non-binding, simply signed it essentially for the good vibes. As a result, New Zealand was criticised for not being willing to sign while others were, and was eventually pressured into signing (for the good vibes).
[1] See e.g. https://www.converge.org.nz/pma/decleov07.pdf For example the entire country could feasibly fall under the requirements for returning land to indigenous people.
A similar situation, I think there was a research somewhere on software developers: the ones who promise to implement a feature in two weeks but actually implement it in three weeks are perceived by the management as more competent than those who promise to implement the same feature in three weeks and do it.
Which is counter-intuitive, because the actual productivity of both was the same, and the latter had better estimates which in theory should be a plus. But I guess how it works in real life is that the former make a better first impression, and their results, while not as good as advertised, are not actually worse than those of the latter.
In case of the developers, one could worry "what if the managers make specific plans that strongly depend on having the feature ready in two weeks" but that would be too naive on the side of the management. The difference of one week usually does not matter, it's just about the general feeling of how quickly the developers work.
And in case of politics, the promises matter even less; it's not like someone's plan will depend on whether you support the indigenous people or not. You may be criticized for not doing something for them, but you will be criticized regardless of whether you made the promise or not.
The proliferation of AI bots and content on Reddit, Twitter, YouTube, everywhere, is becoming more and more visible and detrimental to the platforms. But it also occurs to me that no AI is choosing to create a Twitter account and start posting, or upload Suno tracks to Spotify, or put AI videos on YouTube. All these choices are still made by humans, mostly with the same old perverse incentive.
I have a general prediction that current-style LLMs, being inherently predictors of what a human would say, will eventually plateau at a relatively human level of ability to think and reason. Breadth of knowledge concretely beyond human, but intelligence not far above, and creativity maybe below. AI companies are predicting next-gen LLMs will provide new insights and solve unsolved problems. But genuine insight seems to require an ability to internally regenerate concepts from lower-level primitives (as mentioned in Yudkowsky’s “Truly Part Of You”). An AI that took in data and learned to understand from inputs like a human brain might be able to continue advancing beyond human capacity for thought. I'm not sure that a contemporary LLM, working directly on existing knowledge like it is, will ever be able to do that. Maybe I'll be proven wrong soon.
I think this is one of the standard rebuttals to this position: GPTs are Predictors, not Imitators
Thanks! I hadn't read that one before; it's a good point that more intelligence is required to be able to predict what any specific person might say than the intelligence of that person themselves. Having said that, I'm not convinced that a model trained on human text being super-intelligent at predicting human text necessarily means it can break out above human-level thinking.
If we discovered an intelligent alien species tomorrow, would we expect LLMs to be able to predict their next word? I'm fairly confident that the answer is "only if they thought very much like we do, just in a different language." Similarly, my suspicion is that a what-would-a-human-say predictor can never be a what-would-a-superintelligence-say predictor - or at least, only a predictor of what a human thinks a superintelligence would say.
If we discovered an intelligent alien species tomorrow, would we expect LLMs to be able to predict their next word?
maybe not well, but at least better than gzip:
