This is a special post for quick takes by Sergii. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Some results: - Llama2 starts making mistakes after 5 numbers - Llama3 can do 10, but fails at 20 - GPT-4 can do 20 but fails at 40
The followup questions are: - what should be the name of this metric? - are the other top-scoring models like Claude similar? (I don't have access) - any bets on how many numbers will GPT-5 be able to reverse? - how many numbers should AGI be able to reverse? ASI? can this be a Turing test of sorts?
LLMs live in an abstract textual world, and do not understand the real world well (see "[Physical Concept Understanding](https://physico-benchmark.github.io/index.html#)"). We already manipulate LLM's with prompts, cut-off dates, etc... But what about going deeper by “poisoning” the training data with safety-enhancing beliefs? For example, if training data has lots of content about how hopeless, futile and dangerous for an AI it is to scheme and hack, it might be a useful safety guardrail?
In abstract sense, yes. But for me in practice finding truth means doing a check in wikipedia. It's super easy to mislead humans, so should be as easy with AI.
I have found that when using Anki for words/language learning, I frequently can't remember the correct translation exactly, but can guess the translation as one of top-3 options. In fact, this works well for me -- even knowing vaguely what the word means is very useful.
does anyone else uses Anki with non-exact answers?
The latest short story by Greg Egan is kind of a hit piece on LW/EA/longtermism. I've really enjoyed it. "DEATH AND THE GORGON" https://asimovs.com/wp-content/uploads/2025/03/DeathGorgon_Egan.pdf