Steff

The is a slimmed down version which omits some extra examples but includes my theorizing about ChatGPT, my investigations of it, and my findings. Epistemic status: Pretty uncertain. I think my conclusions are probably more right than wrong, more useful than harmful, and would especially benefit people with only an average or below-average understanding of LLMs. There may be prior art that I'm unaware of; if not, maybe this will provide a launching point for others to begin deeper investigations. In Scott Alexander’s latest link roundup, he wrote: > A surprising LLM failure mode: if you ask questions like “answer with a single word: were any mammoths still alive in December”, chatbots will often answer “yes”. It seems like they lack the natural human assumption that you meant last December, and are answering that there was some December during which a mammoth was alive. I find this weird because LLMs usually seem very good at navigating the many assumptions you need to communicate at all; this one stands as a strange exception. I think I’ve got a satisfactory answer to this strange exception, but first I want to walk through my original theories, then go over the patterns I observed and proposed supporting "Principles", before finally answering about the mammoths question in particular. (Note: Unless specified otherwise, all examples were collected on GPT-4 inside of fresh conversations within fresh windows in incognito mode. When in incognito, either 4 or 5 might be used; you have to ask it “What model are you?” to distinguish them.[1]) Theories Theory 1: Time is confusing Maybe LLMs are just really bad with time? Or maybe it’s being charitable, thinking, “Of course you know there weren’t any mammoths last December. You must be asking about whether mammoths were around during the invention of ‘December’ as a concept, or something like that.” Well, LLMs are certainly not great with time… > Was there a US presidential election last week? > > Yes, the 2023