"Araffe" is a nonsense word familiar to anyone who works with generative AI image models or captioning in the same way that "delve" or "disclaim" are for LLMs and presents yet another clear case of an emergent AI behavior. I'm currently experimenting with generative video again and the word came to my attention as I try to improve the adherence to my prompts and mess around with conditioning. These researchers/artists investigated the origin of 'arafed' and similar words: it appears to be a case of overfitting the BLIP2 vision-language pretraining framework to the COCO dataset of image and caption which always starts off with  "a bed with" "a slice of cake" - the captions always start with 'a', so the model would start captions with 'a' and create a nonsense word like 'arrafed' or 'arrafe' or even 'arraful' around it to score better. Apparently, later versions of the framework don't exhibit this behavior.

The audience questions in the video are interesting in that they suggest to the researchers that they had the opportunity to define the meaning of the word. Which, I suppose, would undermine the point of the presentation and that it is an emergent hallucination of the AI.

Another interesting observation is that some Stable Diffusion users included it in their 'quality salad' (a word salad of words that they claim improves the overall quality of outputs that often includes "RAW photo 4K best quality subsurface scattering" etc. etc.). How true that is depends on how much you believe quality salads work at all or whether it's some kind of confirmation bias. I for one found in at least one case on a non-stable-diffusion model adding 'araffe' to the prompt caused a tiny decrease in subjective quality.

Is it too much to read into this as an example of the symbiosis or perhaps even domination of AI over our language in the future, or this is just another innocuous artefact in the same way that typographical errors like "teh" or "!!1!!" became internet in-jokes?