Adding to the list of models being trained to replicate human communications (such as audio, visual) a recently unveiled model's demos boast that "talking with it feels like talking with a person…full of emotion, nuance, pauses, and a natural cadence". The demos are indeed very impressive and would certainly pass the audio Turing test for almost every listener.

However, I struggle to see what the benefits of making more realistic approximations of humans in audiovisual media are. The team lists these few reasons:

Three reasons: curiosity (aka fun), better understanding between humans and AI, and safety.
Babies learn to talk without ever seeing a word written down. We wanted to see if a model could too. It can. It’s just very hard.

Speech is the most natural interface to any intelligence, super or otherwise, and today’s voice AI wastes it. It flattens what we say into text and can’t keep up with human timing.

AI keeps gaining power and independence, and it’s still nearly deaf to the emotion in our voices. Imagine being in serious pain while a voice AI has no idea and just keeps going. That’s the safety problem. Dessa is a step toward fixing it.

I can agree that curiosity is a good faith reason to do this, but I find the other two points highly suspect at best, or intentionally misleading at worst. Speech is around 50-60% the speed of reading, so "can't keep up with human timing" is plain false. Human language is also not "the most natural interface to any intelligence", much less to a superintelligence. I also struggle to understand what about being in serious pain is being fixed by having an AI assistant which has slightly more human-like speech.

Rather, I see many potential use cases for this that are directly in contravention to their stated goal of keeping humans more safe. By enabling better deepfakes, models such as this are directly contributing to strongly negative utility outcomes for many people, most of which are third parties being affected as a result of their product.

Consider in what scenarios it would be necessary to fool someone that the voice they are listening to is that of a human's, or perhaps the voice of a specific human. Maybe you could construct some very specific scenarios in which it is a net positive, but in most cases the outcome is to fool the recipient of the audio into thinking something or taking an action they might not otherwise have chosen if they had known that the voice did not belong to a real human. POSIWID: a model that enables an AI to pass off as a real human most likely will not be used by people with purely benign intentions.