Summary: I’ve been running experiments using psycholinguistics - studying how people’s use of words sends signals about their psychology in the moment: Are they using words that suggest they’re anxious or overwhelmed? Are they using words that indicate they’re cognitively loaded? Are they using words that make them sound confident, or phrases that make it clear they’re being analytical? Psycholinguistic APIs can be used to quantify a wide range of psychological states. So I’ve been analyzing user prompts this same way, and injecting the resulting psycholinguistic scores into models to test whether model responses become more attuned to the user’s psychological state. Evaluations show that this consistently improves attunement. I’m now exploring whether these same psychological signals can also be used to detect risks like sycophancy and overreliance as they emerge in interactions.

The Problem

For the most part, AI safety is focused on outputs like reducing toxicity, bias, and hallucinations. But outputs aren’t the only risk. Interactions between humans and AI aren’t one-way - they are back-and-forth exchanges, where each message builds on the previous one. Every interaction creates an opportunity for the model to influence the user’s confidence and judgment, and over time this can create dependency. Research by Anthropic and other AI companies has identified interaction risks like sycophancy, overreliance, and the erosion of user trust. The issue is that by focusing primarily on outputs, current safety methods miss the psychological signals in user language that can reveal these risks.

Neither sentiment analysis and emotion detection can identify risky user psychological states like cognitive overload, vulnerability, or a user’s increasing risk tolerance.

Current AI safety approaches are missing a way to measure psychological risk signals in the language of user prompts, which are needed to address issues like sycophancy, overreliance and user trust.

Psycholinguistic signals as a safety vector

There is a large body of research that demonstrates that language contains signals that indicate the psychological state of its speaker or author. Validated frameworks like LIWC (Linguistic Inquiry and Word Count) are used to quantify psychological signals in language, like a person's level of cognitive load, degree of certainty, their stress level, and risk a...

Posts

Wikitag Contributions

Comments