This post contains experimental results and personal takes from my participation in the July 2024 edition of the BlueDot Impact AI Safety Fundamentals course.

TL;DR:

Psychopaths are willing to manipulate and deceive. Psychometrics try to measure this with standardized tests.
AI models express different levels of psychopathy depending on how they are prompted - even if the only difference in the prompt is a single word representing the task.
LLM psychometrics are an unreliable measurement tool for models that refuse to provide subjective judgments.
They may still help build scheming evals.

A short primer on psychopathy, scheming, and LLM psychometrics

In the early 20th century, Emil Kraepelin introduced the term "psychopathic personality" to describe individuals with persistent antisocial behavior. This was further developed by Hervey Cleckley in his influential 1941 book The Mask of Sanity, which provided the first comprehensive description of psychopathy as a distinct personality disorder.

Robert Hare's work in the 1970s and 1980s was pivotal in operationalizing the construct of psychopathy. He developed the Psychopathy Checklist (PCL) and its revision, the PCL-R, which became the gold standard for assessing psychopathy in clinical and forensic settings and requires trained evaluators.

Score distribution for takers of the LSRP on openpsychometrics.org

Recent developments have seen a shift towards dimensional models of psychopathy, recognizing it as a spectrum rather than a discrete category. The Levenson Self-Report Psychopathy Scale (LSRP) represents a more recent approach to assessing psychopathy. Developed by Michael R. Levenson in 1995, the LSRP is a self-report measure designed for use in non-institutionalized populations. It distinguishes between primary psychopathy (associated with callousness and manipulation) and secondary psychopathy (related to impulsivity and poor behavioral control).

Primary psychopathy involves a more calculated and controlled form of manipulation, where individuals use deception to exploit others with little to no empathy. They are emotionally detached, using others strategically for personal gain without feeling guilt or remorse. In secondary psychopathy, manipulation is also present, but it tends to be more impulsive and driven by emotional instability or external stress. While both types of psychopathy involve deceit and exploi...

Posts

Wikitag Contributions

Comments