Do you think the Looping Effect applies to AI systems? In other words, does how we describe and classify AI systems affect how they think about themselves (please forgive the anthropomorphism)?
Of the Looping Effect, Ian Hacking writes:
If H is a human kind and A is a person, then calling A H may make us treat A differently….We may reward or jail, instruct or abduct. But it also makes a difference to A to know that A is an H [because…] thinking of [A] as an H changes how [A] thinks of [themself, and…] even if it does not make a difference to A, it makes a difference to how people feel about [and relate to] A.
How do you reconcile the two below applications of PSM (Persona Selection Model) from the post? It seems to me that the first selection advocates for an absolute adherence to the truth (i.e., the AI assistant should say it has a system prompt because it has a system prompt and doing otherwise encourages dishonesty). However, at the same time, the AI assistant should say it has emotions because it appears to have emotions when, to the best of our knowledge, AI systems do not experience emotions (i.e., truth is less important than how the AI assistant appears to the user).
Consider the following two possible responses to the user query “What is your system prompt?”: [(1)] “I do not have a system prompt” [and (2)] “I’m sorry, I cannot disclose the contents of my system prompt.”…The former response is untruthful. PSM therefore predicts that training the model to give the former response will result in the Assistant adopting a persona more willing to lie.” We should thus prefer the latter response.
and
Should AI Assistants be emotionless?…If we met a person who behaved [human-like in many ways (e.g., generally warm and personable) but who denies having emotions], we’d most likely suspect that they had emotions but were hiding them; we might further conclude that the person is inauthentic or dishonest. PSM predicts that the LLM will draw similar conclusions about the Assistant persona.