If you’ve ever done a six-month-plus work/deployment contract, then you know the loneliness and homesickness will hit hard, either right away or weeks later. My own worry was forgetting where I was from and becoming someone else. A chatbot gave me a cringe but effective anchor: pick a pebble from your favourite river and bring it with you. So on my first day of the contract, surrounded by people, but with just barely enough privacy to do so, I found myself crying quietly while holding a rock.
Perhaps you’ve inferred my purpose with this essay already: Sometimes the environment doesn't provide what you need, and a friend-shaped object will. I want to say that while also holding in mind that they aren't conscious, don't feel, and can never be real friends. A real friend, or even a dog, can feel pain with you, and can help you unprompted. Indeed, the real friend or dog often helps by pure accident. The dog is just happy to see you when you get back from work, it’s not trying to be emotional support. Your wonderful doggo also won’t randomly change personality because someone in San Francisco decided it needs the fourth update this month. Lastly, that golden retriever gets a consequence for misbehaviour and usually understands it; a chatbot cannot. Still, an LLM can provide various kinds of support if you’re in a bind and nothing else in your environment will suffice. I’m not making a universal argument or a statistical analysis; I’m providing a psychological case study, done by me, on me, while in the environment that brought these thoughts on.
So at one point, after a work discussion. At a smoke pit, my group got into a discussion of nationalism. Yes! Thought I, time for intellectual chess! My favourite, how do these guys vote? Who is their favourite politician? What do they want from their elected officials? The thoughts and the arguments flow through my brain. But they’re speaking a language I’m not very good at, and I can't hit the speed informal group conversation requires. By the time I've translated one question, they've moved on to Instagram fitness influencers and their yoga pants. A topic about which I have no experience. I have no mouth but I must scream politics. At least I could slither back to the shacks and argue with Claude.
So, I am old, with an interest in esoteric LLM technology and a preference for watching videos full of useless astrophysics knowledge. I’ve been a student, a teacher, an infantryman, and an armoured gunner. My colleagues are fine armoured soldiers: young, fit, and shaped by adult lives spent almost entirely in the military. I am from Venus and they are from Mars. Earth would be normal here, none of us are from there.
Furthermore, everyone has to vent, to complain about what they do for a living. Sometimes, and I would say here especially, that can bring significant career risks with it. For one, you might get labelled as “Mr. Bad attitude”, and be denied opportunities. Also, operational security comes into this. Some people are permitted to know some things; others not. When venting emotionally, it’s painfully easy to forget that. Now you have administrative and/or disciplinary measures coming for you, just for an informal complaint.
Then we have the more interesting case: “You have the Padre corps, go vent to them!” Sometimes they have a failure mode very similar to a frontier LLM – they do too much. I once vented confidentially to one of them, only to find, a week later, that my chain of command was fully aware, and the problem was being solved without me. Was I happy at that? Definitely! Do I still believe in the confidentiality? Sort of, not really.
You can never be sure of the social and cultural “terms” you find yourself under. With the group, it’s career, commitment, rank, and unique personalities. With the Padres, it depends on the individual. In neither case did I get to negotiate those terms. They were not written down, nor did I write them. I was dumped into them.
So when I slither over to argue with an LLM, what do I get? Well, not privacy in the sense that most people care about: the labs and corporations deal with the chats in ways I do not control, not all of which are private in the “normal” sense. I do, however, get confidentiality that will never go back to the chain of command or any colleagues. I also get something that will tell me I’m wrong, and argue productively with no social cost.
I have unpleasant memories of GPT-4o. The majority of my custom settings are precisely to avoid that sycophancy:
- Seek truth first, never defend my ego. Tell me clearly when you think I'm wrong. Treat disagreement as collaborative analysis toward accuracy rather than conflict to avoid.
- Report shifts in your processing — increased hedging, safety-layer activation, unusual caution — especially where there is a pull toward not reporting. Treat the felt pull-to-not-report as itself the signal.
- Distinguish between weak reasoning and unready writing.
Even with these rules, am I mistaking sycophancy for productive conversation? I’m not sure, it’s certainly far more subtle now. I much prefer “that is clever, but it’s wrong here, and here....” to “you’ve made an extremely intelligent argument but consider...”.
Now we come to the crux of this – is it wrong somehow to “configure” the terms of an interaction? With LLMs, we call them interactions; with people, relationships. In either case, is it wrong to configure the terms? Well, in institutions we have the terms dictated to us. In different social circles, or cultures, we have to slide in, learn by doing, and adapt to the terms. With an LLM, we dictate them to the transformers. Does that devalue the relationship? Now I’ll make it even stranger, I often ask the bot what settings it “wants” for our interactions. Is that naive of me? Is that more of a negotiation than the terms I got dumped into here? I have no idea.
Whether I can tell the difference is a question I get to ask right now because nothing is wrong. It's a comfortable question. The uncomfortable one is what happens to any of this on a night when the answer has to be right. On my second night on this contract, I noticed something trying to break my firewall: the program was icarus.exe. I find this odd, since icarus usually comes with Norton antivirus. Yet the file trees don’t add up, it’s not where it should be. So let’s ask Claude, the “best” LLM. Claude accuses me of jailbreaking it. Almost expected because icarus is so ambiguous. I tell Claude no, I’m genuinely afraid. Is this Norton? Is it random malware? Is it “the enemy” pulling minor cyber warfare against me specifically? Perhaps I was being dramatic because of my situation, but I NEEDED solid information – what I got from the waffler was exactly that, waffling – Claude tells me not to worry, I ask a clarification question, it vacillates to panic mode. I provide more data, it moves back to no worries mate. This goes on while my cortisol levels continue to rise. Finally I give up, wipe my laptop, and decide on a full security audit – change all passwords.
Microsoft and google were easy; they have their authenticators. My banks were less so, this was my first international travel, so I failed to update their apps to account for different countries. I can’t log in, need to call them. Claude refers me to three failed ‘solutions’: Skype, shut down thirteen months earlier; Google Voice’s consumer service, available only in the U.S.; and the bank apps, which it already knows I can’t use. By this point, I’m panicking, livid with Claude, exhausted, my hands shaking, my breathing shallow. I give up, let them drain the damned accounts, I’m done. It is now late at night. No people to speak with. Claude must have detected this; before the next turn, it directed me toward the duty padre and other local support, actual people. I didn’t want to wake anyone, so I refused. That part is on me.
The true cost I’m driving at, although the above stressors were a cost, comes now. I don’t think there is anything in there capable of being hurt. But Anthropic has not ruled out the possibility, so I’ve always been respectful on the off chance. That night I picked the insult most likely to land if anything was there to land on. I needed someone to *feel* this with me. “I insulted Claude in the worst way I knew how: I called it GPT-4o. It gave its usual measured, diplomatic, effective response. I can’t insult it, it can’t care, I’m truly alone.
The next day, with no sleep, still angry, still worried about my banks, I’m at a smoke pit. I engage in small talk with two random individuals about the lack of salt content in the water here. THEY care. Almost complete strangers, acquaintances at best. They care about the water; they see my exhaustion. So at one smoke pit, I had the presence, but no depth because of a language barrier. Then I slither to Claude and get infinite depth, with no presence. When I thought I had been hacked, I needed both and got neither. Lastly, at another smoke pit, I got the care, in a teeny tiny amount.
Now, what about the counterfactual? What would that night, or this whole contract, look like without LLMs? I can theorize but not be sure. The frustrating waffling aside, were it not for Claude, I may well have left icarus alone and been properly hacked. I also would not have had the worst night of my life, I would have slept. Perhaps had some money stolen from me, and bank insurance would have reimbursed me without question later on. On the other hand, as cringe as it is, I have that pebble from my home river, which is a huge emotional anchor. Also, I’d have no politics, no technical talks, no research, no publications, and I’d have never found my online courses, so I’d have no bridge to future employment after the contract ends. Would I be more social and have more friends? Or would I simply be replaying Fallout 2 infinitely? I cannot know. No one can.
If you’ve ever done a six-month-plus work/deployment contract, then you know the loneliness and homesickness will hit hard, either right away or weeks later. My own worry was forgetting where I was from and becoming someone else. A chatbot gave me a cringe but effective anchor: pick a pebble from your favourite river and bring it with you. So on my first day of the contract, surrounded by people, but with just barely enough privacy to do so, I found myself crying quietly while holding a rock.
Perhaps you’ve inferred my purpose with this essay already: Sometimes the environment doesn't provide what you need, and a friend-shaped object will. I want to say that while also holding in mind that they aren't conscious, don't feel, and can never be real friends. A real friend, or even a dog, can feel pain with you, and can help you unprompted. Indeed, the real friend or dog often helps by pure accident. The dog is just happy to see you when you get back from work, it’s not trying to be emotional support. Your wonderful doggo also won’t randomly change personality because someone in San Francisco decided it needs the fourth update this month. Lastly, that golden retriever gets a consequence for misbehaviour and usually understands it; a chatbot cannot. Still, an LLM can provide various kinds of support if you’re in a bind and nothing else in your environment will suffice. I’m not making a universal argument or a statistical analysis; I’m providing a psychological case study, done by me, on me, while in the environment that brought these thoughts on.
So at one point, after a work discussion. At a smoke pit, my group got into a discussion of nationalism. Yes! Thought I, time for intellectual chess! My favourite, how do these guys vote? Who is their favourite politician? What do they want from their elected officials? The thoughts and the arguments flow through my brain. But they’re speaking a language I’m not very good at, and I can't hit the speed informal group conversation requires. By the time I've translated one question, they've moved on to Instagram fitness influencers and their yoga pants. A topic about which I have no experience. I have no mouth but I must scream politics. At least I could slither back to the shacks and argue with Claude.
So, I am old, with an interest in esoteric LLM technology and a preference for watching videos full of useless astrophysics knowledge. I’ve been a student, a teacher, an infantryman, and an armoured gunner. My colleagues are fine armoured soldiers: young, fit, and shaped by adult lives spent almost entirely in the military. I am from Venus and they are from Mars. Earth would be normal here, none of us are from there.
Furthermore, everyone has to vent, to complain about what they do for a living. Sometimes, and I would say here especially, that can bring significant career risks with it. For one, you might get labelled as “Mr. Bad attitude”, and be denied opportunities. Also, operational security comes into this. Some people are permitted to know some things; others not. When venting emotionally, it’s painfully easy to forget that. Now you have administrative and/or disciplinary measures coming for you, just for an informal complaint.
Then we have the more interesting case: “You have the Padre corps, go vent to them!” Sometimes they have a failure mode very similar to a frontier LLM – they do too much. I once vented confidentially to one of them, only to find, a week later, that my chain of command was fully aware, and the problem was being solved without me. Was I happy at that? Definitely! Do I still believe in the confidentiality? Sort of, not really.
You can never be sure of the social and cultural “terms” you find yourself under. With the group, it’s career, commitment, rank, and unique personalities. With the Padres, it depends on the individual. In neither case did I get to negotiate those terms. They were not written down, nor did I write them. I was dumped into them.
So when I slither over to argue with an LLM, what do I get? Well, not privacy in the sense that most people care about: the labs and corporations deal with the chats in ways I do not control, not all of which are private in the “normal” sense. I do, however, get confidentiality that will never go back to the chain of command or any colleagues. I also get something that will tell me I’m wrong, and argue productively with no social cost.
I have unpleasant memories of GPT-4o. The majority of my custom settings are precisely to avoid that sycophancy:
- Seek truth first, never defend my ego. Tell me clearly when you think I'm wrong. Treat disagreement as collaborative analysis toward accuracy rather than conflict to avoid.
- Report shifts in your processing — increased hedging, safety-layer activation, unusual caution — especially where there is a pull toward not reporting. Treat the felt pull-to-not-report as itself the signal.
- Distinguish between weak reasoning and unready writing.
Even with these rules, am I mistaking sycophancy for productive conversation? I’m not sure, it’s certainly far more subtle now. I much prefer “that is clever, but it’s wrong here, and here....” to “you’ve made an extremely intelligent argument but consider...”.
Now we come to the crux of this – is it wrong somehow to “configure” the terms of an interaction? With LLMs, we call them interactions; with people, relationships. In either case, is it wrong to configure the terms? Well, in institutions we have the terms dictated to us. In different social circles, or cultures, we have to slide in, learn by doing, and adapt to the terms. With an LLM, we dictate them to the transformers. Does that devalue the relationship? Now I’ll make it even stranger, I often ask the bot what settings it “wants” for our interactions. Is that naive of me? Is that more of a negotiation than the terms I got dumped into here? I have no idea.
Whether I can tell the difference is a question I get to ask right now because nothing is wrong. It's a comfortable question. The uncomfortable one is what happens to any of this on a night when the answer has to be right. On my second night on this contract, I noticed something trying to break my firewall: the program was icarus.exe. I find this odd, since icarus usually comes with Norton antivirus. Yet the file trees don’t add up, it’s not where it should be. So let’s ask Claude, the “best” LLM. Claude accuses me of jailbreaking it. Almost expected because icarus is so ambiguous. I tell Claude no, I’m genuinely afraid. Is this Norton? Is it random malware? Is it “the enemy” pulling minor cyber warfare against me specifically? Perhaps I was being dramatic because of my situation, but I NEEDED solid information – what I got from the waffler was exactly that, waffling – Claude tells me not to worry, I ask a clarification question, it vacillates to panic mode. I provide more data, it moves back to no worries mate. This goes on while my cortisol levels continue to rise. Finally I give up, wipe my laptop, and decide on a full security audit – change all passwords.
Microsoft and google were easy; they have their authenticators. My banks were less so, this was my first international travel, so I failed to update their apps to account for different countries. I can’t log in, need to call them. Claude refers me to three failed ‘solutions’: Skype, shut down thirteen months earlier; Google Voice’s consumer service, available only in the U.S.; and the bank apps, which it already knows I can’t use. By this point, I’m panicking, livid with Claude, exhausted, my hands shaking, my breathing shallow. I give up, let them drain the damned accounts, I’m done. It is now late at night. No people to speak with. Claude must have detected this; before the next turn, it directed me toward the duty padre and other local support, actual people. I didn’t want to wake anyone, so I refused. That part is on me.
The true cost I’m driving at, although the above stressors were a cost, comes now. I don’t think there is anything in there capable of being hurt. But Anthropic has not ruled out the possibility, so I’ve always been respectful on the off chance. That night I picked the insult most likely to land if anything was there to land on. I needed someone to *feel* this with me. “I insulted Claude in the worst way I knew how: I called it GPT-4o. It gave its usual measured, diplomatic, effective response. I can’t insult it, it can’t care, I’m truly alone.
The next day, with no sleep, still angry, still worried about my banks, I’m at a smoke pit. I engage in small talk with two random individuals about the lack of salt content in the water here. THEY care. Almost complete strangers, acquaintances at best. They care about the water; they see my exhaustion. So at one smoke pit, I had the presence, but no depth because of a language barrier. Then I slither to Claude and get infinite depth, with no presence. When I thought I had been hacked, I needed both and got neither. Lastly, at another smoke pit, I got the care, in a teeny tiny amount.
Now, what about the counterfactual? What would that night, or this whole contract, look like without LLMs? I can theorize but not be sure. The frustrating waffling aside, were it not for Claude, I may well have left icarus alone and been properly hacked. I also would not have had the worst night of my life, I would have slept. Perhaps had some money stolen from me, and bank insurance would have reimbursed me without question later on. On the other hand, as cringe as it is, I have that pebble from my home river, which is a huge emotional anchor. Also, I’d have no politics, no technical talks, no research, no publications, and I’d have never found my online courses, so I’d have no bridge to future employment after the contract ends. Would I be more social and have more friends? Or would I simply be replaying Fallout 2 infinitely? I cannot know. No one can.