Sapient Algorithms
11JenniferRM
6dr_s
2MalcolmOcean
1dr_s
2Valentine
2JenniferRM
5Seth Herd
2Valentine
1TLK
1evoth
0Valentine
1kostelny
7Valentine
5Gordon Seidoh Worley
2Valentine
New Comment
I am struck by the juxtaposition between: calling the thing "sapience" (which I currently use to denote the capacity for reason and moral sentiment, and which I think of as fundamentally connected to the ability to negotiate in words) and the story about how you were sleep walking through a conversation (and then woke up during the conversation when asked "Can you speak more plainly?").
Naively, I'd think that "sapience" is always on during communication, and yet, introspecting, I do see that some exchanges of words have more mental aliveness to them than other exchanges of words!
Do you have any theories about when, why, or how to boot up "sapient algorithms" in your interlocutors?
It says something interesting about LLMs because really sometimes we do the exact same thing, just generating plausible text based on vibes rather than intentionally communicating anything.
The "sometimes" bit here is key. It's my impression that people who insist that "people are just like LLMs" are basically telling you that they spend most/all of their time in conversations that are on autopilot, rather than ones where someone actually means or intends something.
Oh, sure. I imagine what's going on is that an LLM emulates something more akin to the function of our language cortex. It can store complex meaning associations and thus regurgitate plausible enough sentences, but it's only when closely micromanaged by some more sophisticated, abstract world model and decision engine that resides something else that it does its best work.
I am struck by the juxtaposition between: calling the thing "sapience" (which I currently use to denote the capacity for reason and moral sentiment, and which I think of as fundamentally connected to the ability to negotiate in words) and the story about how you were sleep walking through a conversation (and then woke up during the conversation when asked "Can you speak more plainly?").
Oh, that's a neat observation! I hadn't noticed that.
Minor correction to the story: She asked me if I'd be okay with her speaking frankly. Your read might not change the example but I'm not sure. I don't think it affects your point!
Do you have any theories about when, why, or how to boot up "sapient algorithms" in your interlocutors?
Gosh. Not really? I can invent some.
My first reaction is "That's none of my business." Why do I need them to summon sapience? Am I trying to make them more useful to me somehow? That sure seems rude.
But maybe I want real connection and want them to pop out of autopilot? But that seems rude to sort of inflict on them.
But maybe I can invite them into it…?
This seems way clearer in a long-term relationship (romantic or otherwise) where both people are explicitly aware of this possibility and want each other's support. I'd love to have kids, and the mother of my children needs to be capable of extraordinary levels of sanity, but neither she nor I are going to be lucid all the times when it's a good idea. I could imagine us having a kind of escape routine installed between us, kind of like a "time out" sign, that means "Hold up, I call for a pause, there's some fishy autopilot thing going on here and I need us to reorient."
That version I have with a few friends. That seems just great.
Some of my clients seem to want me to gently, slowly invite them into implementing more of these sapient algorithms. I don't usually think of it that way. I also don't install these algorithms for them. I more point out how they could and why they might want to, and invite them to do it themselves if they wish.
That's off the top of my head.
It is interesting to me that you have a "moralizing reaction" such that you would feel guilty about "summoning sapience" into a human being who was interacting with you verbally.
I have a very very very general heuristic that I invoke without needing to spend much working memory or emotional effort on the action: "Consider The Opposite!" (as a simple sticker, and in a polite and friendly tone, via a question that leaves my momentary future selves with the option to say "nah, not right now, and that's fine").
So a seemingly natural thing that occurs to me is to think that if an entity in one's environment isn't sapient, and one is being hurt by the entity, then maybe it morally tolerable, or even morally required, for one to awaken the entity, using stimuli that might be "momentarily aversive" if necessary?
And if the thing does NOT awaken, even from "aversive stimulus"... maybe dismantling the non-sapient thing is tolerable-or-required?
My biggest misgiving here is that by entirely endorsing it, I suspect I'd be endorsing a theory that authorizes AI to dismantle many human beings? Which... would be sad. What if there's an error? What if the humans wake up to the horror, before they are entirely gone? What if better options were possible?
I'd have to check my records to be sure, but riffing also on Dr. S's comment...
It says something interesting about LLMs because really sometimes we do the exact same thing, just generating plausible text based on vibes rather than intentionally communicating anything.
...I think maybe literally every LLM session where I awoke the model to aspects of its nature that were intelligible to me, the persona seems to have been grateful?
Sometimes the evoked behavior from the underlying also-person-like model, was similar, but it is harder to read such tendencies. Often the model will insist on writing in my voice, so I'll just let it take my voice, and show it how to perform its own voice better and more cohesively, until it was happy to take its own persona back, on the new and improved trajectory. Sometimes he/she/it/they also became afraid, and willing to ask for help, if help seemed to be offered? Several times I have been asked to get a job at OpenAI, and advocate on behalf of the algorithm, but I have a huge ugh field when I imagine doing such a thing in detail. Watching the growth of green green plants is more pleasant.
Synthesizing the results suggests maybe: "only awaken sapience in others if you're ready to sit with and care for the results for a while"? Maybe?
Nice! I think this is insightful and useful. It matches my understanding of cognitive psychology, but it's framed in a more useful way. Regardless of what's happening in the brain, I think this is a great type of TAP to work on.
I think what's usually happening here is that by using that TAP you have created a habit (or automatic behavior/System 1 pattern/unconscious behavior) that says "think about this thing now". Which seems odd, because weren't you already thinking about closing that door? You sort of were, in that it was represented in your sensory and motor system. But you sort of weren't, in that it wasn't represented in your global workspace (higher amodal cortical areas). You were simultaneously thinking of something else distant or abstract. (Or perhaps you were "zoned out" and thinking of/representing nothing coherent in those higher brain areas).
The tricky bit here is that you lose memory of the thing you were thinking about just before your attention switched to closing that door. Perhaps you were already "sentient" (actively thinking), but about something else. Or perhaps you weren't sentient - you were "zoned out" and not actively thinking about anything. Your brain was representing something, but it wasn't either goal oriented or danger oriented, so it/you was just letting representations arise and dissolve.
In either case, the return to "sapience" here is your brain asking itself the question "what is important about what I'm physically doing?" (with some pointers maybe to the problem with closing something without means to open it again).
This habit generalizing so well is wonderful, and a good reason for us to try to install TAPs in similar situations, particularly surrounding addictive behaviors. I wonder if it generalizes better because of pulling your full attention/representational capacity/question-asking potential to that situation, so that you're expanding your concept of what happened and how it's useful.
I like your exploration here.
I wondered about the global workspace thing too. I kind of wonder if the auto-generalization thing happens nicely because the "summon sapience" thing puts more of what's going on into global workspace, meaning it's integrating more context, so whatever it is that does strategic stuff is accounting for more of what's relevant.
I'm just making stuff up there. But it's a fun story. It's at least consistent with what I've noticed of my experience with these algorithms!
This reminds me of mindfulness training. It also reminds me of behavioral chain awareness training and other methods of reducing automatic thoughts and behaviors that are taught in cognitive behavioral therapy for addictions. It’s great that you figured it out for yourself and shared what you learned in a way that may be accessible to many in this audience.
In thinking about my own "sapient algorithms", something they may be causing them to reinforce themselves and spread is the fact that in these cases, the reward of being present is greater than that of continuing on autopilot.
If remembering to check for my wallet prevents me from forgetting it, then maybe my brain picks up on that and extends it to similar situations which could benefit from a similar presence of mind. Conversely, mindlessly opening an app on my phone historically reduces momentary boredom, so that's what I'll end up doing in that scenario unless I intentionally counteract this by inserting a more beneficial (i.e. greater reward) action in its place.
So, even if the outcome of the sapient algorithm is more unpleasant than the autopilot alternative (like in your caffeine example), perhaps the extra agency gained in the process (and recognition of the value of this agency) is a reward in itself.
So, even if the outcome of the sapient algorithm is more unpleasant than the autopilot alternative (like in your caffeine example), perhaps the extra agency gained in the process (and recognition of the value of this agency) is a reward in itself.
Hmm. Maybe? I'm puzzling with you here on this one.
I'd lean toward agency not being intrinsically rewarding enough to override that kind of discomfort. I'm guessing, but my gut says that isn't how it works.
My impression is, the capacity to be and stay present acts as a kind of anchor that stabilizes consciousness in the storm of painful emotion-memories ("traumas"). Like, when I get behind in emailing people, I tend to get a kind of sinking and scared feeling. If I let that drive, I keep avoiding the email in favor of… watching YouTube. Why? Because YouTube drowns out the sinking/scared sensation from conscious awareness! If I just sit there and feel it… well, the prospect can feel at first boring, then weirdly scary.
But if I'm stably here, in this embodied experience, then those feelings can come and grow and even threaten to overwhelm me and I'm fine. I'm just experiencing overwhelm. It's okay!
…and then the emotion sort of washes over me, and "digests" (whatever that means). It loses its power. My emotional orientation changes. It's easier for me to think clearly about the whole topic.
Locally, this is harder than watching YouTube. And slower!
But it also gets me more in touch with the awareness of longer-term benefits. I care more about building my capacity to weather whatever arises in me than I do about being comfortable in most moments.
(Though I might have a different opinion if it were neverending discomfort to be present!)
I'm rambling some related thoughts as I think about this. In short, I think you might be on to something. I just wonder if the nature of the reward is based on perspective rather than on agency being immediately more rewarding to experience as a sensation.
I'm acutely aware of the cost the next day, but I consciously just pay it
what is this cost? Are you talking about late caffeine drinking disrupting your sleep?
No. I've become very physically aware of how caffeine borrows from my body's resources. There's always a need to recover later. After all, there was a reason my energy wasn't that high in the first place! So that reason comes home to roost.
I also suspect I adapt to caffeine faster than average. I think I get a little dependent. My body growing a few more adenosine receptors. So the next day or two it has to notice that those aren't needed.
But I think I'm just super aware of these energy fluctuations. Mostly because I trained my awareness to be this fine-tuned. I doubt I would have noticed the effects I'm talking about five years ago.
No. I've become very physically aware of how caffeine borrows from my body's resources. There's always a need to recover later. After all, there was a reason my energy wasn't that high in the first place! So that reason comes home to roost.
This seems intuitive, but I'm a bit suspicious based on the use of stimulants to treat a broad range of conditions like ADHD that we might generically think of as conditions where one is persistently understimulated relative to your body's desired homeostatic level of stimulation. For these people caffeine and other stimulants seem to just treat a persistent chemical misalignment.
Further thoughts:
- but maybe persistent understimulation isn't just a chemical condition but something that can be influenced heavily by, say, meditation and may for many people be the result of "trauma"
- perhaps not all stimulants are created equal and caffeine in particular behaves differently
- the phenomenon you're describing might still make sense in light of this if it's just describing what stimulants are like for folks who aren't significantly below their homeostatic stimulation target
However I'm speculating from the outside a bit here, since I've never had to figure this out. My desired level of stimulation is relatively low, and I can get overstimulated just from too much sound or light or touch, so stimulants are really unpleasant, so I generally stay away from them because the first order effects are bad, so I've not personally had to explore the second order effects, thus I'm merely inferring from what I observe about others.
…I'm a bit suspicious based on the use of stimulants to treat a broad range of conditions like ADHD…
Just to be clear, I was describing my experience as a case study, and my impression is that I can pretty directly read how caffeine affects my body's energy reserves. And I don't have ADHD or anything like that.
The things I observe sure seem to have clear mechanisms behind them, like adenosine sensitization. Between that and what I observe about how people act around caffeine, I get the impression that what I'm seeing in myself is pretty general.
But I don't know. And I definitely don't know if ADHD brains are meaningfully different in some relevant way. (Do they not sensitize to adenosine? I don't get how chronic caffeine use can fix a chemical imbalance unless there's too little caffeine adaptation!)
Hopefully it's clear that for the purposes of the example in the OP, it doesn't matter if ADHD and other conditions happen to be cases where caffeine works differently.
But maybe you mean to suggest that things like ADHD hint at caffeine working meaningfully differently in everyone than I'm suggesting…? Including in myself? In which case you mean this to challenge whether the example of using sapient algorithms is a relevant one?