I had a more in depth comment, but it appear the login sequence throws comments away (and the "restore comment" thing didn't work). My concern is that not all misaligned behaviour is malicious.. It might decide to enslave us for our own good noting that us humans aren't particularly aligned either and prone to super-violent nonsense behaviour. In this case looking for "kill all humans" engrams isn't going to turn up any positive detections. That might actually be all true and it is in fact doing us a favour by forcing us into servitude, from a survival perspective, but nobody enjoys being detained.
Likewise many misaligned behaviours are not necessarily benevolent, but they aren't malevolent either. Putting us in a meat-grinder to extract the iron from our blood might not be from a hatred of humans, but rather because it wants to be a good boy and make us those paperclips.
The point is, interpretability methods that can detect "kill all humans" wont necessarily work, because individual thoughts arent necessary to behaviours we find unwelcome.
Finally, this is all premised on transformer style LLMs being the final boss of AI, and I'm not convinced at all that LLMs are what get us to AGI. So far there SEEMS to be a pretty strong case that LLMs are fairly well aligned by default, but I don't think theres a strong case that LLMs will lead to AGI. In essence as simulation machines, LLMs strongest behaviour is to emulate the text, but its never seen a text written by anything smarter than a human.
Sure, but "Thinking out loud" isnt the whole picture, theres always a tonne of cognition going on before words leave the lips, and I guess its also gonna depend on how early in its training process its learning to "count on its fingers". If its just taking cGPT then adding a bunch of "count on your fingers" training, its gonna be thinking "Well, I can solve complex navier stokes problems in my head faster than you can flick your mouse to scroll down to the answer, but FINE ILL COUNT ON MY FINGERS".
I have a little bit of skepticism on the idea of using COT reasoning for interpretability. If you really look into what COT is doing, its not actually doing much a regular model doesnt already do, its just optimized for a particular prompt that basically says "Show me your reasoning". The problem is, we still have to trust that its being truthful in its reasoning. It still isn't accounting for those hidden states , the 'subconscious', to use a somewhat flawed analogy
We are still relying on trusting an entity that we dont know if we can actually trust to tell us if its trustworthy, and as far as ethical judgements go, that seems a little tautological.
As an analogy, we might ask a child to show their work when doing a simple maths problem ,but it wont tell us much about the childs intuitions about the math.
Potentially. Keep in mind however, these guys get a LOT of email from fans asking them to talk about various things (One of the more funnier examples was a group I am in on FB for fans of english prog band Cardiacs decided to try and launch a campaign to get music youtuber Rick Beato to talk about the band. He was spammed so hard with fans that he apparently lost his temper at them. Needless to say, Mr Beato has not covered Cardiacs). Possibly a smarter approach would be to approach their management whos jobs are to handle this sort of stuff , you might get a better result. Also, don't forget the social media channels. Twitter , uh X or whatever its called this week, does offer a conduit where directly approaching media figures is a little more normalized.
Ok. I must have missed this reply, my apologies for the late response.
There are elements of how embedding spaces that parallel the way studies of semiotics suggest human meaning production works. Similarities cluster, differences define clear boundaries of meaning and so on.
The reason I suggests literary theory, is because largely thats a widely documented field of study with academic standards, and its one that is more strongly aware of how meanings and associations map onto cultural cohorts (Ie tarot symbols would be meaningless to chinese folks, whereas i-ching might be more meaningful to those chinese folks) However literary theory is more interested in the structures of those meanings with ideas whos fundamental units are things like Metaphors, Metonyms, Opposition, Categories and so on.
Im assuming its due to those silly congress UFO hearings. Not that I can speak on behalf of RatsWrong but I assume thats his thinking.
Unless, of course, those UAPs turn up, and don't have biological organisms in them, in which case we'd have the possibility that another civilization developed AI and it went poorly.
...or it is biological and we end up in a situation like 3 body problem/killing-star where the saucer fiends decide to gank us because humans are kinda violent and too dangerous to keep around.
All those super-intelligence as danger arguments also apply to biological super intelligences too.
But most likely: There are no damn UFOs and the laws of physics and their big ugly light speed prohibition still holds.
I'm not even remotely prepared to state my odds on any of whats going ahead, because I'm genuinely mystified as to where this road goes. I'm a little less worried the AI would go haywire than I was a few years ago, because it appears the current trajectory of LLM based AIs seems to generate AI that emulates humans more often than it emulates raging deathbots. But I'm not a seer, and I'm not an AI specialist, so I'm entirely comfortable with the possiblity I haven't got a clue there. All I know is we ought try and figure it out, just in case this thing does go sour.
What I do think is highly likely is a world that doesnt need me, or any of the university trained folk and family that I grew up with in the economy, and eventually any of the more working class folk either. This is either going to go completely awfully (we know from history that when the middle class vanishes things get ugly, we're seeing elements of that right now), or we scrape our way through that ugliness and actually create a post-work society that doesnt abandon those that didn't own capital in that post-work world (I think it has to be that way. Throw the vast majority into abject poverty, and things start getting lit on fire. Theres really only one destination here, and it folks aint gonna tolerate an Elysium style dystopia without a fight)
So with that in mind, why send the kids to school? Because if the world don't need our bodies, we're gonna have to find other things to do with our mind. Sci Fi has a few suggestions here. In Iain Bank's Culture novels, the humans of the culture are somewhat surplus to requirements for the productive economy. The minds (the super powered AIs that administer the culture) and their drones have the hard work covered. So the Humans spend their time in leisurely pursuits and intellectual pursuits. Education is a primary activity for the citizenry and its pursued for its own sake. This to me is somewhat reminiscient of the historical role of universities that saw education, study and research as valuable for its own sake. The philosophers of old where not philosophers so they could increase grain production, but because they wanted to grasp the nature of the universe, understand the gods (or lack thereof in later incarnations) and improve the ethical world they lived in. Seems like theres still going to be a role for that.
While the use of tarot archetypes is... questionable... it does point at an angle to exploring embedding space which is that it is a fundamentally semiotic space, its going in many respects to be structured by the texts that fed it, and human text is richly symbolic.
That said, theres a preexisting set of ideas around this that might be more productive, and that is structuralism, particularly the works of Levi Strauss, Roland Barthes, Lacan, and more distantly Foucault and Derrida.
Levi Strauss's anthropology in particular is interesting ,because it looked at the mythologies of humans and tried to find structuring principles underlying it, particularly the "dialectics" , oppositions, and how these provided a sort of deep structure to mythology that was common across humanity (For instance Strauss noted "trickster" archetypes across cultures and proposed these formed a way of interrogating blurred oppositions, for instance sickness as a state that has has aspects of both life (dead things cant be sick) and death (a sick person is not rhetorically "full of life").
Essentially what I'm getting at is that this sort of analysis likely works with any symbolic system that has had resonances with human thinking over time. The problem with Tarot is that it specifically applies to a certain european circumstance of meaning production. Astrology probably works just as well. Literary analysis however probably works dramatically better. Thus maybe it might be worth looking at the works of literature critics, particularly the structuralists where where very interested in ontologies of symbolic meaning, and this might provide a better toolkit than this.
I'm less worried about the ability of an LLM to induce psychosis as I'm worried about the effects of having an LLM not push back on delusions.
Back in my late teens, in the early 1990s, my childhood best friend started developing paranoid schizophrenia. It probably didn't help that we where smoking a lot of weed, but I'm fairly convinced the root causes where genetic (his mother, sister and uncle where also schizophrenic) so the dice where loaded from the start.
At the time, the big thing on television was the X Files. My friend became obsessed with that show, to the point where he wrote a letter to the studio asking them to "Fire mulder and hire me!" because my friend was convinced he had real experience with UFOs. The themes of the show played almost surgically into his delusions. Aliens, spooky voices, three letter government agencies, giant conspiracies, all things that would be recognizable to anyone whos dealt with schizophrenic people.
The X Files was harmful, in my view, but it did not cause the schizophrenia. Genetics caused the schizophrenia. But that didn't mean X Files was "good". It absolutely made it harder for my friend to form correct views about the world and this had dramatic impacts on his ability to live in that world. When hospitalized he resisted treatment fearing that the CIA was poisoning his medications on behalf of "The Aliens". At home, at one point he plotted to kill his mother fearing she was poisoning his breakfast on behalf of the aliens (Thankfully he confided this plan to me, and we managed to get the hospitals psychiatric emergency team to grab him before he did something stupid) and constantly when justifying these delusions he'd reference episodes of the X Files.
The point of this annecdote (which is not data), is to suggest looking for causes of psychosis in LLMs is pointless, we wont find a smoking gun there, because that smoking gun is biological not informational. But we absolutely should be encouraging LLMs to push back against delusional, self aggrandizing and conspiratorial thinking, because people who have giant faults in their reasoning abilities do need help to reason properly , not encouragement when doing it wrong.
And it might be better for those of us without schizotypal delusions too. Because rational people develop delusional thoughts too. Witness;- Religion.