How to notice being mind-hacked

byshminux21d2nd Feb 201922 comments


Epistemic status: quite sure, but likely nothing new, I have not done the requisite literature search.

Human mind is not designed with security in mind. It has some defenses against basic adversaries that would have prevented our survival as a species, but not much more than that. It is also necessarily open to external influences because humans are social animals and cooperation is essential for survival. So, any security expert would be horrified at how vulnerable to adversarial mind hacking humans are. Humans generally do not like to accept how easy we are to sway, and how often it happens to us, but we can definitely see other people being easily influenced, and most of us aren't special in terms of mind security.

Another common term for it is "manipulation," but there is a slight difference. Manipulation generally presumes that the interests of the manipulator are detrimental to the mind being manipulated. Mind hacking does not have to have this negative connotation.

So, given that our minds are security sieves and we live in the world where influencing others (yet another term for mind hacking), and where we have certainly been mind-hacked by others over and over again, how does one notice a hack (unauthorized breach of mind security), whether when it is about to happen, when in progress, and after the fact? I am limiting the scope to just noticing. I am not implying that one has to try to stop a mind hack in preparation or in progress, or trying to undo it after it happened. Descriptive, not prescriptive.

Let's start with a a few obvious examples.

Your friend, noticing your distress, invites you to their church event, just to get your mind off things. A month later, you have converted to their faith, quote scriptures, believe in salvation and dedicate your life to spreading the gospel.

Or you come across a book, say, HPMoR or From AI to Zombies (I take partial credit/blame for the latter name), learn about rationality, get blown away by Eliezer's genius, and, next thing you know, you are at a local x-risk meetup worrying about an unaligned AI accidentally paper-clipping the universe and donating 10% of your income to an EA cause.

Or you pick up a Siouxsie and the Banshees CD in a record store (back when CDs and record stores were a thing), and soon you are a part of the goth subculture, deathhawk up every weekend, your carefully crafted Rihanna mixtape (another anachronism) gathering dust in the back of the bottom drawer.

Or maybe you end up at a kink munch, seemingly out of idle curiosity, then at a play party, then you discover your submissive side, end up dumping your vanilla partner and go on a sub frenzy and eventually settle as a slave to a Master/Mistress.

Not all mind hacks are as striking. But these somewhat extreme, yet also mainstream examples is a good place to start the analysis. Some salient features:

  • A glaring chasm between your identity before and after the event.
  • Acceptance of your current identity and thinking of yourself before the event as immature/naive/stupid/unenlightened.
  • Realization that the you before the event would likely be similarly disapproving of the change that transpired and would have prevented it if they could anticipate it.
  • [What else?]

The above suggests how to notice the event post hoc (post hack?). The identity disconnect and the feelings around it are a telltale sign.

Noticing a hacking attempt or a hack in progress is probably harder. When skillfully executed, it never rises to the conscious level. You don't necessarily consciously notice your identity changing. Instead, you may be swept in the feelings of insight, being wowed, enlightened, or the opposite, intense guilt, shame and remorse, and often some combination of both. And even if we do recognize it for what it is, these same intense feelings can be too addictive to break the spell, and we can crave them more and more, and rationalize away what is happening. So, to provisionally answer the title non-question, watch out for the mind-hack-associated feelings.

What have been your experiences with noticing being mind hacked, intentionally or accidentally, or with doing it to others, whether on purpose or not?