This post is backported from my old personal blog, originally published 11 October, 2014. It's similar in theme but very different in focus than my recent post, On the Nature of Agency, making for a good complement.

Epistemic Status: Though I wrote this over four years ago and now use different models/lenses/ontologies, what I wrote then still seems largely correct. The exception is the endorsement of noticing which I haven't thought about much in recent years and therefore can't endorse, though I do now want to think about it more.

.

Agency
Agency is the property of agents. An agent has explicit goals which they strive to accomplish by planning and executing appropriate actions. Non-agents unreflectively act out default behaviours without considering whether these actions achieve their goals.

Sphexishness
Coined by Douglas Hofstader in reference to the sphex wasp, sphexishness is the execution of seemingly intelligent behaviour by following a rigid algorithm. Sphexish behaviours are repeated automatically, out of habit, without checking for their effectiveness at achieving desired goals.

When I first learnt the concepts of agency and sphexishness, I didn’t give them much thought. I unreflectively assumed that you were either an agent or you weren’t; you were either sphexish or you weren’t. If you were the kind of person who thought about their goals, read Less Wrong, went to CFAR workshops, then clearly you were an agent – you actually cared about your goals. I, of course, was such as person. It’s the general population that is sphexish, they don’t notice when things don’t get them what they wanted. And it all maps well on the controversial metaphor of PCs and NPCs.

But you don’t just get agent status and that’s it. Rather, as I have learnt, there is a constant slide into sphexishness which you must work damn hard to avoid.

Looking at definition given, sphexishness has two components:

Unreflective, automatic, default behaviours.
These behaviours fail to achieve goals.

Really, it’s 2) which matters. Firstly, not only can you have unreflective, automatic behaviours which are not sphexish, you absolutely need unreflective, automatic behaviours. Attention and cognitive resources are limited and you cannot be constantly using System 2 to examine how you do things to achieve your goals. Instead you must have put in place default behaviours which do the job. It works for walking, for what I eat for breakfast, and for how I go about writing code.

Secondly, there is no level of deliberateness or complexity or meta which guarantees that you are not being sphexish. Suppose I do goal factoring weekly, or I set lots of five minute timers, etc., but using these techniques isn’t improving my life that much – then my use of these techniques is sphexish. Even if I periodically review my rationality practices for effectiveness, I could be sphexish is my review routine doesn’t get results.

So wherein lies agency? How does one avoid sphexishness? Noticing. My current hypothesis is that Noticing is a fundamental skill of rationality. One must learn to notice when one’s actions are failing to get the desired result, and then take corrective action. And “failure” is loosely defined here: one could be making progress towards a goal, but at a much slower rate than one could, and this too I would deem sphexish to some extent. And this Noticing goes infinitely meta – you need to be able to notice when your first-order optimisations aren’t working or are too slow, take corrective action on them, and so on.

To avoid sphexishness one needs to notice that the ways one has always done things unthinkingly aren’t great; one needs to notice that unreflective, automatic behaviours which were optimal when one originally implanted them are no longer so, now that circumstances have changed; one needs to notice when one is stuck in a rut and needs to pull themselves out; one needs to notice that they just aren’t making progress and need to do something different.

On this I will say more.