Ordinarily, when someone is trying to solve a problem, experimentation is where they begin. “Things aren’t going right? Try changing something, and see what happens!” In naturalism, though, experimentation comes at the very end, if it happens at all.

There’s a balance to be kept in naturalist experimentation, and it can be a tricky one. I said in Getting Started that the naturalist stance deemphasizes most ways relating to the world, and regards the world instead as something that may reveal itself to you as you open up to it. The trick in this phase is to balance between open curiosity and agency

By the time you’ve made it to the “Experimentation” phase in a naturalist study, you’ve spent a long time being very hands-off: Only look, do not touch; just observe, do not intervene. In the very beginning of your study, you asked your desperation to wait in the hall, then invited curiosity to take its place, in an attempt to create plenty of space for patient and direct observation. This whole time, you have tried to observe your fulcrum experience in its natural habitat, learning what happens around it by default, when you do nothing to interfere.

In this final phase, you’ll begin to interfere. You’ll deliberately interact with the systems you’ve been observing. You’ll exercise your agency.

It can be quite difficult, while getting much more hands-on with the circumstances you observe, to remain in a state of open curiosity all the while. It can be tempting to stop exploring patiently, and start trying to immediately force the outcomes you want. 

I’m not here to tell you that it is wrong to force the outcomes you want, to grab onto the levers of causality that you’ve discovered and use them to guide the world into your preferred configuration. If you’ve made it this far, you’re probably in a much better position to do that wisely than you were when you started.

In this essay, I only mean to show you that it is possible to choose another way to interact with those levers, if you want to. It is possible to balance agency with patient, direct observation, in a way that causes your understanding of your topic to deepen indefinitely, rather than bringing your study to an abrupt close. Instead of switching to goal-oriented engineer mode to obtain the object-level solution that you wanted at the start, you might prefer to take a much longer path toward deep mastery of the field you’ve really only just begun to explore.

I call that longer path “naturalist experimentation”.

 

 

What is different about naturalist experimentation?

There are a couple of things that set naturalist experimentation apart from a more casual approach to exploratory experimentation, and also from scientific experimentation.

The first is why we experiment. The second is when.

 

Why We Experiment

When I was a kid and our television picked up stations using antennae (which we called "bunny ears"), the picture would sometimes cut out, and we’d try all sorts of things to make it come back. We’d whack the side of the TV, move the bunny ears around, stand in a different part of the room, change the channel. It was all sort of hapless and frantic; we were just changing things largely at random in the hopes that something would solve the problem, whatever it was, and the picture would come back. This is an extreme example, but here it is clear that we experimented to fix things.

In scientific experimentation, we experiment to improve our models, usually by attempting to disconfirm hypotheses generated by them. This is certainly related to naturalist experimentation, and I’ll come back to the relationship in a moment.

In naturalism, we experiment to create space for alternative action

To get what I mean by this, it may help to think in terms of stimulus-response patterns. Yesterday, I was journaling; imagine me at my desk, notebook in front of me, pen in hand, my cat Adagio sitting just to the right of my notebook. Then, something changed: Adagio batted my pen out of my hand, and the pen fell to the table.

Adagio’s batting paw was a stimulus. My default response was to interpret this as an interruption of my intended task. That interpretation was also a stimulus, and my default response to this interpretation was to pick up my pen and continue writing.

I think it is reasonable to consider all of human behavior, whether physical or cognitive, as patterns of stimulus and response (or, in CFAR parlance, as trigger and action). The patterns are often linked like proteins into long and complex chains. They determine not just the movements of our hands, but also where we move our attention, and how we interpret the information we encounter as our attention shifts around.

The first three phases of naturalism are an attempt to intervene in stimulus-response patterns at the level of perception. In Locating Fulcrum Experiences, you pin down which of your stimulus-response patterns interacts with whatever you hope to learn about. During Getting Your Eyes On, you begin to replace one of your default responses with “observation”. During Collection, you train the stimulus→observation pattern until you’re able to observe every time the stimulus happens. By this point, rather than automatically seeing things in the same old familiar ways, you’re reliably looking at something that’s more like what’s actually there, rather than at what you originally expected to see.

If you’re constantly observing in response to a stimulus, rather than immediately taking whatever action you ordinarily would by default, then you have already taken the most crucial step toward breaking a default stimulus-response pattern. You have already created a space between the stimulus and your original default response.

In the Experimentation phase of naturalist study, you’ll use actions that are larger than “observation” to stretch that space. You’ll experiment with saying this, thinking that, or moving your body in such and such a way, until the link between the stimulus and your default response has been severed entirely.

By creating space for alternative action, I mean breaking an existing pattern of stimulus-response, and replacing the default action with agency.


One reason it’s valuable to begin with perception is that your set of available actions is largely determined by your perceptions. If you perceive a teacup as empty when in fact it’s full of tea, it will not occur to you that you could drink the tea. You have to see that the tea is there, first.

But even after you’ve done a lot to correct your perceptions, you may still be locked into habitual ways of thinking and acting in response to those perceptions. Just because you can see the tea does not mean it will in fact occur to you to drink it. You may not recognize the choices available to you until you try to make new ones. For example, even if I can see in perfect detail all the thoughts and impressions comprising conformity bias, I may have no idea what else I could do when those things happen, besides conform.

As we learn, our thoughts and actions become haunted by the ghosts of departed perceptions. Experimentation in naturalism involves exploring the possibility of responding to your perceptions in unfamiliar ways, so that you can move freely in the world as you see it now, instead of restricting your motions to outdated habit.

 

 

When We Experiment

So the first big difference between naturalist experimentation and other approaches is why we experiment. The second big difference is when we experiment.

By the time you begin to experiment in a naturalist study, you’ve progressed all the way through the noticing timeline. You’re able to become reflectively aware in the moments before a fulcrum experience occurs. That means you can watch the circumstances that give rise to the experience as they play out—but it also means that you can choose to intervene during precisely those moments, the ones most directly connected to the events you’re trying to understand.

If “changing things” is the first thing you try, you don’t know which moments matter, so you’re working blindly, by comparison. That’s fine in low-stakes situations when you’ve never tried to solve the problem before. But for recalcitrant problems, or in cases where making blind changes could cause lots of damage, it’s valuable to be gentle and precise.

Let’s return now to the relationship between naturalist experimentation and scientific experimentation.

Consider where someone must be, cognitively, while devising a plan to try to disconfirm a hypothesis. They have a working model—a story, of a sort—that’s strong enough not only to suggest where crucial data might live, but to produce a testable hypothesis. 

Their job as the test proceeds is to observe—but from what perspective are they observing? 

By default, which is all someone has access to before they’ve created space for alternative action, they are observing from the perspective of their working model. They are designing, performing, and interpreting their tests from within whatever stimulus-response chains existed before they began.

So it seems to me that the best way to make progress with scientific experimentation is to begin with a model that is at worst wrong. If you instead begin with a model that is not even wrong, a map that resembles an Escher painting more than any territory that could actually exist, then I think you have some work to do before you begin scientific experimentation—or, at the very least, some work to do concurrently—that is not itself part of scientific methodology as we usually understand it.

Naturalism is, I think, a formalization of the work that makes way for science—and naturalist experimentation is the final stone laid in the bridge between them.

 

How does naturalist experimentation work?

The procedure goes like this.

First, imagine the moment when you mark an experience just before it happens. Ask yourself what you do, by default, in the moments that follow, as the fulcrum experience plays out: What happens with your mind, and with your body?

Next, think of something different that you could do instead. It doesn’t need to be something that will improve the situation, or fix anything. It only needs to be an action that might make something unfamiliar happen, so that you can observe whatever results.

Example: 

I have a tendency to agree to things people propose when I’m overwhelmed. It’s just easier, in the moment, to go along with them than to figure out what I actually want. 

I studied this phenomenon, and I learned to notice when it’s about to happen. There’s a moment that feels like drifting on a current, just before I say “yes”; I learned to “wake up” when I feel that drift, to become reflectively aware. After a while, I could even be reflectively aware just before I started to drift.

What happens with my mind by default, in the moments just after I tap my leg to mark the experience, is a motion like flailing, accompanied by something that reminds me of the drowning roar of white noise. I quickly get tired of flailing, and “give in” to the white noise out of exhaustion. That is when I begin to feel “drifting”, and when going along with the other person’s request or suggestion feels immensely easier than doing anything else.

Because I’m able to be reflectively aware just before the drifting happens, there are many, many actions I could try besides that default sequence of “flailing, drowning, giving in”, provided I make a plan in advance. 

(In fact, I have already found out what happens if I tap my leg when I notice drifting. Tapping my leg is not part of the default string of events, and what results is reflective awareness. “Experimentation” is just an extension of that same practice, a version that includes actions with slightly larger consequences than merely “knowing that the experience is happening”.) 

Here are some of the first possibilities that come to mind for me:
 

  • I could say, “No.”
  • I could say, “Beep!”
  • I could recite the poem “Night Journey”.
  • I could turn away from the person who’s talking to me.
  • I could spin around in a circle.
  • I could do a handstand.
  • I could squat down and touch the ground with my hands.
  • I could say, “I can’t answer that right now.”
  • I could find three blue things in my environment.
  • I could ask for one full minute to think.
  • I could get a snack.
  • I could count my fingers.

 

My goal is to choose one of these actions to perform after tapping my leg that may cause something other than the default experience to happen. 

Note that in my example, the default experience is not just “saying yes” or “going along with the other person’s suggestion”. The default experience is “drifting”, plus the flailing→drowning→giving in sequence that precedes it. So I should choose an action that I think might cause an experience other than “flailing, drowning, giving in, and drifting”.

In social situations, I find that it’s important to start with buying time, whatever else I decide to try later. So from this list, I would first choose, “I need a minute to think.” After that, I think “counting my fingers” is a good experiment, both because I expect to be able to perform it in nearly any situation, and because I expect it to involve mental motions that are very different from “flailing”. I’d like to find out what those motions are, and how they feel in context.

So the next step is to try the experiment, and find out what happens. Just like in Getting Your Eyes On, my job at this point is to take at least one phenomenological snapshot of whatever happens when I ask for a minute to think and then count my fingers, when I would otherwise experience drifting and go along with someone’s request or suggestion.

And once I’ve done that, I repeat. I come up with other actions I could try in the crucial moment, and I find out what happens when I enact them.


A likely result of this process is discovering some action you prefer to take when your default response would otherwise happen—but that is not really the point. The point is that after a few rounds of this, you are no longer trapped in your default pattern

After a few rounds, you have created agency in a critical class of moments. You have freedom to move according to the limits of reality, instead of the limits of your preconceptions.

 

 

And because what you’ve created is not merely a specific new habit, but agency around a critical moment, you can continue to observe and learn to your heart’s content. You can investigate all the systems surrounding your topic until you are an expert in everything to do with it, if you want. You can not just “do a little better than before”, but master a whole swath of territory.

Scientia, as they say, potentia est.

 

 

What is it like to experiment in the context of a naturalist study?

For me, naturalist experimentation feels like refinement, honing, and taking the step from competence to mastery.

Sometimes, it also feels like playing around in frozen time, like the Matrix scene where Neo picks a bullet out of the air. I often look back, during this stage, and remember what it was like to be fully subject to a thing that I’m now skillfully manipulating. There is so much space and freedom after a bit of experimentation.

 

 

How do you know when you’re done?

Since experimentation is largely optional in naturalism, it’s fine to shift your focus elsewhere any time you feel like it.

I don’t think I’ve ever deliberately ended this phase, though. It always just fades into the background of my attention, eventually. It’s seemed that way in my interactions with students, as well. There isn’t a clear end, because somewhere between collection and experimentation, these kinds of moves tend to become so automatic that it would take effort to stop yourself from doing them.

The goal of a naturalist study is not really to obtain a final answer or clear-cut solution, even though you might end up with several of those along the way. The goal of naturalism is to reach a point where you relate to a part of the world in such a way that perpetual learning is inevitable.

A person has “completed” a naturalist study when their new default ways of thinking and acting lead them to better understand their chosen topic. You can be sure this has happened if you look back at the past six months and realize you’ve learned a lot about that thing you once studied deliberately, despite having mostly forgotten about it all this time.
 

 

Appendix 1: Process Summary

 

  • Ask yourself what you do by default during, and just before, your fulcrum experience. (Try to answer in concrete detail.)
  • Think of something different (but innocuous) that you could do instead.
  • Try out the alternative action, and take a phenomenological snapshot of whatever results.
  • Think of more alternative actions you could take, and try some of those out as well.

 


Appendix 2: Glossary

Creating space for alternative action: Breaking an existing pattern of stimulus-response, so the default action can be replaced with agency.


 

New Comment
14 comments, sorted by Click to highlight new comments since: Today at 9:52 AM

The goal of naturalism is to reach a point where you relate to a part of the world in such a way that perpetual learning is inevitable.

I utilize a stance that seems very similar in spirit and a number of details to what is described here, and I would like to emphasize the value of frequent, small experiments to gather knowledge and expand awareness of options. I have found the practice valuable in reducing the complexity and investment requirements of experimentation, as well as synchronizing well with the update speed of mental models and other "deep knowledge".

Meta question about naturalism-in-general: up until recently the LW tag for naturalism just said "Naturalism". I gave it a rough descriptor. I'd also like it to have a name like Naturalism [X], where X disambiguates it from other things you might use the word naturalism to mean. (in particular contrasted with "non-supernaturalism" or "liking/being-about nature (without being necessarily about observation). But I'm not actually sure what X to put there. I typically call it Logan-style Naturalism in casual conversation but not sure what's appropriate here.

I also suspect it'd be good to have secondary tags for most of the posts. In some cases, the right tag is "introspection" (maybe all cases? I'm not sure). We currently have an empiricism tag, we don't have an "observation" tag but maybe we should?

https://www.lesswrong.com/tag/naturalism 

(I maybe also will take this opportunity to note that I don't actually find naturalism a very intuitive name for the thing you're overall aiming at, but, I don't have a better suggestion and you're now almost two sequences in so, shrug)

I am curious whether reading or skimming the Wikipedia articles on "naturalistic observation" and "natural history" helps at all with getting where I'm coming from.

I certainly don't claim it was the best possible term to choose, but to me it seems extremely precise and accurate (though ambiguous, and i recognize that ppl round these parts are more familiar with philosophical naturalism qua ontological claim). In ecology, entomology, etc., the connotations go way beyond liking natural stuff, and suggest an orientation toward research topics and a corresponding set of methodologies. It's the thing Jane Goodall did, and also James Audubon. My stuff is like "What if the naturalist paradigm, but for stuff that includes rationality and not just for finches? What would that look like?"

[edit: on phone, might add links and mb further reply later]

That maybe updates me towards ‘yeah I guess Naturalism is a fine name for the study frame’ but still wanting a good shorthand to disambiguate with.

Yeah, makes sense. I'm pretty bad at this kind of thing I think, but I'll think about it and if I come up with something I'll let you know.

Those Wikipedia articles helped me understand.  I didn't until then, make the connection between "naturalist" as a descriptor of a person and "naturalism" as a descriptor of that person's ... Focus?  Toolkit?  Methodology?

Calling it "Naturalistic Observation" would have been much clearer to me, or really anything that doesn't end in -ism.  If you want to expand past traditional nature topics, perhaps focus on the observation part, not the nature part.

That said, naming things is famously hard, and I no longer have objections for myself - please consider those as suggestions for future readers.

Do you have an ELI5-level intro to what the naturalist frame is, and what other frames you're comparing to?  It feels a bit like you're arguing against a strawman, rather than providing either a contrast to an existing LW-common framing or just a set of hints on how to improve one's model of the universe.  I'm not sure how "naturalism" differs in theory, or if it's "just" a practice of understanding that's compatible with other empirical methods. 

Ordinarily, when someone is trying to solve a problem, experimentation is where they begin.

Not at all.  Observation is where everybody starts.  Observation and hypothesis generation are intertwined, but experimenting is definitely later (though experiments are also observations, so the bigger cycle happens as well).  Where the models actually come from has not been sufficiently explained, to my knowledge, but observation is definitely sufficient to narrow down hypotheses.

Unfortunately, observation can find correlations, but often not very strong ones, and without causality.  Experimentation creates unusual/extreme/simple contexts, which make the correlations much more visible, AND through differences in treatment, give evidence of causality.  Note that this doesn't quite sound like your use of "experiment" - if you just want to know if an intervention has a desirable impact, that's more of a test or treatment-candidate than an experiment.  The difference is whether you're trying to improve your model of the universe (through an experiment) or trying to directly improve an outcome (through trial and testing).

Honestly, many of the points and examples in your posts are really worthwhile, and they're on topics where the models of behavior and reaction are complex and individual enough that formal hypothesis identification and experimentation is probably of limited value.  I value noticing this and having mechanisms to address things without understanding them.  I don't think it's necessary to call it a contradictory thing from empiricism, it's just a set of tools to use in conjunction.  Or I completely misunderstand the idea, also very possible.

The Naturalism Sequence is an overall explanation of what Naturalism is.

My guess at an answer to your question, although I'm not 100% sure I grok Logan's frame:

I think you're right that everyone literally starts with observation, but in practice people often move pretty quickly to problem-solving. Or, if they take awhile before starting to problem solve, what's happening in the meantime is less like "more observation" and more like "observe once (badly), form a stereotype model of what's going on, and then notice the stereotype repeatedly until it prompts them to start experimenting, without ever really deeply observing."

I think the Naturalism sequence is making a fairly strong (but IMO obviously true) claim that people do very little Original Seeing (and the amount of original seeing here Logan is suggesting is multiple weeks of patient observation, actively investing in getting good at seeing all the details of what's going on with a phenomenon. I think hardly anyone does that).

(edit: also, it seems like you're contrasting this with "other formal/deliberate theories on how to observe/act" as opposed to "stuff random people do most of the time")

Thanks @Raemon. I agree with all of that.

>I don't know anyone who recommends "don't put much effort into understanding, just try stuff and see if it works", so I didn't expect that was the baseline that this sequence is arguing against.

@Dagon, I caution you that if you read this sequence (or the intro one) with the assumption that it's primarily trying to argue something, you'll probably be at risk of badly misinterpreting me.

I have a story that you're looking for and evaluating arguments here because you don't know what naturalism is or why it might be worth learning, so you hope to find motivating claims and arguments for paying attention to any of this in the first place. If this is a true story about you, I think that's pretty reasonable! I think it would be ridiculous of me to be like, "Here's a huge amount of work I suggest that you take on without having any particular reason for doing so," and I'm not very surprised if this sequence comes off that way to lots of people. But its actual intended audience is people who already want to learn something like this, for some reason, and are ready to do so.

This sequence is somewhere between a syllabus outline and a how-to guide. If you read a book about "how to design and tend a vegetable patch" through an argumentative lens, I expect you're going to find a lot of completely unsupported or incoherent arguments everywhere, it'll be pretty frustrating, and you probably won't learn nearly as much about how to design and tend a vegetable patch as you otherwise might. "How To Garden" is a completely different book from "Raised Bed Organic Gardening Is Better Than All of the Other Kinds of Gardening, and Here Is Why." I have tried to write the former type of book so far, not the latter.

 

Oh perhaps some of the confusion with this post in particular is coming from the fact that I tried to contrast three different frameworks for experimentation. Sometimes when people contrast different frameworks, they are doing that because they want to convince the reader that one of them is better than the others. I'm definitely not trying to do that here! I contrasted three experimental frameworks because in order to take the actions that are part of the overall naturalist investigative method, it's important to deliberately avoid falling into either of the other two near-by frames. I was trying to describe the mindset that the actions comprising naturalist experimentation come from.

Your story is pretty close.  I'm looking for the "what it is" first, and I kind of hope the "why it might be worth learning" is as obvious from that as it seems.  There's already parts of the sequence that I've enjoyed and found worth learning, especially examples of very complex interactions that are difficult to reduce to independently-modelable parts.  But it's not adding up in my head to anything coherent enough to get a name, certainly not something as evocative as "Naturalism".  

I apologize for my treating it as "compared to empiricism".  It may be I'm overreacting to openings like "Ordinarily, when someone is trying to solve a problem, experimentation is where they begin.", which is both incorrect AND implies that this is something very different from "ordinary".

Thanks.  I think I was looking for that very basic statement of what is being compared, in order to figure out when it's useful and when more traditional (but still rigorous) methods would be useful. 

"other formal/deliberate theories on how to observe/act" as opposed to "stuff random people do most of the time"

Yeah, this didn't occur to me.  I don't know anyone who recommends "don't put much effort into understanding, just try stuff and see if it works", so I didn't expect that was the baseline that this sequence is arguing against.

I realize this is only a response to one small thing in your comment and perhaps I will come back to the rest later, but I want to point out that according to me, I am definitely not arguing against anything at all.