I found the conversation with Robin. (Or, rather, Robin found the conversation. Thanks, Robin.)

Robin: [shared with permission]

[talking about a bit I was trying to re-write in the next essay, Direct Observation]

First paragraph feels great.

the the last line, though, "The processing distance is finite" immediately raises the question of "WOAH what would infinite / extremely large processing distance be...?" and I'm off thinking about that.

Second paragraph: makes sense, and I agree with you, but I have some expectation of... explanation? You've stated that the processing distance is shorter, but without further words on how you know / what that's made out of.

(And how does a person know? By poking something and examining the sensation, and comparing it to the sensation of planning something? It might be a good moment to prompt the reader to try it and examine the difference?)

Logan: 

I tried to write about this, but it was long and clumsy. I'm not sure if I'll take a second pass at it, if I'll just leave it out (especially since it's a pretty new and not-very-investigated line of thought for me). But I thought you'd like to see what I have:

Here’s a test for estimating processing distance: If your expectations happened to be incorrect, how easy would it be to feel the resistance of reality?

I have some expectations around what kinds of foods a cat will enjoy. If you’d asked me two weeks ago, “What treats should I offer my cat while helping him become more comfortable around vacuum cleaners?” I’d recommend little chunks of fresh chicken, beef, or seafood. That was my implicit belief for many years, while I did not actually know any cats.

Recently, while preparing to adopt a kitten, I looked into this question, and multiple websites confirmed that super high-value treats for cats include fresh poultry, meat, and seafood.

But when I actually got the kitten, reality resisted my expectations by literally spitting out chunks of fresh chicken, grilled steak, and shrimp, right in front of me. It would have been hard for me not to notice this happening. It may be true that cats in general enjoy fresh chicken; but it was immediately apparent to me, as I watched Adagio literally turn up his nose at the tasty morsel, that this one does not.

If I’d never offered Adagio chicken, I could have gone on believing indefinitely that he likes chicken. There would have been no opportunity to encounter resistance through my imagination, through books by cat experts, even through personal interactions with other cats. It was only by offering chicken directly to my real-life chicken-hating cat that I gained the opportunity to notice I was wrong.

When I imagined Adagio eating chicken, my imagination filled in a picture of him munching eagerly, even trying to swipe additional bits from my hand. The processing distances between cat palates and human imaginings of cats is quite large, so there’s little opportunity for expectation to encounter resistance. When I read accounts of cats, the distance is a bit less, because those words are downstream of observation of actual cats; if I’d been so wrong as to expect Adagio will like oranges, googling probably could have set me right. Cats supposedly hate citrus. (I offered him an orange just now, to be sure. He got up and backed away from it.)

But even if my priors are strong, and even if I’m very stubborn or invested in my expectations—if I’ve just spent a lot of money on fancy cat food featuring fresh chicken, for instance—I’d have to work pretty hard not to notice my surprise when I offer an actual chicken-hating cat an actual piece of chicken from my own hand.

[end attempted essay section, begin talking directly to Robin again]

so to answer your question, according to my working model, infinite processing distances are those that offer no opportunity whatsoever to encounter resistance from reality when your expectations are incorrect. the distance between snakes and cello sonatas is extremely large, because snakes don't have ears. (not a perfect example, since they do have some other methods of sensing vibrations.) if a snake is somehow mistaken about the properties of a cello sonata, it will have to work extremely hard to even find an opportunity to discover this, nevermind making use of that opportunity. it might have to, i don't know, learn to read musical notation, and even then a lot of potentially crucial information (like the overtones created by the shape of the cello body, or a particular artist's interpretation) will not be available.

(i don't know why i'm talking about snakes. deaf humans also exist.)

so it's like, how tightly entangled is any given experience with the bit of reality you're hoping to learn about from that experience? the totally sound-free experience of smelling a tasty cricket in front of your tongue is not very entangled with a musical performance, even if the performance is happening just meters away from you. a scratchy recording of that performance on an ancient beaten-up record heard decades later on the other side of the planet, much more so.

Robin:

(I did want to see this, yes!)

>”Here’s a test for estimating processing distance: ‘If your expectations happened to be incorrect, how easy would it be to feel the resistance of reality?’”

Oh, I like this. (Though, “the resistance of reality” throws a little ‘?’ For me when I look at it too closely; I can tell my felt-sense for “noticing surprise/confusion” is bubbling up to take the space of the phrase when I don’t look too closely, though, so I must think you mean something like that.)

>”When I read accounts of cats, the distance is a bit less, because those words are downstream of observation of actual cats; if I’d been so wrong as to expect Adagio will like oranges, googling probably could have set me right.”

I feel torn on casting reading/hearing another’s words as being any closer to the territory than imagining, since it doesn’t actually provide an opportunity to encounter the ‘resistance of reality’. (Instead, it seems like straightforwardly an updating-my-map-with-your-map activity?)

That said… you can update your map using someone else’s map so that it’s easier to actually encounter something in the territory. (For example, you could’ve gone your whole life without offering an orange to a cat; but you did try, you moved to encounter that territory, ~because someone shared a map that said you would see a particular thing (cat-upset) at a particular place (the interaction between cat and orange)).  

There is a sort of… temporal closeness there?  

I feel like something important is solidifying for me here; something about how maps are actually useful as maps, as in they might point you toward the appropriate sections of territory for whatever it is you want to look at. Before I was just thinking of ‘maps’ as a useful metaphor for abstraction. Hm.

also... something something the ability to manipulate the territory as a sign that you are maximally close to it?

I typed that, then switched to reading A Process Model by Gendlin, and the following sentences popped out at me:

"As ongoing interaction with the world, our bodies are also a bodily knowing of the world. It is what Merleau-Ponty calls 'the knowing body' (le corps connais- sant). Such knowing consists in much more than external observation; it is the body always already interacting directly with the situation it finds itself in."

The (seeming) relevance made me laugh.

Where does 'interaction' fit in all of this anyway?

Logan:

it somehow fits into the heart of deep mastery [from "Knowing"]

[end of my conversation with Robin]

the conversation with robin you quoted did feel relevant, but the parent comment felt like it was too focused in on math and thereby somewhat orthogonal to or missing the point of what i was trying to figure out. (the real thing i'm interested in isn't even about math but about philosophical intuitions.)

this made me want to try to say the thing differently, this time using the concept of gears-level models:

https://www.lesswrong.com/posts/B7P97C27rvHPz3s9B/gears-in-understanding

(maybe everything i'm saying below is obvious already, but then again, maybe it'll help.)

suppose that you are looking at a drawing of a simple mechanism, like the second image in the article above, which i'll try to reproduce here:

if the drawing is detailed enough and you are able to understand the mechanism well enough, then you can reason out how the mechanism will behave in different circumstances; in the example, you can figure out that if the gear on the top-left turns counter-clockwise, the gear on the bottom-right will turn counter-clockwise as well.

but you haven't interacted with the mechanism at all! everything you've done has happened inside your map!

nevertheless, if you understand how gears work and you look at the drawing and think in detail about how each gear will turn, your model resists the idea that the bottom-right gear can turn clockwise while the top-left one turns counter-clockwise. it might be that your model of how gears work is wrong, or it might be that the drawing doesn't accurately represent how the mechanism works, or you might be misunderstanding it, or you might be making a mistake while thinking about it. but if none of these is true, and the top-left gear turns counter-clockwise, then the bottom-right gear has to turn counter-clockwise as well.

when you work out from the drawing how the bottom-right gear has to turn, are you in contact, not just with any part of the territory, but with the part of the territory that is the actual physical gears? even though you are not physically interacting with the actual gears at all, just thinking about a map of the gears?

the way i'm thinking about it in the top-level comment is that since this process is able to resist misconceptions you may have, and is thereby able to bring your {anticipations / implicit models} about the physical gears more in line with reality, therefore yes it is "contact with that part of the territory" in the sense that is relevant to "knowledge of the territory takes patient and direct contact with the territory."

"a map that clings super tightly to the territory" is a phrase from duncan's reply to my comment on "The Territory" which seems to me to describe gears-level models well.

– # –

i should note that the thing i'm ultimately interested in, namely the way i use philosophical intuitions in my work on agi alignment, isn't even anywhere as detailed as a gears-level model. nevertheless, i still think that these intuitions cling sufficiently tightly to the territory that this work is well worth doing. in the ontology of my top-level comment, my work is betting on these intuitions being good enough to be able to resist and correct my implicit models of agi alignment, and to therefore constitute significant contact with this region of the territory.

something i don't know how to reflect well in a comment like this, and think i should say explicitly, is that the game i'm playing here is not just to find a version of logan's sentence that covers the kind of work i do. it is to find a version that does that and additionally does not lose what i thought i understood when i was taking "contact with the territory" to be the opposite of "it all happens in your map", and therefore would have taken {thinking about a drawing of the mechanism} as not being in contact with the territory, since it consists entirely of thinking about a map.

for some reason i haven't really figured out yet, it seemed really important for this to say that in order to be "contact with the territory", an experience has to be able to resist and correct my {anticipations / implicit models}, not just my explicit verbal models.

(i tried to say some more things about this here, but apparently it needs to gestate more first. it wouldn't be very surprising if i ended up later disagreeing with my current model, since it's not even particularly clear to me yet.)

As a relatively non-verbal person, I always feel like someone is walking upside down with legs sticking out of their head when they make claims about vocabulary being necessary for things besides talking to each other. There must be quite the inferential gap here. Wh... Whyy? Why would having vocabulary for the observation be important to making a good observation? Maybe you mean something I'm not expecting by "good"? Or by "vocabulary"?

I also don't understand "Making good observations seems very dependent on what we are looking for". Do you mean something like, "Whether or not we deem an observation to be 'good' depends on why we're making observations, since 'goodness' only exists in relation to goals?"

Perhaps I just somehow completely don't understand this comment at all. But I guess Robin did? I wonder what Robin heard.