High-level actions don’t screen off intent
28Mateusz Bagiński
9TsviBT
3TsviBT
9Chris Lakin
8testingthewaters
5Kaj_Sotala
4Unnamed
5Unnamed
4Kaj_Sotala
3Richard_Kennaway
2Adam Zerner
2azergante
3Matt Goldenberg
0M. Y. Zuo
New Comment
More generally, it's not just the effect that matters (globally/holistically/long-range-ish-ly speaking), but also whatever is ~causally upstream from the effect, because whatever is ~causally upstream from the effect determines what's gonna happen when the context changes the things upstream from the effect.
Conversely, contexts where high-level actions do screen off intent are hard to construct and fragile, and sometimes valuable, therefore precious. For example, you can have a tournament arbiter who enforces the rules completely and accurately. Ze might show disdain or admiration or whatnot towards some players, but if the rules are well-constructed and ze enforces them, in principle the screening-off could be near complete. (Or not, e.g. if the non-verbals nudge players to not bring up rules objections.) On the other hand, if there's even a small hole in the rules or probability of discretion in enforcement (as there usually is), the screening off is destroyed.
If you can make "golems" that transparently, mechanically enforce the rules, you get even more screening off. E.g. a computer-programmed game.
Guessing the intention of others can also help predict future actions. "Alice who was court ordered to donate bed nets and doesn't want to go to prison" vs. "Alice the EA who has read MacAskill cover to cover" have very different projected patterns of future behaviour, especially if the context changes (e.g. the court order is completed). Notably, that doesn't stop them from both clicking the same buy button on Amazon today.
I think of this as coarse-grained influence vs. fine-grained influence, which basically comes down to how many bits are needed to specify the nature of the influence.
e.g., Betty could cause one more girl to have a mentor either by volunteering as a Big Sister or by donating money to the Big Sisters program.
In the case where she volunteers and mentors the girl directly, it takes lots of bits to describe her influence on the girl being mentored. If you try to stick to the actions->consequences framework for understanding her influence, then Betty (like a gamer) is engaging in hundreds of actions per minute in her interactions with the girl - body language, word choice, tone of voice, timing, etc. What the girl gets out of the mentoring may not depend on every single one of these actions but it probably does depend on patterns in these micro-actions. So it seems more natural to think about Betty's fine-grained influence on the girl she's mentoring in terms of Betty's personality, motivations, etc., and how well she and the girl she's mentoring click, rather than exclusively trying to track how that's mediated by specific actions. If you wanted to know how the mentoring will go for the girl, you'd probably have questions about those sorts of things - "What is Betty like?", "How is she with kids?", etc.
In the case where Betty donates the money, the girl being mentored will still experience the mentoring in full detail, but most of those details won't be coming directly from Betty so Betty's main role is describable with just a few bits (gave $X which allowed them to recruit & support one more Big Sister). e.g., For the specific girl who got a mentor thanks to Betty's donation, it probably doesn't make any difference what facial expression Betty was making as she clicked the "donate" button, or whether she's kind or bitter at the world. Though there are still some indirect paths to Betty influencing fine-grained details for girls who receive Big Sisters mentoring, as the post notes, since the organization could change its operations to try to appeal to potential donors like Betty.
Subliminal learning in LLMs might also be an example of this (surely a sequence of random numbers can't convey any information about an intent to love owls, right?).
Hm. My instinct here is that the degree to which intent matters depends a lot on the situation. Often times it matters, often times it doesn't, often times it's somewhere on the spectrum.
I'm having trouble figuring out how that instinct meshes with the claim that this post makes. The post says that "in the main" intent matters, but yeah, I'm not really sure what that means.
I found the apology example helpful though. It made me realize that intent in those situations matters more than I previously thought, and that being honest/genuine is an even better heuristic than I previously thought. Maybe the idea behind this post is that people tend to undervalue how much intent matters.
The more interesting question is why would anyone ever assume that to be the case, in the first place?
Unless they’ve literally never encountered a deceptive person in their life, it just seems implausible to not notice this.
Curated and popular this week