42
On Trust
28Dweomite
15Raemon
3Sune
24Raemon
6Gordon Seidoh Worley
1Sune
17Gordon Seidoh Worley
15Richard_Kennaway
15Thane Ruthenis
6Thane Ruthenis
9TekhneMakre
8David Lorell
6AnthonyC
8Sune
1Jacob G-W
8Alexander Gietelink Oldenziel
7JBlack
1polytope
7Yoav Ravid
5Adam Zerner
5Yoav Ravid
2Adam Zerner
4David Gross
4David Zeller
3David Joshua Sartor
3Htarlov
2johnswentworth
New Comment
In security engineering, a trusted component of a system is a component that has the ability to violate the system's security guarantees. For instance, if a security engineer says "Alice is trusted to guard the cookie jar", that means "Alice has the ability to render the cookie jar unguarded".
I notice that the four examples at the beginning of this post all seem to slot pretty nicely into this definition:
- "I decided to trust her" => I assigned Alice to guard the cookie jar by herself
- "Should I trust him?" => Should I allow Bob to guard the cookie jar?
- "Trust me" => Please allow me to access the cookie jar
- "They offered me their trust" => They granted me access to the cookie jar
If you think about them as being about security policies, rather than epistemic states, then they seem to make a lot more sense.
I think the layperson's informal concept of "trust" is more muddled than this, and conflates "I'm giving you the power to violate my security" with "I am comfortable with you having the power to violate my security" and maybe some other stuff.
Oh, this actually feels related to my relational-stance take on this. When I decide to trust a friend, colleague or romantic partner, I'm giving them the power to hurt me in some way (possibly psychologically). There's practical versions of this, but part of it is something like "we are choosing to be the sort of people who going to share secrets and vulnerabilities to each other."
This is also just another way of saying “willing to be vulnerable” (from my answer below) or maybe “decision to be vulnerable”. Many of these answers are just saying the same thing in different words.
This feels kinda straw-vulcany, sort of missing the point about what people are often using trust for.
I'm not actually sure what trust is, but when I imagine people saying the sentences at the beginning, at least 35% and maybe 75% of what's going on is more about managing a relational stance, i.e something like "do you respect me?".
I do expect you'll followup with "yeah, I am also just not down to respect people the particular way they want to be respected."
So a major part of how I handle this sort of thing is usually conveying somehow "I don't believe that, and generally don't trust your epistemics enough to believe-you-by-default in this domain, but, also, I [respect, in whatever way I do happen to respect] the person." (Usually this is managed more by vibes than by words, though if I found myself in a situation where someone I cared about said "trust me" it probably means something is fragile enough to require some careful linguistic work as well)
I've been really frustrated in the past with folks who equate trust with respect.
My ex-wife frequently complained that I didn't trust her. Why? Because she'd ask me to do something and, rather than simply do it, I'd ask why. Most of the time I was just curious (can you imagine? someone who posts on LessWrong was curious?) and wanted to know more, but she read it as me distrusting and thus not respecting or being committed to her.
Mostly I just select myself out of relationships with people who are like this now.
The flip side of this, though, is that there's a part of my life where I do things without asking why, which is as part of my Zen practice. Our rituals exist because someone created them, but the intent is intentionally not communicated. This is to create the experience of not knowing why you do something and having to live with not knowing. You might eventually come up with your own reason for why you do a particular ritual, but then that's something you added that you can explore rather than something you've taken as given from a teacher, senior student, etc. For example, new people often ask why we bow. The answer: because that's what we do. If bowing is to mean something, it's up to you to figure out what it means.
What does respect mean in this case? That is a word I don’t really understand and seems to be a combination of many different concepts being mixed together.
There's a popular position that goes something like "trust people when they tell you how they are feeling, what it's like to be them, etc.". I have a hard time with this, because I am so often wrong about what I think it's like to be me, and my experience is that many people are similarly deluded into a false sense of confidence that they understand themselves well. But also because talk is cheap evidence, and thus must be discounted when updating beliefs.
Now first off I have to say that the opposite position also seems wrong to me. People clearly have some insight into what's going on with themselves. It's not as if when someone says "ouch, you hurt me" they're just saying random words uncorrelated with reality. But also talk is cheap and you can say the words "ouch, you hurt me" when in fact you were not hurt. If there's incentives to say "ouch, you hurt me" even when not hurt, you may utter those words even if not hurt because doing so is likely to generate a good outcome for you. For bonus points, to make your utterance really convincing and thus more likely to effectively get you what you want, you might even convince yourself that you really were hurt even when you weren't.
This means that when someone makes a self-claim, I'm forced to discount that claim against the probability that the self-claim reflects hallucination, confusion, delusion, or outright deception. Self-claims become more reliable when backed by costly signals, but without such costly signals it's hard to update much on most self-claims.
"Do I trust this rope bridge to carry my weight?" = "Having estimated its strength relative to my weight as best I can, am I now going to cross it?"
The main exception here is self-fulfilling prophecy, but that’s not obviously centrally involved in whatever “I decided to trust her” means.
I think it might be.
I think the core of the contractual interpretation is right, but I'd phrase it differently. I think "deciding to trust" someone implies a deal where you ally yourself with them in a way that leaves you vulnerable to hostile actions from their side – you're choosing "cooperate" in the Prisoner's Dilemma in expectation of greater mutual gain. In exchange, they (1) commit not to take said hostile actions, (2) potentially "reciprocate trust" by showing symmetrical vulnerability (either to ensure MAD, or because pooling resources improves your ability to work together), and (3) bias their policy towards satisfying your preferences, which includes being more responsible when interacting with you and giving you some veto on their actions (and if they fail that, they've "betrayed" your trust).
And there's some degree of acausal culture-based negotiation going on. E. g., it's not that people explicitly track all of the above consciously, it's that we have subconscious pre-computed scripts of "if someone utters 'I trust you' and your model marks them as an ally, modify the prior used by your plan-making processes towards satisfying their preferences more". Which is implemented by automatically-activated emotional responses; e. g., the feeling of responsibility automatically flaring up when you're thinking about someone who put their trust in you.
So to some extent it is a self-fulfilling prophecy. If you've decided to trust someone, and their learned-instinct systems recognize that, they'll automatically reciprocate and become more trustworthy. (Or so the underlying calculations go. It might not work: if they jailbroke out of these systems, if they've never had them installed (psychopaths), or if other systems overrode that.)
Not that there isn't all kinds of sloppy language involved, e. g. when "I trust her" actually just means "I've evaluated that the process she uses for [whatever] is reliable" or "leaving myself vulnerable here is a risk, but the CBA is in favour", instead of "I've initiated the 'trust' social dance with her and I don't think she's defecting".
And there's some degree of acausal culture-based negotiation going on
Actually, this seems like an extremely clever instance of that. What evolution (either biological or cultural/memetic) has on its hands is a bunch of scheming, myopic mesa-optimizers walking around in suits of autonomous heuristics. It wants them to be able to cooperate, but the mesa-optimizers are at once too myopic and too clever for that – they pretend to ally then backstab each other.
So what it does is instill a joint social and cognitive protocol which activates whenever someone shows vulnerability to someone else: it initiates a process in the trustee's mind which immediately strongly biases their short-term plans towards reciprocating that trust by showing vulnerability in turn (which reads to us as, e. g., warm feelings of empathy), which establishes mutually assured destruction, which sets up a lasting incentive structure that ensures the scheming optimizer won't immediately betray even if the biochemistry stops actively brainwashing them (which it must stop, because that impairs clear thinking).
Or at least that's the angle on the issue that just occurred to me. May be missing some important pieces, obviously.
Seems fascinating, regardless.
"Trust" is like "invest". It's an action-policy; it's related to beliefs, such as "this person will interpret agreements reasonably", "this person will do mostly sane things", "this person won't breach contracts except in extreme circumstances", etc., but trust is the action-policy of investing in plans that only make sense if the person has those properties.
1. "Trust" does seem to me to often be an epistemically broken thing that rides on human-peculiar social dynamics and often shakes out to gut-understandings of honor and respect and loyalty etc.
2. I think there is a version that doesn't route through that stuff. Trust in the "trust me" sense is a bid for present-but-not-necessarily-permanent suspension of disbelief, where the stakes are social credit. I.e. When I say, "trust me on this," I'm really saying something like, "All of that anxious analysis you might be about to do to determine if X is true? Don't do it. I claim that using my best-effort model of your values, the thing you should assume/do to fulfill them in this case is X. To the extent that you agree that I know you well and want to help you and tend to do well for myself in similar situations, defer to me on this. I predict you'll thank me for it (because, e.g., confirming it yourself before acting is costly), and if not...well I'm willing to stake some amount of the social credit I have with you on it." [Edit: By social credit here I meant something like: The credence you give to it being a good idea to engage with me like this.]
Similarly:
- "I decided to trust her" -> "I decided to defer to her claims on this thing without looking into it much myself (because it would be costly to do otherwise and I believe-- for some reason-- that she is sufficiently likely to come to true conclusions on this, is probably trying to help me, knows me fairly well etc.) And if this turns out badly, I'll (hopefully) stop deciding to do this."
- "Should I trust him?" -> "Does the cost/benefit analysis gestured at above come out net positive in expectation if I defer to him on this?"
- "They offered me their trust" -> "They believe that deferring to me is their current best move and if I screw this up enough, they will (hopefully) stop thinking that."
So, I feel like I've landed fairly close to where you did but there is a difference in emphasis or maybe specificity. There's more there than asking “what do they believe, and what caused them to believe it?” Like, that probably covers it but more specifically the question I can imagine people asking when wondering whether or not to "trust" someone is instead, "do I believe that deferring these decisions/assumptions to them in this case will turn out better for me than otherwise?" Where the answer can be "yes" because of things like cost-of-information or time constraints etc. If you map "what do they believe" to "what do they believe that I should assume/do" and "what caused them to believe it" to "how much do they want to help me, how well do they know me, how effective are they in this domain, ..." then we're on the same page.
I think this is a big part of it. It can also include, "I have information I'm not supposed to share, don't know how to share, or don't have time to share."
1My favourite definition of trust is “willingness to be vulnerable” and I think this answers most of the questions in the post. For example it explains why trust is a decision that can exist independently from your beliefs: if you think someone is genuinely on your side with probability 95%, you can choose to trust them, by doing something that benefit you in 95% of cases and hurt you on the 5% of cases, or you can decide not to, by taking actions that are better in the 5% of cases. Similar for trusting a statement about the world.
I think this definition comes from psychology, but I also found it useful when talking about trusted third parties in cryptography. Also in this case, we don’t care about the probability that the third part is malicious, what matters is that you are vulnerable if and only if they are malicious.
Yes, this is pretty much how I see trust. It is an abstraction over how much I would think that the other person will do what I would want them to do.
Trusting someone means that I don't have to double-check their work and we can work closer and faster together. If I don't trust someone to do something, I have to spend much more time verifying that the thing that they are doing is correct.
Another curious thing about trust. The median LW reader is probably better informed about a great many things than their immediate family, friends from their home town. Yet, I suspect you will agree, family members almost never defer to them epistemically.
This seems to be a general phenomenon - many people could plausibly be epistemically better off by finding and deferring to their epistemic superiors. This doesn't need to entail blank-acceptance of any claim that somebody that seems smarter / better informed on a topic - just putting effort in finding who are the best informed on certain domains and comparing them to other would-be experts. In many cases this would be much more effective than trying to figure out things oneself. People generally seem not so interested in finding and comparing experts.
Society has created institutions to combat this curious phenomenon. The magic of credentialism endows a select group of (hopefully) relatively highly informed people with a magic paper that compells peasants to submit to the scholar-clergy.
To me, the phrase “I decided to trust her” throws an error. It’s the “decided” part that’s the problem: beliefs are not supposed to involve any “deciding”. There’s priors, there’s evidence, and if it feels like there’s a degree of freedom in what to do with those, then something has probably gone wrong.
Trust isn't about having a particular belief, it's about acting on a particular type of hypothesis despite lack of convincing evidence supporting it.
"I decided to trust her" does not mean that you believe that she absolutely won't do some bad X, but that you decided to act as if she won't. That is, you were prepared to take the risk.
Likewise to "trust me" doesn't mean believing that I will never do a bad thing, but to take the risk despite the lack of sufficient evidence.
> lack of sufficient evidence.
Perhaps more specifically, evidence that is independent from the person that is to be trusted or not. Presumably when trusting someone else that something is true, often one does so due to believing that the other person is being honest and reliable enough such that that their word is sufficient evidence to then take some action. It's just that there isn't sufficient evidence without that person's word.
I've always felt basically the same about trust. It's nice to see that I'm not the only one.
I'd add/emphasize that trust is really a 2-place word. It doesn't really make sense to say "I trust Alice". Instead, it'd make sense to say "I trust Alice to do X" or "I trust Alice's belief about X" or "I expect Alice's prediction about X to be true". Ie. instead of trust(person), it's trust(person, thing).
(And of course, as mentioned in the post, it isn't binary either. The output of the function is a probability. A number between zero and one. Not a boolean.)
To me, the phrase “I decided to trust her” throws an error. It’s the “decided” part that’s the problem: beliefs are not supposed to involve any “deciding”.
To trust is more than a passive cognitive reflection like a belief, it is also an action taken upon the world. This might be more easily seen if you consider the more awkward phrasing "I decided to extend my trust to her".
The thing which is natural to me is: when someone makes a claim, or gives me information, I intuitively think “what process led to them making this claim or giving me this information, and does that process systematically make the claim/information match the territory?”.
Ah, that's what trust is to me! That resonates strongly. I hadn't properly thought through the concept before.
The big question floating around my mind is - to what extent should I let *this* kind of trust (or lack of trust) determine who I'm close to?
It’s the “decided” part that’s the problem: beliefs are not supposed to involve any “deciding”.
I can pretty easily shift my perspective such that learning what I'm going to do feels like realizing that my action is overdetermined, rather than like "deciding", for almost every action (and better meditators can get every action to feel this way). What I do to achieve this is: manually redefine my identity to exclude most of my decision-making process.
Similarly, many people include part of their world-model in their identity, such that learning about the world can feel like deciding something. The world-model's doing very similar computation to the planner and whatnot, it seems reasonable for some people to include it.
There’s priors, there’s evidence, and if it feels like there’s a degree of freedom in what to do with those, then something has probably gone wrong.
Can just as easily say "There's beliefs, there's values, and if it feels like there’s a degree of freedom in what to do with those, then something has probably gone wrong.". There's only one optimal decision, given a set of beliefs and values. (Of course we're bounded, but that applies just as well to what we do with beliefs and evidence.)
I think that in an ideal world where you could review all priors to very minute details having as much time as needed, and where people were fully rational, then "trust" as a word would not be needed.
We don't live in such a world though.
If someone says "trust me" then in my opinion it conveys two meanings on two different planes (usually both, sometimes only one):
- Emotional. Most people base their choices on emotions and relations, not rational thought. Words like "trust me" or "you can trust me" convey an emotional message asking for an emotional connection or reconsideration, usually because of some contextual reason (like the other person being in a position that on an emotional level seems to be trustworthy, f.ex. a doctor).
- Rational. Time for reconsideration. The person asks you to take more time to reconsider your position because she or he thinks you didn't consider well enough why she or he is to be trusted in a given scope or that person just presented some new information (like "trust me, I'm an engineer").
"I decided to trust her about ..." - for me, it is a short colloquial term for "I took time to reconsider if things she says on the topic ... are true and now I think that is more likely that they are".
For many people, it also has emotional and bonding components.
Another thing is that people tend to trust or mistrust another person in general broad scope. They don't go into detail and think separately on every topic or thing someone says and decide separately for each of them. That's an easy heuristic that usually is good enough, so our minds are wired to operate like that. So people usually say that they trust a person generally, not trust that person within some subject/scope.
P.S. I'm from a different part of the world (central EU, Poland). We don't use phrases like "accept trust" here - which is probably an interesting difference in how differences in language create different ways of thinking. For us here "trust" is not like a contract. It is more a one-way thing (but with some expectation of mutuality in most circumstances).
Curated and popular this week