All of bmgarfinkel's Comments + Replies

Re-posting a link here, on the off-chance it's of interest despite its length. ESRogs and I also had a parallel discussion on the EA Forum, which led me to write up this unjustifiably lengthy doc partly in response to that discussion and partly in response to the above comment.

1Lukas_Gloor4y
Thanks for this! My thinking is similar (I have an early draft about why realists and anti-realists diagree with one another, and have been trying to get closer to passing the Ideological Turing Test for realism. It was good to be able to compare my thinking to that of someone with stronger sympathies toward realism!)

Just wanted to say I really appreciate you taking the time to write up such a long, clear, and thoughtful response!

(If I have a bit of time and/or need to procrastinate anything in the near future, I may write up a few further thoughts under this comment.)

It sounds as though you're expecting anti-realists about normativity to tell you some arguments that will genuinely make you feel (close to) indifferent about whether to use Bayesianism, or whether to use induction.

Hm, this actually isn't an expectation I have. When I talk about "realists" and "anti-realists," in this post, I'm thinking of groups of people with different beliefs (rather than groups of people with different feelings). I don't think of anti-realism as having any strong link to feelings of indifference about behavior. For example: I certai

... (read more)
1Lukas_Gloor4y
Yeah, that makes sense. I was mostly replying to T3t's comment, especially this part: Upon re-reading T3t's comment, I now think I interpreted them uncharitably. Probably they meant that because induction seems impossible to justify, one way to "explain" this or come to terms with this is by endorsing anti-realism. (That interpretation would make sense to me!) I see. I think I understand the motivation to introduce irreducibly normative concepts into one's philosophical repertoire. Therefore, saying "I don't see the use" was a bit misleading. I think I meant that even though I understand the motivation, I don't actually think we can make it work. I also kind of see the motivation behind wanting libertarian free will, but I also don't think that works (and probably you'd agree on that one). So, I guess my main critique is that irreducibly normative concepts won't add anything we can actually make use of in practice, because I don't believe that your irreducibly normative concepts can ever be made coherent. I claim that if we think carefully about how words get their meaning, and then compare the situation with irreducibly normative concepts to other words, it'll become apparent that the irreducibly normative concepts have connotations that cannot go together with each other (at least not under the IMO proper account of how words get their meaning). So far, the arguments for my claim are mostly just implicitly in my head. I'm currently trying to write them up and I'll post them on the EA forum once it's all done. (But I feel like there's a sense in which the burden of proof isn't on the anti-realists here. If I was a moral realist, I'd want to have a good sense of how I could, in theory under ideal conditions, figure out normative truths. Or, if I accept the interpretation that it's conceivable that humans are forever incapable of figuring out normative truths, I'd at least need to have *some sense* of what it would mean for someone to not be forever incapable of f

Belatedly, is this a fair summary of your critique?

When someone thinks about another person (e.g. to predict whether they'll submit to blackmail), the act of thinking about the other person creates a sort of 'mental simulation' that has detailed concious experiences in its own right. So you never really know whether you're a flesh-and-blood person or a 'mental simulation' based on a flesh-and-blood person.

Now, suppose you seem to find yourself in a situation where you've been blackmailed. In this context, it's reasonable to wonder whether you're actually a

... (read more)

Hmm, I think focusing on a simpler case might be better for getting at the crux.

Suppose Alice says: "Eating meat is the most effective way to get protein. So if you want to get protein, you should eat meat."

And then Bob, an animal welfare person, responds: "You're wrong, people shouldn't eat meat no matter how much they care about getting protein."

If Alice doesn't mean for her second sentence to be totally redundant -- or if she is able to interpret Bob's response as an intelligible (if incorrect) statement of disagreement with her second sentence -- then

... (read more)
2Richard_Ngo4y
I don't think you can declare a sentence redundant without also considering the pragmatic aspects of meaning. In this example, Alice's second sentence is a stronger claim than the first, because it again contains an implicit clause: "If you want to get protein, and you don't have any other relevant goals, you should eat meat". Or maybe it's more like "If you want to get protein, and your other goals are standard ones, you should eat meat." Compare: Alice says "Jumping off cliffs without a parachute is a quick way to feel very excited. If you want to feel excited, you should jump off cliffs without a parachute." Bob says "No you shouldn't, because you'll die." Alice's first sentence is true, and her second sentence is false, so they can't be equivalent - but both of them can be interpreted as goal-conditional empirical sentences. It's just the case that when you make broad statements, pragmatically you are assuming a "normal" set of goals. It's not entirely unintelligible, because Alice is relying on an implicit premise of "standard goals" I mentioned above, and the reason people like Bob are so outspoken on this issue is because they're trying to change that norm of what we consider "standard goals". I do think that if Alice really understood normativity, she would tell Bob that she was trying to make a different type of claim to his one, because his was normative and hers wasn't - while conceding that he had reason to find the pragmatics of her sentence objectionable. Also, though, you've picked a case where the disputed statement is often used both in empirical ways and in normative ways. This is the least clear sort of example (especially since, pragmatically, when you repeat almost the same thing twice, it makes people think you're implying something different). The vast majority of examples of people using "if you want..., then you should..." seem clearly empirical to me - including many that are in morally relevant domains, where the pragmatics make their e

Okay, this seems like a crux of our disagreement. This statement seems pretty much equivalent to my statement #1 in almost all practical contexts. Can you point out how you think they differ?

This stuff is definitely a bit tricky to talk about, since people can use the word "should" in different ways. I think that sometimes when people say "You should do X if you want Y" they do basically just mean to say "If you do X you will receive Y." But it doesn't seem to me like this is always the case.

A couple examples:

1. "Bayesian updating has a certain asympto

... (read more)
3Richard_Ngo4y
I was a little imprecise in saying that they're exactly equivalent - the second sentence should also have a "in the limit of infinite compute" qualification. Or else we need a hidden assumption like "These asymptotic convergence properties give us reason to believe that even low-compute approximations to Bayesianism are very good ways to understand the world." This is usually left implicit, but it allows us to think of "if you want to understand the world, you should be (approximately) a Bayesian" as an empirical claim not a normative one. For this to actually be an example of normativity, it needs to be the case that some people consider this hidden assumption unnecessary and would endorse claims like "You should use low-compute approximations to Bayesianism because Bayesianism has certain asymptotic convergence properties, even if those properties don't give us any reason to think that low-compute approximations to Bayesianism help you understand the world better." Do you expect that people would endorse this?

It's important to disentangle two claims:

  1. In general, if you have the goal of understanding the world, or any other goal that relies on doing so, being Bayesian will allow you to achieve it to a greater extent than any other approach (in the limit of infinite compute).

  2. Regardless of your goals, you should be Bayesian anyway.

Believing #2 commits you to normative realism as I understand the term, but believing #1 doesn't - #1 is simply an empirical claim about what types of cognition tend to do best towards a broad class of goals. I think that many r

... (read more)
6Richard_Ngo4y
Okay, this seems like a crux of our disagreement. This statement seems pretty much equivalent to my statement #1 in almost all practical contexts. Can you point out how you think they differ? I agree that some statements of that form seem normative: e.g. "You should go to Spain if you want to go to Spain". However, that seems like an exception to me, because it provides no useful information about how to achieve the goal, and so from contextual clues would be interpreted as "I endorse your desire to go to Spain". Consider instead "If you want to murder someone without getting caught, you should plan carefully", which very much lacks endorsement. Or even "If you want to get to the bakery, you should take a left turn here." How do you feel about the normativity of the last statement in particular? How does it practically differ from "The most convenient way to get to the bakery from here is to take a left turn"? Clearly that's something almost everyone is a realist about (assuming a shared understanding of "convenient") at Less Wrong and elsewhere. I think there's a difference between a moral statement with conditions, and a statement about what is best to do given your goals (roughly corresponding to the difference between Kant's categorical and hypothetical imperatives). "You should give money to charity if you have enough money to live comfortably" is an example of the former - it's the latter which I'm saying aren't normative in any useful sense.

I left a sub-comment under Wei's comment (above) that hopefully unpacks this suggestion a bit

I think there's a distinction (although I'm not sure if I've talked explicitly about it before). Basically there's quite possibly more to what the "right" or "reasonable" action is than "what action that someone who tends to 'win' a lot over the course of their life would take?" because the latter isn't well defined. In a multiverse the same strategy/policy would lead to 100% winning in some worlds/branches and 100% losing in other worlds/branches, so you'd need some kind of "measure" to say who wins overall. But what the right measure is seems to be (or

... (read more)

If there is anything that anyone should in fact do, then I would say that meets the standards of "realism."

Does "anyone" refer to any human, or any possible being?

Sorry, I should have been clearer. I mean to say: "If there exists at least one entity, such that the entity should do something, then that meets the standards of 'realism.'"

I understand "moral realism" as a claim that there is a sequence of clever words that would convince the superintelligent spider that reducing human suffering is a good thing.

I don't think I'm aware of anyone who id

... (read more)
2Viliam4y
Some people seem to believe that about artificial intelligence. (Which will likely be more different from us than spiders are.)

Terminology definitely varies. FWIW, the breakdown of normative/meta-normative views I prefer is roughly in line with the breakdown Parfit uses in OWM (although he uses a somewhat wonkier term for "realism"). In this breakdown:

"Realist" views are ones under which there are facts about what people should do or what they have reason to do. "Anti-realist" views are ones under which there are no such facts. There are different versions of "realism" that claim that facts about what people should do are either "natural" (e.g. physical) or "non-natural" facts. If

... (read more)

Thanks for sharing this, was not aware of the survey! Seems like this suggests I've gotten a skewed impression of the distribution of meta-ethical views, so in that sense the objection I raise in this post may only be relevant to a smaller subset of the community than I'd previously thought.

I agree with a lot of the spirit of PMR (that people use the word "should" to mean different things in different contexts), but think that there's a particularly relevant and indespensible sense of the word "should" that points toward a not-easily-reducible property. Th

... (read more)
3ESRogs4y
From looking at the footnotes, I think maybe you mean the one that begins, "These metaphysical and epistemological issues become less concerning if..." Wanted to note that this is showing up as #15 for me.

If there is anything that anyone should in fact do, then I would say that meets the standards of "realism." For example, it could in principle turn out to be the case that the only normative fact is that the tallest man in the world should smile more. That would be an unusual normative theory, obviously, but I think it would still count as substantively normative.

I'm unsure whether this is a needlessly technical point, but sets of facts about what specific people should do also imply and are implied by facts about what everyone should do. For example, supp

... (read more)
1TAG4y
You should in fact pay your taxes. Which is to say that if a socially defined obligation is enough, then realism is true. But that might be setting the bar too low.
4Viliam4y
Does "anyone" refer to any human, or any possible being? Because it it refers to humans, we could argue that humans have many things in common. For example, maybe any (non-psychopathic) human should donate at least a little to effective altruism, because effective altruism brings the change they would wish to happen. But from the perspective of a hypothetical superintelligent spider living on Mars, donating to projects that effectively help humans is utterly pointless. (Assuming that spiders, even superintelligent ones, have zero empathy.) I understand "moral realism" as a claim that there is a sequence of clever words that would convince the superintelligent spider that reducing human suffering is a good thing. Not merely because humans might reciprocate, or because it would mean more food for the spider once the space train to Mars is built, but because that is simply the right thing to do. Such thing, I believe, does not exist.

If you mean "compelling" in the sense of "convincing" or "motivating," then I actually don't mean to suggest there are any "universally compelling normative statements." I think it's totally possible for there to be something that somone "should" do (e.g. being vegetarian), without this person either believing they should do it or acting on their belief.

This doesn't seem too problematic to me, though, since most other kinds of statements also fail to be at least universally convincing. For example, I also think that the statement "the universe is billions

... (read more)
8Said Achmiz4y
I think you’ve misunderstood the question, actually. “Compelling” here is to be read as in “No Universally Compelling Arguments”. So the question that clone of saturn was asking, it seems to me (he can correct me if I’m misinterpreting) is: suppose I claim that it’s the case that Bob, or all humans, or all Americans living in Florida whose name begins with a ‘B’, or any other proper subset A of “all agents”, should do X. (And suppose that X is a general injunction, in which all terms are properly quantified, etc., so that its limited applicability is not due to any particular features of the situation(s) which agents in subset A find themselves in; in other words, “agents outside subset A should also do X” could be true, but—I claim—it is not.) Now, is this realism, or anti-realism? I would not assent to the claim that “All agents should do [properly quantified] X”; yet nor would I assent to the claim “There is no fact of the matter about whether agents in subset A should do X”!

I wish when people did this kind of thing (i.e., respond to other people's ideas, arguments, or positions) they would give some links or quotes, so I can judge whether whatever they're responding to is being correctly understood and represented.

Fair point!

It's definitely possible I'm underestimating the popularity of realist views. In which case, I suppose this post can be take as a mostly redundant explanation of why I think people are sensible to have these views :)

I guess there are few reasons I've ended up with the impression that realist views aren

... (read more)
8Wei Dai4y
I think there's a distinction (although I'm not sure if I've talked explicitly about it before). Basically there's quite possibly more to what the "right" or "reasonable" action is than "what action that someone who tends to 'win' a lot over the course of their life would take?" because the latter isn't well defined. In a multiverse the same strategy/policy would lead to 100% winning in some worlds/branches and 100% losing in other worlds/branches, so you'd need some kind of "measure" to say who wins overall. But what the right measure is seems to be (or could be) a normative fact that can't be determined by just looking at or thinking "who tends to 'win' a lot'. ETA: Another way that "tends to win" isn't well defined is that if you look at the person who literally wins the most, they might just be very lucky instead of actually doing the "reasonable" thing. So I think "tends to win" is more of an intuition pump for what the right conception of "reasonable" is than actually identical to it.
9Richard_Ngo4y
It's important to disentangle two claims: 1. In general, if you have the goal of understanding the world, or any other goal that relies on doing so, being Bayesian will allow you to achieve it to a greater extent than any other approach (in the limit of infinite compute). 2. Regardless of your goals, you should be Bayesian anyway. Believing #2 commits you to normative realism as I understand the term, but believing #1 doesn't - #1 is simply an empirical claim about what types of cognition tend to do best towards a broad class of goals. I think that many rationalists would defend #1, and few would defend #2 - if you disagree, I'd be interested in seeing examples of the latter. (One test is by asking "Aside from moral considerations, if someone's only goal is to have false beliefs right now, should they believe true things anyway?") Either way, I agree with Wei that distinguishing between moral normativity and epistemic normativity is crucial for fruitful discussions on this topic. Another way of framing this distinction: assume there's one true theory of physics, call it T. Then someone might make the claim "Modelling the universe using T is the correct way to do so (in the limit of having infinite compute available)." This is analogous to claim #1, and believing this claim does not commit you to normative realism, because it does not imply that anyone should want to model the universe correctly. I would characterise "realism about rationality" as approximately equivalent to claim #1 above (plus a few other similar claims). In particular, it is a belief about whether there is a set of simple ideas which elegantly describe the sort of "agents" who do well at their "goals" - not a belief about the normative force of those ideas. Of course, under most reasonable interpretations of #2, the truth of #2 implies #1, but not vice versa.
1TAG4y
I wouldn't expect lesswrongians to be keen on Platonic style moral realism, where moral facts correspond to supernatural objects, but there are other classes of morally realistic theories' where moral facts depend on analytical truths or natural states of affairs. Lesswrongians are definitely keen utilitarianism, where ethical claims depend on natural facts about preferences, and is therefore, arguably, a naturalistic form of moral realism. The is-ought gap remains a problem which I touch on below. If normative realism is just the claim that there are meaningful and true statements about what you should do if you want to achieve some X, then they are abundant.. things like game theory and engineering, actually any kind of methodology, have plenty of them. What are the problems with normative realism about moral claims, then?Maybe that they are categorical, lacking an "if you want to do X" condition.
0Said Achmiz4y
This seems wrong to me. Could you say more about why you think this?