Dagon

Just this guy, you know?

Comments

Comparing Utilities

Ah, I think I understand better - I was assuming a much stronger statement of what social choice function is rational for everyone to have, rather than just that there exists a (very large) set of social choice functions, and it it rational for an agent to have any of them, even if it massively differs from other agents' functions.

Thanks for taking the time down this rabbit hole to clarify it for me.

Dagon's Shortform

Useful pointers. I do remember those conversations, of course, and I think the objections (and valid uses) remain - one can learn from unlikely or impossible hypotheticals, but it takes extra steps to specify why some parts of it would be applicable to real situations. I also remember the decoupling vs contextualizing discussion, and hadn't connected it to this topic - I'm going to have to think more before I really understand whether Newcomb-like problems have clear enough paths to applicability that they can be decoupled by default or whether there's a default context I can just apply to make sense of them.

Covid 9/17: It’s Worse

If you think the identical processes and level of caution should be used for an emergent pandemic as for relatively small-scale long-standing viruses, you're not doing cost/benefit analysis very well. It's very hard for me to simultaneously believe that it's so risky that we should all avoid travel and most leisure activities, AND that the vaccine is so unimportant that we shouldn't accept more risks than we otherwise would.

I'll respond to Natalie Dean's quote, because they're easy bullet points.

Gives people a false sense of security if efficacy is really low

Perhaps true, but efficacy would have to be ridiculously low for it to be a net loss. Which will show in early trials and uses.

Diverts resources away from other interventions (fixing testing!)

Do both!

Makes it harder to evaluate better vaccines

Only to the extent that it's effective and very common. Which is a good outcome in itself.

Jeopardizes safety

More than a 6-month delay would? I doubt it.

Erodes trust in the process

That implies that anyone trusts the process now.

Making the Monte Hall problem weirder but obvious

An easier illustration is "Monty doesn't open any doors, he just gives the player to stay with their chosen door, or switch to ALL of the other doors". This is isomorphic to showing losers for all but one of the other doors.

Like the original, it's somewhat ambiguous if you don't specify that Monty will do this regardless of whether the player guessed right the first time.

Dagon's Shortform

I always enjoy convoluted Omega situations, but I don't understand how these theoretical entities get to the point where their priors are as stated (and especially the meta-priors about how they should frame the decision problem).

Before the start of the game, Omega has some prior distribution of the Agent's beliefs and update mechanisms. And the Agent has some distribution of beliefs about Omega's predictive power over situations where the Agent "feels like" it has a choice. What experiences cause Omega to update sufficiently to even offer the problem (ok, this is easy: quantum brain scan or other Star-Trek technobabble)? But what lets the Agent update to believing that their qualia of free-will is such an illusion in this case? And how do they then NOT meta-update to understand the belief-action-payout matrix well enough to take the most-profitable action?

Comparing Utilities

I follow a bit more, but I still feel we've missed a step in stating whether it's "a social choice function, which each agent has as part of it's preference set", or "the social choice function, shared across agents somehow". I think we're agreed that there are tons of rational social choice functions, and perhaps we're agreed that there's no reason to expect different individuals to have the same weights for the same not-me actors.

I'm not sure I follow that it has to be linear - I suspect higher-order polynomials will work just as well. Even if linear, there are a very wide range of transformation matrices that can be reasonably chosen, all of which are compatible with not blocking Pareto improvements and still not agreeing on most tradeoffs.

If you imagine that you're trying to use this argument to convince someone to be utilitarian, this is the step where you're like "if it doesn't make any difference to you, but it's better for them, then wouldn't you prefer it to happen?"

Now I'm lost again. "you should have a preference over something where you have no preference" is nonsense, isn't it? Either the someone in question has a utility function which includes terms for (their beliefs about) other agents' preferences (that is, they have a social choice function as part of their preferences), in which case the change will ALREADY BE positive for their utility, or that's already factored in and that's why it nets to neutral for the agent, and the argument is moot. In either case, the fact that it's a Pareto improvement is irrelevant - they will ALSO be positive about some tradeoff cases, where their chosen aggregation function ends up positive. There is no social aggregation function that turns a neutral into a positive for Pareto choices, and fails to turn a non-Pareto case into a positive.

To me, the premise seems off - I doubt the target of the argument is understanding what "neutral" means in this discussion, and is not correctly identifying a preference for pareto options. Or perhaps prefers them for the beauty and simplicy of them, and that doesn't extend to other decisions.

If you're just saying "people don't understand their own utility functions very well, and this is an intuition pump to help them see this aspect", that's fine, but "theorem" implies something deeper than that.

Comparing Utilities
I'm feeling a bit of "are you trolling me" here.

Me too! I'm having trouble seeing how that version of the pareto-preference assumption isn't already assuming what you're trying to show, that there is a universally-usable social aggregation function. Or maybe I misunderstand what you're trying to show - are you claiming that there is a (or a family of) aggregation function that are privileged and should be used for Utilitarian/Altruistic purposes?

So a pareto improvement is a move that is > for at least one agent, and >= for the rest.

Agreed so far. And now we have to specify which agent's preferences we're talking about when we say "support". If it's > for the agent in question, they clearly support it. If it's =, they don't oppose it, but don't necessarily support it.

The assumption I missed was that there are people who claim that a change is = for them, but also they support it. I think that's a confusing use of "preferences". If it's =, that strongly implies neutrality (really, by definition of preference utility), and "active support" strongly implies > (again, that's the definition of preference). I still think I'm missing an important assumption here, and that's causing us to talk past each other.

When I say "Pareto optimality is min-bar for agreement", I'm making a distinction between literal consensus, where all agents actually agree to a change, and assumed improvement, where an agent makes a unilateral (or population-subset) decision, and justifies it based on their preferred aggregation function. Pareto optimality tells us something about agreement. It tells us nothing about applicability of any possible aggregation function.

In my mind, we hit the same comparability problem for Pareto vs non-Pareto changes. Pareto-optimal improvements, which require zero interpersonal utility comparisons (only the sign matters, not the magnitude, of each affected entity's preference), teach us nothing about actual tradeoffs, where a function must weigh the magnitudes of multiple entities' preferences against each other.

Comparing Utilities
The Pareto-optimality assumption isn't that you're "just OK" with Pareto-improvements, in a ≥ sense. The assumption is that you prefer them, ie, >.

That's not what Pareto-optimality asserts. It only talks about >= for all participants individually. If you're making assumptions about altruism, you should be clearer that it's an arbitrary aggregation function that is being increased.

And then, Pareto-optimality is a red herring. I don't know of any aggregation functions that would change a 0 to a + for a Pareto-optimal change, and would not give a + to some non-Pareto-optimal changes, which violate other agents' preferences.

My primary objection is that any given aggregation function is itself merely a preference held by the evaluator. There is no reason to believe that there is a justifiable-to-assume-in-others or automatically-agreeable aggregation function.

if you assent to Pareto improvements as something to aspire to

This may be the crux. I do not assent to that. I don't even think it's common. Pareto improvements are fine, and some of them actually improve my situation, so go for it! But in the wider sense, there are lots of non-Pareto changes that I'd pick over a Pareto subset of those changes. Pareto is a min-bar for agreement, not an optimum for any actual aggregation function.

I should probably state what function I actually use (as far as I can tell). I do not claim universality, and in fact, it's indexed based on non-replicable factors like my level of empathy for someone. I do not include their preferences (because I have no access). I don't even include my prediction of their preferences. I DO include my preferences for what (according to my beliefs) they SHOULD prefer, which in a lot of cases correlates closely enough with their actual preferences that I can pass as an altruist. I then weight my evaluation of those imputed-preferences by something like an inverse-square relationship of "empathetic distance". People closer to me (including depth and concreteness of my model for them, how much I like them, and likely many other factors I can't articulate), including imaginary and future people who I feel close to get weighted much much higher than more distant or statistical people.

I repeat - this is not normative. I deny that there exists a function which everyone "should" use. This is merely a description of what I seem to do.

The Axiological Treadmill
The obvious reason that Moloch is the enemy is that it destroys everything we value in the name of competition and survival.

Moloch is not always the enemy. Competition (among imperfectly-aligned agents) is the most efficient arbitration of different values. For almost everyone, survival is in fact something they value quite highly. Moloch happens when these pressures become so great (or are perceived as such) that they crowd out other values. Moloch destroys nothing except the illusion of freedom. Moloch creates value, just not the exact mix of value types that some or all participants would prefer.

But this is missing the bigger picture. We value what we value because, in our ancestral environment, those tended to be the things that helped us with competition and survival.

"tended to be" and "are exactly and only" are very different statements. You're saying the first, but your argument requires the second. My preferences as an individual human vary greatly from a historical human average. Even to the extent that they're mutable and change with time, I have meta-preferences about the ways in which they change, and those are ALSO different from any historical or aggregate set.

If the things that help us compete and survive end up changing, then evolution will ensure that the things we value change as well.

Not even. There's a bit of Slack even between genotype and phenotype, and a whole lot between biology and psychology.

Applying the Counterfactual Prisoner's Dilemma to Logical Uncertainty

I always enjoy convoluted Omega situations, but I don't understand how these theoretical entities get to the point where their priors are as stated (and especially the meta-priors about how they should frame the decision problem).

Before the start of the game, Omega has some prior distribution of the Agent's beliefs and update mechanisms. And the Agent has some distribution of beliefs about Omega's predictive power over situations where the Agent "feels like" it has a choice. What experiences cause Omega to update sufficiently to even offer the problem (ok, this is easy: quantum brain scan or other Star-Trek technobabble)? But what lets the Agent update to believing that their qualia of free-will is such an illusion in this case? And how do they then NOT meta-update to understand the belief-action-payout matrix well enough to take the most-profitable action?

Moved to my shortform - it's not a direct answer to the post.

[This comment is no longer endorsed by its author]Reply
Load More