(apologies for taking a couple of days to respond, work has been busy)
I think your robot example nicely demonstrates the difference between our intuitions. As cubefox pointed out in another comment, what representation you want to use depends on what you take as basic.
There are certain types of preferences/behaviours which cannot be expressed using arithmetic averaging. These are the ones which violate VNM, and I think violating VNM axioms isn't totally crazy. I think its worth exploring these VNM-violating preferences and seeing what they look like when more fleshed out. That's what I tried to do in this post.
If I wanted a robot that violated one of the VNM axioms, then I wouldn't be able to describe it by 'nailing down the averaging method to use ordinary arithmetic averaging and assigning goodness values'. For example, if there were certain states of the world which I wanted to avoid at all costs (and thus violate the continuity axiom), I could assign zero utility to it and use geometric averaging. I couldn't do this with arithmetic averaging and any finite utilities [1].
A better example is Scott Garrabrant's argument regarding abandoning the VNM axiom of independence. If I wanted to program a robot which sometimes preferred lotteries to any definite outcome, I wouldn't be able to program the robot using arithmetic averaging over goodness values.
I think that these examples show that there is at least some independence between averaging methods and utility/goodness.
(ok, I guess you could assign 'negative infinity' utility to those states if you wanted. But once you're doing stuff like that, it seems to me that geometric averaging is a much more intuitive way to describe these preferences. )
Thanks for pointing this out, I missed a word. I have added it now.
Without wishing to be facetious: how much (if any) of the post did you read? If you disagree with me, that's fine, but I feel like I'm answering questions which I already addressed in the post!
Are you arguing that we ought to (1) assign some "goodness" values to outcomes, and then (2) maximize the geometric expectation of "goodness" resulting from our actions?
I'm not arguing that we ought to maximize the geometric expectation of "goodness" resulting from our actions. I'm exploring what it might look like if we did. In the conclusion, (and indeed, many other parts of the post) I'm pretty ambivalent.
But then wouldn't any argument for (2) depend on the details of how (1) is done? For example, if "goodnesses" were logarithmic in the first place, then wouldn't you want to use arithmetic averaging?
I don't think so. I think you could have a preference ordering over 'certain' world states and the you are still left with choosing a method for deciding between lotteries where the outcome is uncertain. I describe that this is my position in the section titled 'Geometric Expectation Logarithmic Utility'.
Is there some description of how we should assign goodnesses in (1) without a kind of firm ground that VNM gives?
This is what philosophers of normative ethics do! People disagree on the how exactly to do it, but that doesn't stop them from trying! My post tries to be agnostic as to what exactly it is we care about and how we assign utility to different world states, since I'm focusing on the difference between averaging methods.
The word 'utility' can be used in two different ways: normative and descriptive.
You are describing 'utility' in the descriptive sense. I am using it in the normative sense. These are explained in the first paragraph of the Wikipedia page for 'utility'.
As I explained in the opening paragraph, I'm using the word 'utility' to mean the goodness/desirability/value of an outcome. This is normative: if an outcome is 'good' then there is the implication that you ought to pursue it.
Thanks for the comment. Naively, I agree that this sounds like a good idea, but I need to know more about it.
Do you know if anyone has explicitly written down the value learning solution to the corrigibility problem and treated it a bit more rigorously ?
Thanks for this comment-it explains your view very clearly and I understand what you are getting at now.
I think its a fair criticism. I've added footnotes within the post, linking people to your comment.
I still think it's a problem that this argument rests on the idea that investors are irrationally not renting land they own, but you don't provide any evidence for that.
I disagree. Firstly, even if, they were renting out their land, this would still be bad, for reasons described in the article (landlords extract land rent without doing anything productive etc.)
The section of the post which argues about empty homes rests on the fact that there are empty homes and a land tax would reduce them. I then provide evidence that there are, indeed, a significant number of empty homes in the UK. I do not speculate about the rationality/irrationality of the people holding them because it is irrelevant to the argument. Do you disagree on the 700,000 figure?
Is your view something like 'I find it hard to believe that people would leave houses empty because they are leaving money on the table. Therefore I disbelieve the 700,000 empty homes figure.'? If so, I guess I'm not super interested in disputing the government figures.
Or is your view something like 'The argument in the post hinges on the fact that people are irrationally leaving homes empty and the author of the post needs to explain why they aren't behaving rationally in order to make the argument work.'? If so, hopefully I explained why the argument rests on the fact that empty homes exist, not the rationality/irrationality of the people holding them.
For what its worth, I find it pretty easy to believe that people are leaving homes empty. People often behave 'irrationally' when it comes to money: a lot of people gamble and most people have their savings in a low-interest account. I know at least two middle class families who own a flat in a city where they don't live. They visit it maybe once a year and occasionally let friends visit. They aren't super interested in squeezing it for every penny its worth and don't want to sell it for sentimental reasons and they know the price will go up so they keep it as is. They think that renting it out would cause them too much stress and they don't feel that they need any more money and they enjoy visiting it so they don't rent it out . Is this irrational? Maybe from the point of view of optimizing their income, but they are just optimizing other aspects of their life.
The second link is to a Scottish political campaign that doesn't claim to know way the houses are empty (at least on this page) and doesn't contain the 700,000 number in the link text (the linked political campaign claims 46,000 in Scotland and doesn't seem to say anything about the UK).
The phrase '700,000 empty homes throughout' the UK has different links for each word: one for England, Northern Ireland, Scotland, and Wales. If you follow the link on 700,000, you will be taken to this page which gives a figure of 676,304 empty homes in England. Add this to the Scotland, Northern Ireland and Wales figures and you get a total over 700,000. As explained in the link for English data (which makes up most of the total), the figure comes from Council tax data (council taxes are paid by the owners of a property and charges different rates depending on whether the home is occupied or unoccupied).
Hope this helps!
Fine. But the purpose of exploring different averaging methods is to see whether it expands the richness of the kind of behaviour we want to describe. The point is that using arithmetic averaging is a choice which limits the kind of behaviour we can get. Maybe we want to describe behaviours which can't be described under expected utility. Having an 'avoid at all costs state' is one such behaviour which finds natural description using a non-arithmetic averaging which can't be described in more typical VNM terms.
If your position is 'I would never want to describe normative ethics using anything other than expected utility' then that's fine, but some people (like me) are interested in looking at what alternatives to expected utility might be. That's why I wrote this post. As it stands, I didn't find geometric averaging very satisfactory (as I wrote in the post), but I think things like this are worth exploring.
You are right. Geometric averaging on its own doesn't give allow violations of independence. But some other protocol for deciding over lotteries does. It's described more in the Garrabrant post linked above.