To extract out the terminal values we have to inspect this mishmash of valuable things, trying to figure out which ones are getting their value from somewhere else.

What if all the supposed "terminal values" are actually instrumental values from...some more terminal value?  Sure "less violence" is more of a terminal value than "fewer guns," but is there a still more terminal value?  Less violence why?

My answer: Because I want to live in a world of happiness/flourishing. https://www.lesswrong.com/posts/DJeBQrBSqFYqBaeQW/rock-bottom-terminal-value

Main question: am I right?

More interesting follow-up: if that is the root, bottom, terminal value driving us humans, did it come from evolution, or is it simply a feature of conscious beings?

Thanks, and yes evolution is the source of many values for sure...I think the terminal vs instrumental question leads in interesting directions.  Please let me know how this sits with you!

Though I am an evolved being, none of your examples seem to be terminal values for me the whole organism.  Certainly there are many systems within me, and perhaps we could describe them as having their own terminal values, which in part come from evolution as you describe.  My metabolic system's terminal value surely has a lot to do with regulating glucose.  My reproductive system's terminal value likely involves sex/procreation. (But maybe even these can drift, like when a cell becomes cancerous, it seems its terminal value changes.)

But to me as a whole, these values (to the extent which I hold them at all) are instrumental. Sure I want homeostasis, but I want it because I want to live (another instrumental value), and I want to live because I want to be able to pursue my terminal value of happiness/flourishing. Other values that my parts exhibit (like reproduction) I the whole might reject even as an instrumental value, heck I might even subvert the mechanisms afforded by my reproductive system for my own happiness/flourishing.

Also for my terminal value for happiness/flourishing, did that come from evolution?  Did it start out as survival/reproduction and drift a bit?  Or is there something special about systems like me (which are conscious of pleasure/pain/etc) that just by their nature they desire happiness/flourishing, the way 2+2=4 or the way a triangle has 3 sides?  Or...other?

And lastly does any of this port to non-evolved beings like AIs?

Do you think the "basket of goods" (love the pun) could be looked at as instrumental values that derive from the terminal value (desiring happiness/flourishing)?

I don't understand shard theory well enough to critique it, but is there a distinction between terminal and instrumental within shard theory?  Or are these concepts incompatible with shard theory?

(Maybe some examples from the "basket of goods" would help.)

people including alignment researchers just seem more confident about their own preferred solution to metaethics, and comfortable assuming their own preferred solution is correct as part of solving other problems, like AI alignment or strategy. (E.g., moral anti-realism is true, therefore empowering humans in straightforward ways is fine as the alignment target can't be wrong about their own values.)

Obviously committed anti-realists would be right not to worry -- if they're correct!  But I agree with you, we shouldn't be overconfident in our metaethics...which makes me wonder, do you really think metaethics can be "solved?"

Secondly, even if it were solved (and to avoid the anti-realist apathy, let's assume moral realism is true), how do you think that would help with alignment?  Couldn't the alignment-target simply say, "this is true, but I don't care, as it doesn't help me achieve my goals?"  Saying "1+1=2, but I'm going to act as if it equals 3" might keep you from achieving your goal.  Saying, "stealing is wrong, but I would really like to have X" might actually help you achieve your goal.

A worthwhile cosmopolitanism should probably fight Moloch in this regard and extend its own agency to things which do not wield the tools of autopoesis themselves.

[warning: anthropomorphisation of autopoetic processes]

*autopoiesis, and autopoietic (brought to you by the pedantic spelling police)

Hi! Like others on this thread, I'm a long time reader who's finally created an account to try to join the discussion. I'm curious, if I comment on a 15 year old article or something, is anyone likely to see that? I love browsing around the Concepts pages, but are comments there (.e.g.) likely to be seen?

My intuition is that comments on recent trending articles are more likely to get engagement, but can anyone confirm or deny or give suggestions on the best ways/places to engage?

Thanks!

Imagine you're immortal. Would you rather get a dust speck in your eye for 3^^^3 straight days, or endure 50 years of torture, never to be bothered by a dust speck again?

Going with dust specks is the obvious choice to me.  The way I see it, life is still very much worth living when I'm dealing with a dust speck. Torture might make me wish for death in the first day...especially with the knowledge that 18,261 days were forthcoming.

I can’t tell if:

a) I dislike dust specks less than you

b) I dislike torture more than you

c) Aggregating pain/discomfort/suffering has weird results. (e.g. perhaps 5 minutes of torture is immeasurably more than 60x as bad as 5 seconds of torture.)

or d) there's something else I'm missing.

I think this is very relevant to the complexity of values and how difficult it is for humans to grasp their own utility functions.  Cards on the table, the aggregation/math issues make me unable to embrace consequentialism...but if I'm overlooking something I'd love to be shown the light.