All of ExCeph's Comments + Replies

That's a fair point.  I should elaborate on the concept of stagnation, to avoid giving people the wrong impression about it.  

Stagnation is the fundamental liability defined by predictable limitations on people's motivations.  

Like the other liabilities, stagnation is also an intrinsic aspect of conscious existence as we know it.  Predictable motivations are what allow us to have identity, as individuals and as groups.  Identity and stagnation are two sides of the same coin--stagnation is just what we call it when it interferes wit... (read more)

I appreciate your questions and will do my best to clarify.  

The values from the section you quoted pertain to civilization as a whole.  You are correct that individual motivations/desires/ambitions require other concepts to describe them (see below).  I apologize for not making that clear.  The "universal values" are instrumental values in a sense, because they describe a civilization in which individuals are more able to pursue their own personal motivations (the terminal values, more or less) without getting stuck.  

In other wor... (read more)

Ah, that's where the anti-zombie shibboleths come in handy.  People who are afraid of zombies "know" that zombies can't understand the values of regular, living people.  (The zombies being a metaphor for a distorted view of one's ideological opponents.)  

All I have to do is describe why being alive is good and being a zombie is bad, and that proves I'm not a zombie.  That calms people down, to the point where we can explore some possible advantages of zombiehood and disadvantages of having vital function, and what we can do about that w... (read more)

That's a valid way to look at it.  I used to use three axes for them: increase versus decrease, experience versus influence, and average versus variance (or "quantity versus quality").  

I typically just go with the eight desires described above, which I call "motivations".  It's partially for thematic reasons, but also to emphasize that they are not mutually exclusive, even within the same context.  

It is perfectly possible to be both boldness-responsive and control-responsive: seeking to accomplish unprecedented things and expecting to... (read more)

2M. Y. Zuo2mo
It's certainly possible for people to have these conflicting desires in their mind. Though I don't see how that translates to observed desires? Since reality must obey physical principles. (Though purely internal desires are of course relevant to the person experiencing it, the desires must be demonstrable and observable for anyone else to take it into consideration, otherwise the presumption will be that it's made up.) -------------------------------------------------------------------------------- For a real world example, no amount of effort or desire can make a river go uphill and downhill simultaneously. Someone may 'seek to accomplish the unprecedented' of making the river do so and 'expect to achieve this without interference or difficulty' but it would be so unusual an activity that a prank would be the likely first guess. Even if they spent real resources on the river, it will just look like how you would expect it flowing downhill, or flowing uphill with a pumping system if they're really motivated, or stagnant if perfectly level. They could rapidly change the flow direction back and forth to try to demonstrate their desires, and simultaneously verbally claim it's effortless, easy-as-pie, etc., and that the river's really going both ways at once. But this would just look like a convoluted prank to a random observer. I'm not even sure how such a conflicting desire could be credibly demonstrated. Maybe if they are willing to take bets that the river will in fact go uphill and downhill simultaneously, and since so it's so effortless they're willing to bet their life savings, home, first born, and so on? (Though it would practically be reducing themselves to penury, since there's a 100% chance of losing the bet.) -------------------------------------------------------------------------------- For a physically possible but very unlikely and completely impractical desire, maybe someone has the desire to build a triple decker train wagon since they

That's where the deconstruction method comes in: 

  1. Make them comfortable.
  2. Make them think.
  3. Make them choose.

The first step is most important.  You don't have to start by convincing someone there are no zombies.  You just have to show them that you're not going to let any zombies get them.  Sometimes that means making small concessions by agreeing to contingencies against hypothetical zombies.  

You can tell someone that there's nothing in the dark basement, but to get them to make it five feet in to the light switch, sometimes it's most... (read more)

1Anon User2mo
Yes, unfortunately you have to figure out how to do all that when there are politicians they consider to be of their side screaming "look, zombies, zombies!!!" and the people you are trying to claim down suspect that you might also be a secret zombie...

I count eight fundamental desires, but they can take countless forms based on context.  For example, celebration might lead one person to seek out a certain type of food, while leading another person to regularly go jogging.  It's the same motivation, but manifesting for two different stimuli.  

Here are the eight fundamental desires: 

  • Celebration, the desire to bring more of something into one's experience
  • Acquisition, the desire to bring more of something into one's influence
  • Insulation, the desire to push something out of one's experienc
... (read more)
2M. Y. Zuo2mo
Seems like your eight desires are 4 fundamental desires with the possibility of increase or decrease. If there were 50 gradations, then 0 to -25 would signify desires for less, and 0 to +25 would signify desires for more.

As you say, the ability to coordinate large-scale action by decree requires a high place in a hierarchy.  With the internet, though, it doesn't take authority just to spread an idea, as long it's one that people find valuable or otherwise really like.  I'm not sure why adjacency has to be "proper"; I'm just talking about social networks, where people can be part of multiple groups and transmit ideas and opinions between them.  

Regarding value divergence: Yes, there is conflict because of how people prioritize desires and values differently. ... (read more)

1kyleherndon2mo
I don't think most people are consciously aware, but I think most people are unconsciously aware that "it is merely their priorities that are different, rather than their fundamental desires and values" and furthermore our society largely looks structured such that only the priorities are different, but that the priorities differ significantly enough because of the human-sparseness of value-space [https://www.lesswrong.com/posts/BLddiDeE6e9ePJEEu/the-village-and-the-river-monsters-or-less-fighting-more?commentId=oejit36kk5iWKCtXg] .
1kyleherndon2mo
I approximately mean something as follows: Take the vector-value model I described previously. Consider some distance metric (such as the L2 norm), D(a, b) where a and b are humans/points in value-space (or mind-space, where a mind can "reject" an idea by having it be insufficiently compatible). Let k be some threshold for communicability of a particular idea. Assume once an idea is communicated, it is communicated in full-fidelity (you can replace this with a probabilistic or imperfect communication model, but it's not necessary to illustrate my point). If you create the graph amongst all humans in value-space, where an edge exists between a and b iff D(a,b) < k, it's not clear to me that this graph is connected, or even has many edges at all. If this is true for a particular idea/k pair, then the idea is unlikely to undergo information cascade, because additional effort is needed in many locations to cross the inferential gap [https://www.lesswrong.com/tag/inferential-distance]. Somewhat related, somewhat tangential, I think the internet itself is organized hierarchically as nested "echo-chambers" or something similar where the smallest echo chambers are what we currently call echo-chambers. This means you can translate any idea/concept as existing somewhere on the hierarchy of internet communities, and only ideas high on the hierarchy can effectively spread messages/information cascades widely. Is there anywhere you can concretely point to in my model(s) you would disagree with? I agree this is (potentially) high leverage. My strategy has general been that expressing ideas with greater precision more greatly aids communication. An arbitrary conversation is unlikely to transmit the full precision of your idea, but it becomes less likely that you transmit something you don't mean and that makes a huge difference. The domain of politics seems mostly littered with extremely low precision communication, and in particular, often deceptively precise communication, w
2M. Y. Zuo2mo
I agree with your comments mostly so far. There is low-hanging fruit even in complex areas, regardless of the prevailing cynicism. There does seem to be a lot of folks who match that description. But there are also folks who understand that the world can get better yet nonetheless act like crabs in a bucket due to their desires. The latter group, when they exist in numbers past a certain threshold, likely increase the height of the fruit.

Not all human politics is low-hanging fruit, to be sure.  I was thinking of issues like the economy, healthcare, education, and the environment.  It seems like there are some obvious win-win improvements we can make in those contexts if we just shift the discussion in a constructive direction.  We can show people there are more ideas for solutions than just the ones they've been arguing about.  

It is true that the process shown in this story is not sufficient to dismantle religion.  Such an undertaking requires a constructive meta-... (read more)

4kyleherndon2mo
I disagree and will call any national or global political issues high-hanging fruit. I believe there is low-hanging fruit at the local level, but coordination problems of million or more people are hard. In my experience, it's not clear that there is really much "proper adjacency." Sufficiently high dimensional spaces make any sort of clustering ambiguous and messy if even possible. Even more specifically, I haven't seen much of any ideas in politics that spread quickly that wasn't also coordinated from (near) the top, suggesting to me that information cascades in this domain are impractical. I think that largely that's what is even meant by hierarchical structures. Small/low elements have potentially rich, complicated inner lives, but have very little signal they can send upwards/outwards. High/large structures have potentially bureaucratically or legally constrained action space, but their actions have wide and potentially large influences. Great. Keep on doing it, then. My message for everyone else [https://principiadiscordia.com/book/45.php]. Say there are 100 fundamental desires, and all desires stem from these 100 fundamental desires. Each can still take on any number from -1 to 1, allowing a person to care about each of these things in different proportions. Even if we restrict the values to 0 to 1, you still get conflict because what is most important to one person is not what's most important to another, causing real value divergences. I can think of some that you didn't explicitly mention. * You can make the world just a slightly better place by normal means, trying to be kind, etc. * You can have kids, and teach them a better way to think while they're still especially pliable, and ignore trying to teach old dogs new tricks * Maximize your inclusive genetic fitness, live a long life, make sure your ideas are such good ones that your kids will teach their kids and eventually outlive and out-compete inferior ideas *

Good questions!  

Most pro-choice people I have discussed the issue with are already on the same page about how personhood does not start at conception, and for similar reasons.  I don't usually run the the thought experiments by them to see if our reasoning processes are the same; I should do that.  I do know that some pro-choice people do think that a zygote is a "person" but that its rights do not supersede its parent's bodily autonomy, at least in the early stages.  

When pro-life people brush the thought experiments and intuition pum... (read more)

You raise a good point.  This story does not contain politicians who profit from playing factions against one another and maintaining polarization.  It might make the story a bit more applicable to our world if there were villagers who gained social influence from being the champions of each side while never engaging in negotiation or brainstorming, and who subsequently lost that power once the villagers learned how to do those things for themselves.  I may go back and add that in; thanks for the suggestion!  

As for our own world, I pre... (read more)

1Anon User2mo
But to what extent the divisions are driven by genuine desire to address the issue(s) vs just a raw "us vs them" drives (think - divisions between fandoms of rival sports teams) where the actual issues are just an excuse to think "we are better then them"? Think of a spectrum between a world with overabundant resources where trying to hoard them is stupid and "learn to be friends with everybody" is the right strategy vs a literal zombie apocalypse scenario where anybody even thinking of being friendly to the zombies endangers not only themselves, but their whole community, and hoarding is essential to survival. The reality of this world is that for quite a while resources tended to indeed become more abundant, resulting in "we are all in this together" mentality tending to win over "us vs them" one more often than not, but if somebody is truly in zombie apocalypse survival mode, there is fairly little you can do to convince them to embrace "zombies".

Do you think there would be a problem with attempting to reconcile people's values on abortion?  

You jest, but abortion is actually on my list of future Midmorning Zone articles.  The Midmorning Zone series follows discussions between two characters representing different sides of various issues.  In doing so, it demonstrates how they can use the reconciliation method to figure out constructive approaches they can collaborate on.  

Part of what makes it difficult for humans to discuss abortion is the need to detangle the cultural baggage... (read more)

5Richard_Kennaway2mo
I have to wonder how you conceive of the "personhood question" and how you are presenting it to them. Surely their answer will typically be that they ascribe full rights from conception. Intuition pumps about a foetus at various stages not being a "person" would be, to them, beside the point. You don't mention pro-choice people. Have you had more success with getting them to acknowledge the personhood question?
3kyleherndon2mo
I don't think there's many potential negative consequences in trying. My response wasn't a joke so much as taking issue with I think it really, really is not low hanging fruit. The rights and personhood line seems quite a reasonable course of discussion to go down, but you're frequently talking to people who don't want to apply reason, at least not at the level of conversation. Religion is a "reasonable choice" in that you buy a package and it's pretty solid and defended by a conglomerate with the intent that you defend and get some defense back. I don't think you're going to get far without dismantling institutions such as religions, and I don't think your process is sufficient to dismantle those institutions. Many people have effectively made the decision "you are not in my tribe, so I will not engage with you in a productive way, because I need to assume you are deceiving me." I think amongst any parties that aren't pre-opposed to one another, looking for win-wins is the default, sane thing that basically everyone does all the time. The problem is all coordination problems are downstream of effective communication, and there are many people with whom you will not communicate with effectively. The real potential negative consequence that is likely is you waste your time, and frankly, I don't think you'll be the one to solve this, because I don't think there are win-wins on this subject, and a good number of other subjects from politics.

(Made a few cosmetic tweaks to make some sentences less awkward.)  

This seems like a good analysis of how a person can use what I call the mindsets of reputation and clarification.  

Reputation mindset combines the mindsets of strategy and empathy, and it deals with fortifying impressions.  It can help one to be aware of emotional associations that others may have for things one may be planning to say or do.  That way one can avoid unintended associations where possible, or preemptively build up positive emotional associations to counteract negative ones that can't be avoided, such as by demonstrating one un... (read more)

1. Ah, now I see. Yes, removing assumptions is one good way to direct one's use of synthesis mindset. It helps with exploring the possibilities.

2. Organization can gather information efficiently, but integrating it all and catching contradictions is normally a job more suited for analysis. It's still possible to combine the two. That can end up forming strategy or something similar, or it could be viewed as using the mindsets separately to support each other.

Does that make sense?

2Pattern2y
Yes.

Thanks for the input!

1. You mean we can fiddle with the explicit assumptions we use with synthesis mindset? That can help, but to get the full benefit of synthesis I find it's often better to let go of explicit assumptions, and then apply other mindsets with those explicit assumptions to the results yielded by synthesis.

Otherwise our explicit assumptions may cause synthesis to miss hypotheses that ultimately point us in a helpful direction, even though the hypothesis itself violates the explicit assumptions. Sometimes the issue is that we make too m... (read more)

2Pattern2y
1. I haven't fully digested your framework yet. (Connecting 'synthesis', and these other mindsets, to experience.) I mean that, if you have explicit assumptions then: * They're easier to examine * they can be messed with (as a way of generating hypothesis/ideas) It's 'random' but constrained enough that it has better odds of hitting something useful, or figuring out why something is wrong improves your understanding. (Emphasis added.) I might just be describing your 'synthesis' concept. (Or something similar, with a more systematic focus.*) Math examples: * If multiplication is repeated addition, then what's repeated multiplication? Repeated powers? * What kind of space doesn't obey the triangle inequality? * If the sum of the interior angles of a shape are always the same, then squash the shape flat to find the sum. *These don't seem like problems in math. (Except with untranslated or high level hypothesis. And smooth navigation takes time to build up.) 2. I read this: "Conversely, compared to analysis, organization mindset may miss some points of mismatch between its maps and reality, and can fail to apply enough distinct checking to catch flaws in its plans." and thought if you focus on gaining information instead of some other goal, that downside might go away. Information is a funny resource, but it can be accumulated over time. And before trying to do XYZ where X, Y, and Z are simple and therefore XYZ is simple, sometimes there's the option of first doing them individually (which should be easy because they are simple in theory).

I confess, your comment surprised me by calling for a different epistemic standard than I figured this article required. I had to unpack and address several issues, listed below.

  1. I can make a bibliography from the links I’ve already included, if it would help.
  2. Are there any specific assertions in this article that you think call for more evidence to support them over the alternatives?
  3. This article is meant to build the foundation for explaining the concepts that we'll be working with in the next article. After that article, we'll mostly b
... (read more)

This post came across to me as mostly speculative but trying to be academic, I may well be wrong. Habryka in the other comment suggested that your claims have some grounding that I was not aware of. Additionally, I do not subscribe to the local lore of Eliezer's contrarianism and extreme Bayesianism. The metaphor of "reality joints", or "reality fluid", falls flat for me, as well. If you perspective is different, then feel free to disregard my comment, it's not like you and I can square our epistemic views in a comment thread.

Practice with different example problems is indeed important for helping people internalize the principles behind the skills they're learning. However, just being exposed to these problems doesn't always mean a person figures out what those principles are. Lack of understanding of the principles usually means a person finds it difficult to learn the skill and even more difficult to branch out to similar skills.

However, if we can explicitly articulate those principles in a way people can understand, such as illustrating them with analogies or ... (read more)

With finesse, it's possible to combine the techniques of truth-seeking with friendliness and empathy so that the techniques work even when the person you're talking to doesn't know them. That's a good way to demonstrate the effectiveness of truth-seeking techniques.

It's easiest to use such finesse on the individual level, but if you can identify general concepts which help you understand and create emotional safety for larger groups of people, you can scale it up. Values conversations require at least one of the parties involve... (read more)

Yes, that's exactly what I meant, and that's a great clarification. I do prefer looking at the long-term expected utility of a decision, as a sort of Epicurean ideal. (I'm still working on being able to resist the motivation of relaxation, though.)

The specific attributes I was referring to in that sentence are three out of what I call the four primary attributes:

  • Initiative (describes how much one relies on environmental conditions to prompt one to start pursuing a goal)
  • Resilience (describes how much one relies on environmental conditions to allow one to continue pursuing a goal)
  • Mobility (describes how rapidly one can effectively change the parameters of one's efforts)
  • Intensity (describes how far one can continue pushing the effects of one's efforts)

I had only been using intensity since I ... (read more)

5mako yass4y
That's a funny thing to say. The point of an agent is for it to be controlled by its motivations. But I think I know what you mean. Part of this skill is maintaining a high level overview of everything we value, never getting destructively obsessed with a few passions to the detriment of the others, yes. The hard thing about this is it really feels like the weightings of the components of the utility function change over time. If I were drunk and mad, for instance, I have to ask myself whether maybe I really do care more, in that moment, about punching that guy over there, than I care about not getting arrested. I can think the thought "but if I assault someone I'll get arrested" and go on to think "it's worth it. I have to". And maybe that's not a malfunction. Maybe that's just what humans like to be. And maybe that means I should take care to avoid ever getting into situations where I might get drunk and mad. Or maybe part of the eschatology skill is developing a stable heart, an unwavering sense of good, or a sense of some underlying unwavering good, like a Kokoimudji always knows where north is, perhaps we must learn to always see roughly where the longest term good is even when we're lost among our passions.

You raise a good point about the multiple factors that go into motivation and why it's important to address as many of them as possible.

I'm having trouble interpreting your second paragraph, though. Do you mean that humanity has a coordination problem because there is a great deal of useful work that people are not incentivized to do? Or are you using "coordination problem" in another sense?

I'm skeptical of the idea that a solution is unlikely just because people haven't found it yet. There are thousands of problems that were only solved in the past few

... (read more)

I'm painfully familiar with the issue of lack of group participation, since I can't even get people to show up to a meetup.

Because of that, I've been doing research on identifying the factors contributing towards this issue and how to possibly mitigate them. I'm not sure if any of this will be new to you, but it might spark more discussion.
These are the first ideas that come to mind:

1. For people to be intrinsically motivated to do something, the process of working on it has to be fun or fulfilling.

2. Extrinsic motivation, as yo... (read more)

Currently, Difficult Conversations is the only book I recommend to literally all people, because it establishes the principles and practices of effective collaborative truth-seeking. If you want a good chance of persuading someone of something they are already opposed to, you have demonstrate that you understand their point of view and value their well-being. (On a similar note, I read Ender's Game in middle school and took to heart the idea of understanding your adversaries so well that you love them.)

Can the art of influencing emotions be used f... (read more)

How to actually construct the AI was not part of the scope of the essay request, as I understood it. My intention was to describe some conceptual building blocks that are necessary to adequately frame the problem. For example, I address how utility functions are generated in sapient beings, including both humans and AI. Additionally, that explanation works whether or not huge paradigms shifts occur.
No amount of technical understanding is going to substitute for an understanding of why we have utility functions in the first place, and what shapes they ... (read more)

Submitting this entry for your consideration: https://www.lesserwrong.com/posts/bkoeQLTBbodpqHePd/ai-goal-alignment-entry-how-to-teach-a-computer-to-love. I'll email it as well. Your commitment to this call for ideas is much appreciated!

Based on my understanding of the wide variety of human thought, there are several basic mindsets which people use to address situations and deal with problems. Many people only use the handful that come naturally to them, and the mindsets dealing with abstract reasoning are some of the least common. Abstract reasoning requires differentiating and evaluating concepts, which are not skills most people feel the need to learn, since in most cases concepts are prepackaged for their consumption. Whether these packages represent reality in any useful way is an... (read more)

Just to add some more examples, I frequently pick up on some of the following things in casual social situations:

  • Use of textbook biases and logical fallacies
  • Reliance on "common sense" or "obviousness"
  • Failure to recognized nuanced situations (false dichotomies)
  • Failing at other minds
  • Failure to recognize diminishing marginal returns
  • Failure to draw a distinction between the following concepts:
    • Correlation and causation
    • Description and norm (is and ought)
    • Fact and interpretation
    • Necessary and sufficient
    • Entertaining an idea and accepting it

What... (read more)

1[comment deleted]5y

I've been afraid that most people lack abstract reasoning for quite some time. Thank you for describing the phenomenon so clearly. However, I also fear that you may be underestimating its biggest consequence in your life.

I strongly suspect that the biggest consequence of people lacking abstract reasoning isn't that different methods are required to explain concepts to pattern-matching people, but rather that most of the systems and institutions around you have been designed by people who have or had poor abstract reasoning skills, and that th... (read more)

5[comment deleted]5y

I strongly suspect that you cannot, with a feedback loop as you describe. If you measure discontent based on social media, suffering that does not get posted to social media effectively does not exist. The AI would need a way of somehow recognizing that the social media are only its window to the discontent that exists in the world beyond, which is what it is intended to minimize. Proverbially, it would need to be able to look at the moon rather than the finger pointing to it.

1Kazi Siddiqui5y
Thanks. As far as I can tell, this is symptomatic of a general class of problems with trying to gauge what humans want. If you want to maximize fun, you need a model of what leads to fun. You must somehow gather this data from humans. The only foolproof way to do this is to recreate the human mind in the computer, which brings its own problems. Is there a way out I'm not seeing?

I would argue that for larger, more complex projects, it seems crucial to have basic proficiency in supporting skills as well as the core skills. It is not uncommon for a person with extreme skill in one area to fail or experience diminishing marginal returns on their skill, because it is necessary but not sufficient to succeed in their goal.

Between a person with core skills for a project and one with supporting skills, the person with core skills will get better results. However, between a person with core skills and one with core and supporting skill... (read more)

Apologies in advance for the long response. Hopefully this will be worth the read.

I greatly appreciate your post because it challenges some of my own beliefs and made me reassess them. I agree that a person can get by in this world with bad epistemological hygiene. However, humans are animals that evolved to adapt behaviors for many environments. Getting by is cheap. The problem with poor epistemological hygiene (EH) isn't that a person can't get by. As I see it, there are three issues:

  1. The worse your EH is, the more your success is based
... (read more)
1zlrth5y
I like how you said this. In social situations, I've been trying to find a delicate and concise way to get across that, "'Everyone has their own truth' is not an experience-constraining belief. Saying it is a marker of empathy--good for you (seriously!). But if I wanted to falsify that belief, I wouldn't know where to begin. What trade-offs do you think you're making by saying, 'Everyone has their own truth'?" "Everyone has their own truth" is just one example of these kinds of applause-lights-y nonbeliefs. I say them too when I'm trying to signal empathy, and not much else. Because they live as if none of these possibilities exist (i.e. their experiences are constrained), couldn't you say that for some definition of "believe," they don't actually believe in poltergeists? They're committing a minor sin by saying out loud that they believe in poltergeists, while not living as though they do. That said, I'd still say that aligning your stated beliefs with how you behave is admirable and effective.