by lc
1 min read

2

This is a special post for quick takes by lc. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Shortform
457 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings
[-]lc702

It is both absurd, and intolerably infuriating, just how many people on this forum think it's acceptable to claim they have figured out how qualia/consciousness works, and also not explain how one would go about making my laptop experience an emotion like 'nostalgia', or present their framework for enumerating the set of all possible qualitative experiences[1]. When it comes to this particular subject, rationalists are like crackpot physicists with a pet theory of everything, except rationalists go "Huh? Gravity?" when you ask them to explain how their theory predicts gravity, and then start arguing with you about gravity needing to be something explained by a theory of everything. You people make me want to punch my drywall sometimes.

For the record: the purpose of having a "theory of consciousness" is so it can tell us which blobs of matter feel particular things under which specific circumstances, and teach others how to make new blobs of matter that feel particular things. Down to the level of having a field of AI anaesthesiology. If your theory of consciousness does not do this, perhaps because the sum total of your brilliant insights are "systems feel 'things' when they're, y'... (read more)

or present their framework for enumerating the set of all possible qualitative experiences (Including the ones not experienced by humans naturally, and/or only accessible via narcotics, and/or involve senses humans do not have or have just happened not to be produced in the animal kingdom)

Strongly agree. If you want to explain qualia, explain how to create experiences, explain how each experience relates to all other experiences.

I think Eliezer should've talked more about this in The Fun Theory Sequence. Because properties of qualia is a more fundamental topic than "fun".

And I believe that knowledge about qualia may be one of the most fundamental types of knowledge. I.e. potentially more fundamental than math and physics.

[-]lc253

I think Eliezer should've talked more about this in The Fun Theory Sequence. Because properties of qualia is a more fundamental topic than "fun".

I think Eliezer just straight up tends not to acknowledge that people sometimes genuinely care about their internal experiences, independent of the outside world, terminally. Certainly, there are people who care about things that are not that, but Eliezer often writes as if people can't care about the qualia - that they must value video games or science instead of the pleasure derived from video games or science.

His theory of fun is thus mostly a description of how to build a utopia for humans who find it unacceptable to "cheat" by using subdermal space heroin implants. That's valuable for him and people like him, but if aligned AGI gets here I will just tell it to reconfigure my brain not to feel bored, instead of trying to reconfigure the entire universe in an attempt to make monkey brain compatible with it. I sorta consider that preference a lucky fact about myself, which will allow me to experience significantly more positive and exotic emotions throughout the far future, if it goes well, than the people who insist they must only feel satisfied after literally eating hamburgers or reading jokes they haven't read before.

This is probably part of why I feel more urgency in getting an actually useful theory of qualitative experience than most LW users.

3Shiroe
Utilitarianism seems to demand such a theory of qualitative experience, but this requires affirming the reality of first-person experience. Apparently, some people here would rather stick their hand on a hot stove than be accused of "dualism" (whatever that means) and will assure you that their sensation of burning is an illusion. Their solution is to change the evidence to fit the theory.
4lc
It does if you're one of the Cool People like me who wants to optimize their qualitative experience, but you can build systems that optimize some other utility target. So this isn't really quite true. This is true.
1Q Home
I'm interested in qualia for different reasons: 1. For me personalities of other people are an important type of qualia. I don't consider knowing someone's personality to be a simple knowledge like "mitochondria is the powerhouse of the cell". So, valuing other people makes me interested in qualia more. 2. I'm interested in knowing properties of qualia (such as ways to enumerate qualia), not necessarily using them for "cheating" or anything. I.e. I'm interested in the knowledge itself.
3lc
Personalities aren't really qualia as I'm defining them. They're an aggregation of a lot of information about people's behavior/preferences. Qualia is things people feel/experience.
2Q Home
Would you consider the meaning of a word (at least in a specific context) to be qualia? For me personalities are more or less holistic experiences, not (only) "models" of people or lists of arbitrary facts about a person. I mean, some sort of qualia should be associated with those "models"/facts anyway? People who experience synesthesia may experience specific qualia related to people. Maybe it's wishful thinking, but I think it would be cool if awareness about other conscious beings was important for conscious experience.
5Dagon
Seems weird for your blob of matter to react so emotionally to the sounds or shapes that some blobs have emitted bout other blobs.  Why would you expect anyone to have a coherent theory of something they can't even define and measure? It seems even weirder for you to take such reporting at face value about having any relation to a given blob's "inner life", as opposed to a variance in the the evolved and learned verbal and nonverbal signaling that such behaviors actually are.
[-]TAG100

Why would you expect anyone to have a coherent theory of something they can’t even define and measure?

Because they say so. The problem then is why they think they have a coherent theory of something they can't define or measure.

6lc
Just the way I am bro I expect people who say they have a coherent theory of something to be able to answer any relevant questions at all about that something. Are you referring the NYPost link? I think people's verbal and nonverbal signaling has some relationship with their inner experience. I don't think this woman is forgoing anaesthetic during surgeries because of pathologies. But if you disagree, then fine: How do we modify people to have the inner life that that woman is ~pretending to have?
3Dagon
Probably should have included a smiley in my comment, but I do want to point out that it's reasonable to model people (and animals and maybe rocks) as having highly variant and opaque "inner lives" that bear only a middling correlation to their observable behaviors, and especially to their public behaviors. For the article on the woman who doesn't experience pain, I have pretty high credence that there is some truth to her statements, but much lower credence that it maps as simply as presented to "natural stoicism" as presented in the article.  And really no clue on "what it's like" to live that experience, whether it's less intense and interesting in all dimensions, or just mutes the worst of it, or is ... alien. And since I have no clue how to view or measure an inner life, I have even less understanding of how or whether to manipulate it.  I strongly suspect we could make many people have an outer life (which includes talking about one's inner life) more like the one given, with the right mix of drugs, genetic meddling, and repeated early reinforcement of expectations.
2lc
Agreed, basically. That's part of why we need the theory!
0qvalq
4dr_s
On this forum, or literally everywhere? Because for example I keep seeing people arguing with absolute conviction, even in academic papers, that current AIs and computers can't possibly be conscious and I can't figure out how they could ever know that of something that is fundamentally unfalsifiable. I envy their secret knowledge of the world gained by revelation, I guess!
1Towards_Keeperhood
Huh, interesting. Could you make some examples for what people seem to claim this, and if Eliezer is among them, where he seems to claim this? (Would just interest me.)
1Dan
Attentional Schema Theory. That's the convincing one. But still very rudimentary.  But you know if something is poorly understood. The guy who thought it up has a section in his book on how to make a computer have conscious experiences.  But any theory is incomplete as the brain is not well understood. I don't think you can expect a fully formed theory right off the bat, with complete instructions for making a feeling thinking conscious We aren't there yet.
6lc
I'm actually cool with proposing incomplete theories. I'm just annoyed with people declaring the problem solved via appeals to "reductionism" or something, without even suggesting that they've thought about answering these questions.
[-]lc508

The Prediction Market Discord Message, by Eva_:

Current market structures can't bill people for the information value that went into the market fairly, can't fairly handle secret information known to only some bidders, pays out most of the subsidy to whoever corrects the naive bidder fastest even though there's no benefit to making it a race, offers almost no profit to people trying to defend the true price from incorrect bidders unless they let the price shift substantially first, can't be effectively used to collate information known by different bidders, can't handle counterfactuals / policy conditionals cleanly, implement EDT instead of LDT, let you play games of tricking other bidders for profit and so require everyone to play trading strategies that are inexploitable even if less beneficial, can't defend against people who are intentionally illegible as to whether they have private information or are manipulating the market for profit elsewhere...

But most of all, prediction markets contain supposedly ideal economic actors who don't really suspect each other of dishonesty betting money against each other even though it's a net-zero trade and the aggreeement theorem says th

... (read more)
[-]Raemon126

I’m pretty interested in this as an exercise of ‘okay yep a bunch of those problems seem real. Can we make conceptual or mechanism-design progress on them in like an afternoon of thought?’

5lc
I'm interested too. I think several of the above are solvable issues. AFAICT: Solved by simple modifications to markets: * Races to correct naive bidders * Defending the true price from incorrect bidders for $ w/o letting price shift Seem doable with thought: * Billing for information value * Policy conditionals Seem hard/idk if it's possible to fully solve: * Collating information known by different bidders * Preventing tricking other bidders for profit * General enterprise of credit allocation for knowledge creation
3interstice
Good post, it's underappreciated that a society of ideally rational people wouldn't have unsubsidized, real-money prediction markets. Of course in real prediction markets this is exactly what we see. Maybe you could think of PMs as they exist not as something that would exist in an equilibrium of ideally rational agents, but as a method of moving our society closer to such an equilibrium, subsidized by the bets of systematically irrational people. It's not a perfect such method, but does have the advantage of simplicity. How many of these issues could be solved by subsidizing markets? What discord is this, sounds cool.
[-]lc38-5

[Redacted]

<3. Thanks for letting us know.

2mukashi
That's very brave
6Evan R. Murphy
What happened to lc? They contributed to some good discussions on here, but look to have suddenly disappeared.
[-]lc360

To the LW devs - just want to mention that this website is probably now the most well designed forum I have ever used. The UX is almost addictively good and I've been loving all of the little improvements over the past year or so.

9Yitz
Ditto here; kudos to everyone involved in creating such an excellent forum design!
4RHollerith
I find LW.com hard to use (because I have yet to find a way to disable the mouseovers, which quickly deplete my orienting response, about which I can explain more if asked) but LW is better than most sites in that alternative interfaces can be created. In particular, I use greaterwrong.com as my interface and am pretty satisfied with it (though it was slow for a lot of the last 2 months). But I strongly upvoted parent because it is good reminder to me of the cognitive diversity in the human population.
4EniScien
I wrote somewhere that this is the only forum that looks to me the result of Intelligent Design, and not an accident. It's the only one that looks like I AM trying to intelligently design the forum MYSELF, including going back in time after discovering problems and fixing them (or just thinking in advance for five minutes on each aspect "how can I hack this / what are the vulnerabilities of this system of rules /how trolls can use it"). The point is not only that, unlike many other sites, I don’t think every five minutes “why can’t X be here”, the point is that I look somewhere and see in advance that something is provided that I don’t I had time to think, and some kind of protection was made against the exploitation of vulnerabilities in the system of rules or even involuntary errors in human psychology.
3lc
I agree completely. The last N weeks or so there have been performance problems, but all of the little things... Version history on posts, strong upvotes/downvotes, restoration of comments... They make writing things just fun.
2EniScien
If we still talk about shortcomings, then I would still be able to name 4, I wrote about the first two in the questions: lack of arrows between sequences; useless SEQ RERUNs for the sake of comments and problems with missing nested answers to questions in old comments; lately performance problems (which turned out to be lesswrong problems, not mine, so I didn't count them before); the fact that from time to time they vote for you once in the minus for completely incomprehensible reasons and then this value does not return to the plus (but as far as I understand, setting the need to indicate the reason for a bad vote will be either harmful or not very useful measure). But considering that for all this time I have found the number of minuses that can be counted on the fingers of one hand, while on any other site they literally pour from every element every second like from a cornucopia and instead of eliminating them, monthly useless graphic updates are made .. All in all, this is just a surprisingly good result, although (there is no limit to perfection) I hope at least three of them will be fixed in the coming months (how about the last one I do not know and there seems to be reasons why it is not fixed. But just in case I I note this minus, otherwise, as in the case of glitches, it turns out recently that everyone simply did not report it).
[-]lc3514

Pretty much ~everybody on the internet I can find talking about the issue both mischaracterizes and exaggerates the extent of child sex work inside the United States, often to a patently absurd degree. Wikipedia alone reports that there are anywhere from "100,000-1,000,000" child prostitutes in the U.S. There are only ~75 million children in the U.S., so I guess Wikipedia thinks it's possible that more than 1% of people aged 0-17 are prostitutes. As in most cases, these numbers are sourced from "anti sex trafficking" organizations that, as far as I can tell, completely make them up.

Actual child sex workers - the kind that get arrested, because people don't like child prostitution - are mostly children who pass themselves off as adults in order to make money. Part of the confusion comes from the fact that the government classifies any instance of child prostitution as human trafficking, regardless of whether or not there's evidence the child was coerced. Thus, when the Department of Justice reports that federal law enforcement investigated "2,515 instances of suspected human trafficking" from 2008-2010, and that "forty percent involved prostitution of a child or child sexual exploit... (read more)

They do the same thing with "child pornography": that's mostly teenagers sexting. And a girl was convicted for it too, charged with distributing child pornography of herself: link.

6ZY
Haven't looked too closely at this, but my initial two thoughts: 1. child consent is tricky. 2. likely many are foreign children, which may or may not be in the 75 million statistic It is good to think critically, but I think it would be beneficial to present more evidence before making the claim or conclusion
5lemonhope
The other day I was trying to think of information leaks that a competent conspiracy couldn't prevent, regarding this. I just thought of one small one: people will sometimes randomly die or have their homes raided. If the slavery is common, then sometimes the slaves will be discovered during these events. Even if the escapees wanted to silence the story out of shame, cops would probably gossip to the press. So you can probably tally such events, crunch the numbers, and get a decent conspiracy-resistant estimate.
2ChristianKl
Epstein seemed to run something like an organized child rape cabal and most people involved in it didn't go to prison.
9lc
Epstein was an amateur rapist, not a pro rapist. His cabal - the parts of it that are actually confirmed and not just speculated about baselessly - seems extremely limited in scope compared to the kinds of industrial conspiracies that people propose about child sex work. Most of epstein's victims only ever had sex with Epstein, and only one of them - Virginia Giuffre - ever appears to have publicly claimed being passed around to many of Epstein's friends. What I am referring to are claims an underworld industry for exploiting children the primary purpose of which is making money. For example, in the Sound of Freedom, a large part of the plot hinges on the idea that there are professionals who literally bring kidnapped children from South America into the United States so that pedophiles here can have sex with them. I submit that this industry in particular does not exist, or at least would be a terrible way to make money on a risk-adjusted basis compared to drug dealing.
2ChristianKl
Wikipedia currently writes "Epstein installed concealed cameras in numerous places on his properties to allegedly record sexual activity with underage girls of prominent people for criminal purposes such as blackmail." This suggests that more than one underage girl was passed around. The fact that the others don't have the same courage as Virginia Giuffre to speak publically about it, does not mean that there wasn't a problem.  It does show that the system to suppress information about it is working well, which is also shown by the fact that the FBI did not lose that video stack as material for prosecuting all those people.  The blackmail potential existing matters in addition to being directly paid. J. Edgar Hoover could be effectively blackmailed by the mafia into saying that there's no mafia by having photos of his homosexual activities, but these days that wouldn't be enough to blackmail anybody.  Understanding how organizations like Epsteins operate is hard because they do everything they can to avoid being well understood. As far as industry goes, it's quite old but https://wikileaks.org/wiki/an_insight_into_child_porn is a good read about how child pornography worked two decades ago. It's not the same as child prostitution but it's a good article about ground realities. 
[-]lc350

The Nick Bostrom fiasco is instructive: never make public apologies to an outrage machine. If Nick had just ignored whoever it was trying to blackmail him, it would have been on them to assert the importance of a twenty-five year old deliberately provocative email, and things might not have ascended to the point of mild drama. When he tried to "get ahead of things" by issuing an apology, he ceded that the email was in fact socially significant despite its age, and that he did in fact have something to apologize for, and so opened himself up to the Standard Replies that the apology is not genuine, he's secretly evil etc. etc.

Instead, if you are ever put in this situation, just say nothing. Don't try to defend yourself. Definitely don't volunteer for a struggle session.

Treat outrage artists like the police. You do not prevent the police from filing charges against you by driving to the station and attempting to "explain yourself" to detectives, or by writing and publishing a letter explaining how sorry you are. At best you will inflate the airtime of the controversy by responding to it, at worst you'll be creating the controversy in the first place.

7Viliam
Do not assume good faith on Twitter. Ever. Not because all people online are bad, but because Twitter is a "dark forest". If there are 999 good people and 1 bad person, it's the bad person who will take your tweet, maybe modify it a little, put it into most outrageous possible context, write an article about why you are the worst person ever, and share it on all social networks. And that's the lucky case. In the unlucky case, the story will uncritically be accepted by journalists, then added to Wikipedia, you will get fired, and for the rest of your life, random people on the street will keep yelling at you. Twitter should be legally required to show you this as a warning every time you are making a tweet. EDIT: This was written before I learned the details. Now the analogy with not talking to police seems even better: indeed, every word you say is a potential new incriminating evidence against you (and if it is not, it will simply be ignored), and the worst outcome is that the new evidence will hurt you in a way the old evidence could not. Question: If I get in trouble with the police, I know I need to find a lawyer. If I get in trouble with an internet mob, and I understand the need to defer to a more experienced person's advice to navigate the minefield, and I am willing to pay them, whose services exactly should I find? Is there an obvious answer, such as "lawyer" in case of legal trouble?
7ChristianKl
The professional class would be PR people.  A vaguely remember reading that the firm that handled Biden's sexual assault allegations also did good work for other people.
4lc
I actually thought of this extension and cut it from the original post, but, if you need to defend yourself and have simple exonerating evidence, one way might be to find a friend willing to state your reservations without referring to the fact that they've spoken to you or you're feeding them information. This way they can present your side of the story without giving it extra fuel, lending significance to the charges, or directly quoting you with statements you can be hanged for by the Twitter mob. However, this may also just extend the half life of the controversy.
4DragonGod
Very strongly agree and endorse this message. I'm not above giving into incentives, and if the incentives are such that you should not apologise for wrongdoing, then so be it.
2Dagon
Does this generalize to "just ignore Twitter (and other blathering by "the masses") for most things"?  Outside of a pretty small group, I haven't heard much handwringing, condemnation nor defense of Bostrom's old messages or his recent apology.   I personally think that personal honor is better supported by a thoughtful apology when something is brought to one's attention, than by simply ignoring it.  Don't engage in a back-and-forth, and don't expect the apology to convince the more vocal part of the 'verse.  But do be honest and forthright with yourself and those who you respect enough to value their opinions. From what I can tell (and I haven't looked that deeply, as I don't particularly care), Professor Bostrom has done this pretty well, and I don't expect him to suffer much long-term harm from his early mistakes.
-1Noosphere89
IMO I disagree with the implication that Nick Bostrom shouldn't have apologized, since for once the Twitter machine is actually right to criticize the apology. From titotal's post on why Bostrom's apology isn't good, there are several tests that he failed at: Link below: https://forum.effectivealtruism.org/posts/KB8XPfh7dJ9uJaaDs/does-ea-understand-how-to-apologize-for-things Disclaimer: This is a rare action for me to take, and just because I think the Twitter sphere is somewhat right does not equal that any of their conclusions are automatically right, nor does this mean I will care much about what Twitter thinks.
[-]lc350

The problem with trade agreements as a tool for maintaining peace is that they only provide an intellectual and economic reason for maintaining good relations between countries, not an emotional once. People's opinions on war rarely stem from economic self interest. Policymakers know about the benefits and (sometimes) take them into account, but important trade doesn't make regular Americans grateful to the Chinese for providing them with so many cheap goods - much the opposite, in fact. The number of people who end up interacting with Chinese people or intuitively understanding the benefits firsthand as a result of expanded business opportunities is very small.

On the other hand, video games, social media, and the internet have probably done more to make Americans feel aligned with the other NATO countries than any trade agreement ever. The YouTubers and Twitch streamers I have pseudosocial relationships with are something like 35% Europeans. I thought Canadians spoke Canadian and Canada was basically some big hippie commune right up until my minecraft server got populated with them. In some weird alternate universe where people are suggesting we invade Canada, my first instinctual... (read more)

2Dagon
This doesn't seem like an either-or question.  Freer trade and more individual interactions seem complementary to me.
5lc
I should note that I'm also pro free trade, because I like money and helping people. I'm just not pro free trade because I think it promotes peace.
[-]lc302

The "people are wonderful" bias is so pernicious and widespread I've never actually seen it articulated in detail or argued for. I think most people greatly underestimate the size of this bias, and assume opinions either way are a form of mind-projection fallacy on the part of nice/evil people. In fact, it looks to me like this skew is the deeper origin of a lot of other biases, including the just-world fallacy, and the cause of a lot of default contentment with a lot of our institutions of science, government, etc. You could call it a meta-bias that causes the Hansonian stuff to go largely unnoticed.

I would be willing to pay someone to help draft a LessWrong post for me about this; I think it's important but my writing skills are lacking.

6Dagon
I haven't seen it articulated, or even mentioned.  What is it?  It sounds like this is just the common amnesia (or denial) of the rampant hypocrisy in most humans, but I've not heard that phrasing.   would it be fair to replace the first "are" (and maybe the second) with something that doesn't imply essentialism or identity?  "people are assumed to be" or "people claim to be" followed by "more altruistic than their behavior exhibits"?
9lc
The most salient example of the bias I can think of comes from reading interviews/books about the people who worked in the extermination camps in the holocaust. In my personal opinion, all the evidence points to them being literally normal people, representative of the average police officer or civil service member pre-1931. Holocaust historians nevertheless typically try very hard to outline some way in which Franz Stangl and crew were specially selected for lack of empathy, instead of raising the more obvious hypothesis that the median person is just not that upset by murdering strangers in a mildly indirected way, because the wonderful-humans bias demands a different conclusion. This goes double in general for the entire public conception of killing as the most evil-feeling thing that humans can do, contrasted with actual memoirs of soldiers and the like who typically state that they were surprised how little they cared compared to the time they lied to their grandmother or whatever.
6Dagon
I may have the same bias, and may in fact believe it's not a bias.  People are highly mutable and contextual in how they perceive others, especially strangers, especially when they're framed as outgroup. The fact that a LOT of people could be killers and torturers in the right (or very wrong) circumstances doesn't seem surprising to me, and this doesn't contradict my beliefs that many or perhaps most do genuinely care about others with a better framing and circumstances. There is certainly a selection effect, likewise for modern criminal-related work, that people with the ability to frame "otherness" and some individual-power drive, tend to be drawn to it.  There are certainly lots of Germans who did not participate in those crimes, and lots of current humans who prefer to ignore the question of what violence is used against various subgroups*. But there's also a large dollop of "humans aren't automatically ANYTHING".  They're far more complex and reactive than a simple view can encompass.  * OH!  that's a bias that's insanely common.  I said "violence against subgroups" rather than "violence by individuals against individuals, motivated by membership and identification with different subgroups".  
3Adam Zerner
Yeah, I echo this. I've gone back and forth with myself about this sort of stuff. Are humans altruistic? Good? Evil? On the one hand, yes, I think lc is right about how in some situations people exhibit just an extraordinary amount of altruism and sympathy. But on the other hand, there are other situations where people do the opposite: they'll, I dunno, jump into a lake at a risk to their own life to save a drowning stranger. Or risk their lives running into a burning building to save strangers (lots of volunteers did this during 9/11). I think the explanation is what Dagon is saying about how mutable and context-dependent people are. In some situations people will act extremely altruistically. In others they'll act extremely selfishly. The way that I like to think about this is in terms of "moral weight". How many utilons to John Doe would it take for you to give up one utilon of your own? Like, would you trade 1 utilon of your own so that John Doe can get 100,000 utilons? 1,000? 100? 10? Answering these questions, you can come up with "moral weights" to assign to different types of people. But I think that people don't really assign a moral weight and then act consistently. In some situations they'll act as if their answer to my previous question is 100,000, and in other situations they'll act like it's 0.00001.
4Dagon
My model of utility (and the standard one, as far as I can tell) doesn't work that way.  No rational agent ever gives up a utilon - that is the thing they are maximizing.  I think of it as "how many utilons do you get from thinking about John Doe's increased satisfaction (not utilons, as you have no access to his, though you could say "inferred utilons") compared to the direct utilons you would otherwise get". Those moral weights are "just" terms in your utility function. And, since humans aren't actually rational, and don't have consistent utility functions, actions that imply moral weights are highly variable and contextual.
2Adam Zerner
Ah yeah, that makes sense. I guess utility isn't really the right term to use here.
4Garrett Baker
Recommendations for such memoirs?
2jmh
Not really memoirs but a German documentary about WWII might be of interest for you. Der unbekannte Soldat I watched on Amazon Prime and you can still find the title there in a search, not sure if it is only available for rent/sale now or if you can stream with Prime membership.
4trevor
I'm not sure to what extent this is helpful, or if it's an example of the dynamic you're refuting, but Duncan Sabien recently wrote a post that intersects with this topic: Where it connects is that if someone sees [making the world a better place] like simply selecting a better Nash Equilibria, they absolutely will spend time exploring solutionspace/thinking through strategies similar to Goal Factoring or Babble and Prune. Lots of people throughout history have yearned for a better world in a lot of different ways, with varying awareness of the math behind Nash Equilibira, or the transhumanist and rationalist perspectives on civilization (e.g. map & territory & biases & scope insensitivity for rationalism, cryonics/anti-aging for transhumanism). But their goal here is largely steering culture away from nihilism (since culture is a Nash Equilibria) which means steering many people away from themselves, or at least the selves that they would have been. Maybe that's pretty minor in this case e.g. because feeling moderate amounts of empathy and living in a better society are both fun, but either way, changing a society requires changing people, and thinking really creatively about ways to change people tears down lots of chesterton-Schelling fences and it's very easy to make really big damaging mistakes in the process (because you need to successfully predict and avoid all mistakes as part of the competent pruning process, and actually measurably consistently succeeding at this is thinkoomph not just creative intelligence). Add in conflict theory to the mistake theory I've described here, factor in unevenly distributed intelligence and wealth in addition to unevenly distributed traits like empathy and ambition and suspicion-towards-outgroup (e.g. different combinations of all 5 variables), and you can imagine how conflict and resentment would accumulate on both sides over the course of generations. There's tons of examples in addition to Ayn Rand and Wokeness.
3ChristianKl
It's worth separating what people actually believe about how altruistic other people are from what they pretend to believe about the altruism of other people.  If you ask someone whether they believe that there's a chance that their partner would cheat on them, they are most likely to tell you that their partner would cheat on them. The same person might take a few signs that point in the direction of their partner cheating as a huge problem. I would also expect that beliefs differ a lot between people.
3Adam Zerner
I'm not looking to write a post about this, but I'd be happy to go back and forth with you in the comments about it (no payment required). Maybe that back and forth will help you formulate your thoughts. For starters, I'm not sure if I understand the bias that you are trying to point to. Is it that people assume others are more altruistic than they actually are? Do any examples come to your mind other than this?
3Jacob G-W
Related: Saving the world sucks People accept that being altruistic is good before actually thinking if they want to do it. And they also choose weird axioms for being altruistic that their intuitions may or may not agree with (like valuing the life of someone in the future the same amount of someone today).
1cubefox
"The x are more y than they actually are" seems like a contradiction?
3lc
Rewrote to be more clear.
[-]lc300

So apparently in 2015 Sam Altman said:

“AI will probably most likely lead to the end of the world, but in the meantime, there’ll be great companies.”

Serious question: Is he a comic book supervillain? Is this world actually real? Why does this quote not garner an emotive reaction out of anybody but me?

I was surprised by this quote. On following the link, the sentence by itself seems noticeably out of context; here's the next part:

On the growing artificial intelligence market: “AI will probably most likely lead to the end of the world, but in the meantime, there’ll be great companies.” 

On what Altman would do if he were President Obama: “If I were Barack Obama, I would commit maybe $100 billion to R&D of AI safety initiatives.” Altman also shared that he recently invested in a company doing "AI safety research" to investigate the potential risks of artificial intelligence.

[-]lc260

PSA: I have realized very recently after extensive interactive online discussion with rationalists, that they are exceptionally good at arguing. Too good. Probably there's some inadvertent pre- or post- selection for skill at debating high concept stuff going on.

Wait a bit until acceding to their position in a live discussion with them where you start by disagreeing strongly for maybe intuitive reasons and then suddenly find the ground shifting beneath your feet. It took me repeated interactions where I only later realized I'd been hoodwinked by faulty reasoning to notice the pattern.

I think in general believing something before you have intuition around it is unreliable or vulnerable to manipulation, even if there seems to be a good System 2 reason to do so. Such intuition is specialized common sense, and stepping outside common sense is stepping outside your goodhart scope where ability to reliably reason might break down.

So it doesn't matter who you are arguing with, don't believe something unless you understand it intuitively. Usually believing things is unnecessary regardless, it's sufficient to understand them to make conclusions and learn more without committing to belief. And certainly it's often useful to make decisions without committing to believe the premises on which the decisions rest, because some decisions don't wait on the ratchet of epistemic rationality.

5the gears to ascension
I'm on board with this. it's a common failure of reasoning in this community and humanity in general imo - people believing each other too early because of confident sounding reasoning. I've learned to tell people I'll get back to them after a few nights' sleep when someone asks me what my update is about a heavily philosophical topic.
6Vladimir_Nesov
That's a tricky thing: the method advocated in the Sequences is lightness of belief, which helps in changing your mind but also dismantles the immune system against nonsense, betting that with sufficient overall rationality training this gives a better equilibrium. I think aiming for a single equilibrium is still inefficient use of capabilities and limitations of human mind, and it's better to instead develop multiple segregated worldviews (something the Sequences explicitly argue against). Multiple worldviews are useful precisely to make the virtue of lightness harmless, encouraging swift change in details of a relevant worldview or formation of a new worldview if none account for new evidence. In the capacity of paradigms, some worldviews might even fail to recognize some forms of evidence as meaningful. This gives worldviews opportunity to grow, to develop their own voice with full support of intuitive understanding expected in a zealot, without giving them any influence over your decisions or beliefs. Then, stepping back, some of them turn out to have a point, even if the original equilibrium of belief would've laughed their premises out of consideration before they had a chance of conveying their more nuanced non-strawman nature.
5Viliam
I feel like "what other people are telling me" is a very special type of evidence that needs to be handled with extra care. It is something that was generated by a potentially adversarial intelligence, so I need to check for some possible angles of attack first. This generally doesn't need to be done with evidence that is just randomly thrown at me by the universe, or which I get as a result of my experiments. The difference is, basically, that the universe is only giving me the data, but a human is simultaneusly giving me the data (potentially filtered or falsified) and also some advice how to think about the data (potentially epistemically wrong). Furthermore, there is a difference between "what I know" and "what I am aware of at this very moment". There may be some problem with what the other person is telling me, but I may not necessarily notice it immediately. Especially when the person is drawing my attention away from that on purpose. So even if I do not see any problem with what that person said right now, I might notice a few problems after I sleep on it. My own mind has all kinds of biases; how I evaluate someone's words is colored by their perceived status, whether I feel threatened by them, etc. That is a reason to rethink the issue later when the person is not here. In other words, if someone tells me a complex argument "A, therefore B, therefore C, therefore D, therefore you should give me all your money; in the name of Yudkowsky be a good rationalist and update immediately", I am pretty sure that the rational reaction is to ignore them and take as much time as I need to rethink the issue alone or maybe with other people whom I trust.
2Vladimir_Nesov
By worldviews I mean more than specialized expertise where you don't yet have the tools to get your head around how something unfamiliar works (like how someone new manipulates you, how to anticipate and counter this particular way of filtering of evidence). These could instead be unusual and currently unmotivated ways of looking at something familiar (how an old friend or your favorite trustworthy media source or historical truths you've known since childhood might be manipulating you; how a "crazy" person has a point). The advantage is in removing the false dichotomy between keeping your current worldview and changing it towards a different worldview. By developing them separately, you take your time becoming competent in both, and don't need to hesitate in being serious about engaging with a strange worldview on its own terms just because you don't agree with it. But sure, getting more intuitively comfortable with something currently unfamiliar (and potentially dangerous) is a special case.
3the gears to ascension
while I definitely see your argument, something about this seems weird to me and doesn't feel likely to work properly. my intuition is that you just have one mashed worldview with inconsistent edges; while that's not necessarily terrible or anything, and keeping multiple possible worldviews in mind is probably good, my sense is that "full support [as] expected in a zealot" is unhealthy for anyone. something or other overoptimization? I do agree multiple worldviews discussing is an important thing in improving the sanity waterline.
3Vladimir_Nesov
It is weird in the sense that there is no widespread practice. The zealot thing is about taking beliefs-within-a-worldview (that are not your beliefs) seriously, biting the bullet, which is important for naturally developing any given worldview the way a believer in it would, not ignoring System 2 implications that challenge and refine preexisting intuition, making inferences according to its own principles and not your principles. Clearly even if you try you'll fail badly at this, but you'll fail even worse if you don't try. With practice in a given worldview, this gets easier, an alien worldview obtains its own peculiar internal common sense, a necessary aspect of human understanding. The named/distinct large worldviews is an oversimplification, mainly because it's good to allow any strange claim or framing to have a chance of spinning up a new worldview around itself if none would take it as their own, and to merge worldviews as they develop enough machinery to become mutually intelligible. The simplification is sufficient to illustrate points such as a possibility of having contradictory "beliefs" about the same claim, or claims not being meaningful/relevant in some worldviews when they are in others, or taking seriously claims that would be clearly dangerous or silly to accept, or learning claims whose very meaning and not just veracity is extremely unclear. Studying math looks like another important example, with understanding of different topics corresponding to worldviews where/while they remain sparsely connected, perhaps in want of an application to formulating something that is not yet math and might potentially admit many kinds of useful models. Less risk of wasting attention on nonsense, but quite a risk of wasting attention on topics that would never find a relevant application, were playing with math and building capacity to imagine more kinds of ideas not a goal in itself.
2Dagon
Note also that they may be taking positions which are selected for being easy to argue - they're the ones they were convinced by, of course.  Whether you think that has correlation with truth is up to you - I think so, but it's not a perfect enough correlation for it to be enough. I don't know exactly what you mean by "acceding" to a position in a discussion - if you find the arguments strong, you should probably acknowledge that - this isn't a battle, it's a discussion.  If you don't find yourself actually convinced, you should state that too, even if your points of disagreement are somewhat illegible to yourself (intuition).  And, of course, if you later figure out why you disagree, you can re-open the discussion next time it's appropriate.
[-]lc221

Interesting whitepill hidden inside Scott Alexander's SB 1047 writeup was that lying doesn't work as well as predicted in politics. It's possible that if the opposition had lied less often, or we had lied more often, the bill would not have gotten a supermajority in the senate.

2Akash
Where does he say this? (I skimmed and didn’t see it.) Link here: https://www.astralcodexten.com/p/sb-1047-our-side-of-the-story
[-]lc208

"Spy" is an ambiguous term, sometimes meaning "intelligence officer" and sometimes meaning "informant". Most 'spies' in the "espionage-commiting-person" sense are untrained civilians who have chosen to pass information to officers of a foreign country, for varying reasons. So if you see someone acting suspicious, an argument like "well surely a real spy would have been coached not to do that during spy school" is locally invalid.

[-]lc200

Many of you are probably wondering what you will do if/when you see a polar bear. There's a Party Line, uncritically parroted by the internet and wildlife experts, that while you can charge/intimidate a black bear, polar bears are Obligate Carnivores and the only thing you can do is accept your fate.

I think this is nonsense. A potential polar bear attack can be defused just like a black bear attack. There are loads of youtube videos of people chasing Polar Bears away by making themselves seem big and aggressive, and I even found some indie documentaries of people who went to the arctic with expectations of being able to do this. The main trick seems to be to resist the urge to run away, make yourself look menacing, and commit to warning charges in the bear's general direction until it leaves.

8Richard_Kennaway
I can’t decide what the epistemic status of that post is, but in the same spirit, here’s how to tell the difference between a brown bear and a grizzly. Climb a tree. A brown bear will climb up after you and eat you, while a grizzly will knock down the tree and eat you.
3rotatingpaguro
I guess the point of the official party line is to avoid kids going and trying to scare polar bears.
5eukaryote
As opposed to other species of bear, which are safe for children to engage with?
4Viliam
Teddy bears. Now that I think about it, maybe teddy bears teach our kids some really dangerous habits.
2rotatingpaguro
There are too many nonpolar bears in the US to keep up the lie.
2Shankar Sivarajan
I believe the intended message of the "fight back, lay down, goodnight" maxim is "Thou shalt not generalize your experience with black bears to grizzlies!" I don't expect there is much danger of someone asking "if not friend, why friend shaped?" of polar bears; they just fill out the Rule of Three.  It's a lot like the "red touches yellow, he's a friendly fellow; red touches black, you're dead, Jack" mnemonic for snakes: people are very likely to encounter the (relatively) harmless one, and you really want them not to learn the wrong lessons from that.
[-]jbash150

Your snake mnemonic is not the standard one and gives an incorrect, inverted result. Was that intentional?

This is a coral snake, which is dangerously venomous:

Eastern Coral Snake | National Geographic

 

This is a king snake, which is totally harmless unless you're a vole or something:

3MichaelDickens
The mnemonic I've heard is "red and yellow, poisonous fellow; red and black, friend of Jack"
[-]lc200

Does anybody here have any strong reason to believe that the ML research community norm of "not taking AGI discussion seriously" stems from a different place than the oil industry's norm of "not taking carbon dioxide emission discussion seriously"?

I'm genuinely split. I can think of one or two other reasons there'd be a consensus position of dismissiveness (preventing bikeshedding, for example), but at this point I'm not sure, and it affects how I talk to ML researchers.

4Richard_Kennaway
"It is difficult to get a man to understand something, when his salary depends upon his not understanding it!" Upton Sinclair
4Dagon
I'm not sure the "ML Research Community" is cohesive enough (nor, in fact, well-defined enough) to have very strong norms about this.  Further, it's not clear that there needs to be a "consensus reasoning" even if there is a norm - different members could have different reasons for not bringing it up, and once it's established, it can be self-propagating: people don't bring it up because their peers don't bring it up. I think if you're looking for ways to talk to ML researchers, start small, and see what those particular researchers think and how they react to different approaches.  If you find some that work, then expand it to more scalable talks to groups of researchers.
1Purged Deviator
I don't expect AI researchers to achieve AGI before they find one or more horrible uses for non-general AI tools, which may divert resources, or change priorities, or do something else which prevents true AGI from ever being developed.
1