I don't think it's possible for mere mortals to use Twitter for news about politics or current events and not go a little crazy. At least, I have yet to find a Twitter user who regularly or irregularly talks about these things, and fails to boost obvious misinformation every once in a while. It doesn't matter what IQ they have or how rational they were in 2005; Twitter is just too chock full of lies, mischaracterizations, telephone games, and endless, endless, endless malicious selection effects, which by the time you're done using it are designed to appeal to whichever reader in particular you are. It's just impossible to use the site as people normally do and also practice the necessary skepticism about each individual post one is reading.
It doesn't matter what IQ they have or how rational they were in 2005
This is a reference to Eliezer, right? I really don't understand why he's on Twitter so much. I find it quite sad to see one of my heroes slipping into the ragebait Twitter attractor.
Only inasmuch he's a proof-by-example. By that I mean he's one of the most earnest/truthseeking users I found when I was still using the platform, and yet he still manages to retweet things outside his domain of expertise that are either extraordinarily misleading or literally, factually incorrect - and I think if you sat him down and prompted him to think about the individual cases he would probably notice why, he just doesn't because the platform isn't conducive to that kind of deliberate thought.
I recall a rationalist I know chiding Eliezer for his bad tweeting, and then Eliezer asked him to show him an example of a recent tweet that was bad, and then the rationalist failed to find anything especially bad.
Perhaps this has changed in the 2-3 years since that event. But I'd be interested in an example of a tweet you (lc) thought was bad.
It's not the tweets, it's the retweets. People's tweets on Twitter are usually not that bad. Their retweets, and, for slightly crazier people, their quote tweets are what contain the bizarre mischaracterizations, because they're the pulls from the top of the attention-seeking crab bucket.
I run a company that sells security software to large enterprises. I remember seeing this (since deleted) post Eliezer retweeted last year during the Crowdstrike blue screen incident, and thinking: "Am I crazy? What on earth is this guy talking about?"
The audit requirements Mark is talking about don't exist. He just completely made them up. ChatGPT's explanation here is correct; even if you're selling to the federal government[1], there's no "fast track" for big names like Crowdstrike. At absolute maximum your auditor is going to ask for evidence that you use some IDS solution, and you'll have to gather the same evidence no matter what solution you're using.
Now, Yudkowsky is not a mendacious person, and he isn't going to pump misinfo into the ether himself. But naturally if anybody goes on Twitter long enough they're gonna see stuff like this, and it will just feel plausible to you. It ...
I also think that a more insidious problem with Twitter than misinfo is the way it teaches you to think. There are certain kinds of arguments people make and positions people hold which very clearly are there because of Twitter (though not necessarily becuase they read them on Twitter). They are usually sub-par, simple-minded, and very vibes (read: not evidence) based. A common example here is the "we're so back" sort of hype-talk.
From my limited experience following AI events, agreed. Whole storms of nonsense can be generated by some random accounts posting completely non-credible claims, some people unthinkingly amplifying those, then other people seeing that they are being amplified, thinking it means there's something to them, amplifying them further, etc.
In my experience, if I look at the Twitter account of someone I respect, there's a 70–90% chance that Twitter turns them into a sort of Mr. Hyde self who's angrier, less thoughtful, and generally much worse epistemically. I've noticed this tendency in myself as well; historically I tried pretty hard to avoid writing bad tweets, and avoid reading low-quality Twitter accounts, but I don't think I succeeded, and recently I gave up and just blocked Twitter using LeechBlock.
I'm sad about this because I think Twitter could be really good, and there's a lot of good stuff on it, but there's too much bad stuff.
This framing underplays the degree to which the site is designed to produce misleading propaganda. The primary content creators are people who literally do that as a full time job.
Like, I'll show you a common pattern of how it happens. It's an extremely unfortunate example because a person involved has just died, but it's the first one I found, and I feel like it's representative of how political discourse happens on the platform:
First I'll explain what's actually misleading about this so I can make my broader point. The quote tweeted account, "Right Angle News Network", reports that "The official black lives matter account has posted a video stating that black people 'have a right to violence' amid... the slaying of Iryna Zarutska". The tweet is designed so that, while technically correct, it appears to be saying the video is about Iryna's murder. But actually:
As is typical, the agitator's tweet (which was...
The background of the Stanislav Petrov incident is literally one of the dumbest and most insane things I have ever read (attached screenshot below):
Appreciate that this means:
It's a wonder we're still here.
It is both absurd, and intolerably infuriating, just how many people on this forum think it's acceptable to claim they have figured out how qualia/consciousness works, and also not explain how one would go about making my laptop experience an emotion like 'nostalgia', or present their framework for enumerating the set of all possible qualitative experiences[1]. When it comes to this particular subject, rationalists are like crackpot physicists with a pet theory of everything, except rationalists go "Huh? Gravity?" when you ask them to explain how their theory predicts gravity, and then start arguing with you about gravity needing to be something explained by a theory of everything. You people make me want to punch my drywall sometimes.
For the record: the purpose of having a "theory of consciousness" is so it can tell us which blobs of matter feel particular things under which specific circumstances, and teach others how to make new blobs of matter that feel particular things. Down to the level of having a field of AI anaesthesiology. If your theory of consciousness does not do this, perhaps because the sum total of your brilliant insights are "systems feel 'things' when they're, y'...
or present their framework for enumerating the set of all possible qualitative experiences (Including the ones not experienced by humans naturally, and/or only accessible via narcotics, and/or involve senses humans do not have or have just happened not to be produced in the animal kingdom)
Strongly agree. If you want to explain qualia, explain how to create experiences, explain how each experience relates to all other experiences.
I think Eliezer should've talked more about this in The Fun Theory Sequence. Because properties of qualia is a more fundamental topic than "fun".
And I believe that knowledge about qualia may be one of the most fundamental types of knowledge. I.e. potentially more fundamental than math and physics.
I think Eliezer should've talked more about this in The Fun Theory Sequence. Because properties of qualia is a more fundamental topic than "fun".
I think Eliezer just straight up tends not to acknowledge that people sometimes genuinely care about their internal experiences, independent of the outside world, terminally. Certainly, there are people who care about things that are not that, but Eliezer often writes as if people can't care about the qualia - that they must value video games or science instead of the pleasure derived from video games or science.
His theory of fun is thus mostly a description of how to build a utopia for humans who find it unacceptable to "cheat" by using subdermal space heroin implants. That's valuable for him and people like him, but if aligned AGI gets here I will just tell it to reconfigure my brain not to feel bored, instead of trying to reconfigure the entire universe in an attempt to make monkey brain compatible with it. I sorta consider that preference a lucky fact about myself, which will allow me to experience significantly more positive and exotic emotions throughout the far future, if it goes well, than the people who insist they must only feel satisfied after literally eating hamburgers or reading jokes they haven't read before.
This is probably part of why I feel more urgency in getting an actually useful theory of qualitative experience than most LW users.
Why would you expect anyone to have a coherent theory of something they can’t even define and measure?
Because they say so. The problem then is why they think they have a coherent theory of something they can't define or measure.
Bad people underestimate how nice some people are and nice people underestimate how bad some people are.
As soon as you convincingly argue that there is an underestimation, it goes away.
… provided that it can be propagated to all the other beliefs, thoughts, etc. that it would affect.
In a human mind, I think the dense version of this looks similar to deep grief processing (because that's a prominent example of where a high propagation load suddenly shows up and is really salient and important), and the sparse version looks more like a many-year-long sequence of “oh wait, I should correct for” moments which individually have a high chance to not occur if they're crowded out, and the sparse version is much more common (and even the dense version usually trails off into it to some degree).
There's probably intermediary versions of this where broad updates can occur smoothly but rapidly in an environment with (usually social) persistent feedback, like going through a training course, but that's a lot more intense than just having something pointed out to you.
The Prediction Market Discord Message, by Eva_:
Current market structures can't bill people for the information value that went into the market fairly, can't fairly handle secret information known to only some bidders, pays out most of the subsidy to whoever corrects the naive bidder fastest even though there's no benefit to making it a race, offers almost no profit to people trying to defend the true price from incorrect bidders unless they let the price shift substantially first, can't be effectively used to collate information known by different bidders, can't handle counterfactuals / policy conditionals cleanly, implement EDT instead of LDT, let you play games of tricking other bidders for profit and so require everyone to play trading strategies that are inexploitable even if less beneficial, can't defend against people who are intentionally illegible as to whether they have private information or are manipulating the market for profit elsewhere...
...But most of all, prediction markets contain supposedly ideal economic actors who don't really suspect each other of dishonesty betting money against each other even though it's a net-zero trade and the aggreeement theorem says th
I’m pretty interested in this as an exercise of ‘okay yep a bunch of those problems seem real. Can we make conceptual or mechanism-design progress on them in like an afternoon of thought?’
I notice that although the loot box is gone, the unusually strong votes that people made yesterday persist.
If you attend a talk at a rationalist conference, please do not spontaneously interject unless the presenter has explicitly clarified that you are free to do so. Neither should you answer questions on behalf of the presenter during a Q&A portion. People come to talks to listen to the presenter, not a random person in the audience.
If you decide to do this anyways, you will usually not get audiovisual feedback from the other audience members that it was rude/cringeworthy for you to interject, even if internally they are desperate for you to stop doing it.
at a rationalist conference
Not that I expect you to disagree, but to make it explicit, I don't think this is something that is specific to rationalist conferences. I think it applies to a large majority of conferences.
If you decide to do this anyways, you will usually not get audiovisual feedback from the other audience members that it was rude/cringeworthy for you to interject, even if internally they are desperate for you to stop doing it.
You also very well might not get this feedback from the presenter. They may not be confrontational enough to call you out on it. And with the spotlight on them, they may feel uncomfortable doing things like sighing in exasperation or showing frustration in their facial expressions and body language.
whether you're asking a clarifying question that other audience members found useful
This is a frequent problem in math heavy research presentations. Someone presents their research, but they commit a form of the typical mind fallacy, where they understand their own research so well that they fatally misjudge how hard it is to understand for others. If the audience consists of professionals, often nobody dares to stop the presenter with clarificatory questions, because nobody wants to look stupid in front of all the other people who don't ask questions and therefore clearly (right!?) understand the presented material. In the end, probably 90% have mentally lost the thread somewhere before the finish line. Of course nobody admits it, lest your colleagues notice your embarrassing lack of IQ!
If the interjection is about your personal hobbyhorse or pet peave or theory or the like, then definitely shut up and sit down.
I make the simpler request because often rationalists don't seem to be able to tell when this is (or at least tell when others can tell)
The problem with trade agreements as a tool for maintaining peace is that they only provide an intellectual and economic reason for maintaining good relations between countries, not an emotional once. People's opinions on war rarely stem from economic self interest. Policymakers know about the benefits and (sometimes) take them into account, but important trade doesn't make regular Americans grateful to the Chinese for providing them with so many cheap goods - much the opposite, in fact. The number of people who end up interacting with Chinese people or intuitively understanding the benefits firsthand as a result of expanded business opportunities is very small.
On the other hand, video games, social media, and the internet have probably done more to make Americans feel aligned with the other NATO countries than any trade agreement ever. The YouTubers and Twitch streamers I have pseudosocial relationships with are something like 35% Europeans. I thought Canadians spoke Canadian and Canada was basically some big hippie commune right up until my minecraft server got populated with them. In some weird alternate universe where people are suggesting we invade Canada, my first instinctual...
Pretty much ~everybody on the internet I can find talking about the issue both mischaracterizes and exaggerates the extent of child sex work inside the United States, often to a patently absurd degree. Wikipedia alone reports that there are anywhere from "100,000-1,000,000" child prostitutes in the U.S. There are only ~75 million children in the U.S., so I guess Wikipedia thinks it's possible that more than 1% of people aged 0-17 are prostitutes. As in most cases, these numbers are sourced from "anti sex trafficking" organizations that, as far as I can tell, completely make them up.
Actual child sex workers - the kind that get arrested, because people don't like child prostitution - are mostly children who pass themselves off as adults in order to make money. Part of the confusion comes from the fact that the government classifies any instance of child prostitution as human trafficking, regardless of whether or not there's evidence the child was coerced. Thus, when the Department of Justice reports that federal law enforcement investigated "2,515 instances of suspected human trafficking" from 2008-2010, and that "forty percent involved prostitution of a child or child sexual exploit...
To the LW devs - just want to mention that this website is probably now the most well designed forum I have ever used. The UX is almost addictively good and I've been loving all of the little improvements over the past year or so.
The Nick Bostrom fiasco is instructive: never make public apologies to an outrage machine. If Nick had just ignored whoever it was trying to blackmail him, it would have been on them to assert the importance of a twenty-five year old deliberately provocative email, and things might not have ascended to the point of mild drama. When he tried to "get ahead of things" by issuing an apology, he ceded that the email was in fact socially significant despite its age, and that he did in fact have something to apologize for, and so opened himself up to the Standard Replies that the apology is not genuine, he's secretly evil etc. etc.
Instead, if you are ever put in this situation, just say nothing. Don't try to defend yourself. Definitely don't volunteer for a struggle session.
Treat outrage artists like the police. You do not prevent the police from filing charges against you by driving to the station and attempting to "explain yourself" to detectives, or by writing and publishing a letter explaining how sorry you are. At best you will inflate the airtime of the controversy by responding to it, at worst you'll be creating the controversy in the first place.
"Treaties" and "settlements" between two parties can be arbitrarily bad. "We'll kill you quickly and painlessly" is a settlement.
The two guys from Epoch on the recent Dwarkesh Patel podcast repeatedly made the argument that we shouldn't fear AI catastrophe, because even if our successor AIs wanted to pave our cities with datacenters, they would negotiate a treaty with us instead of killing us. It's a ridiculous argument for many reasons but one of them is that they use abstract game theoretic and economic terms to hide nasty implementation details
So apparently in 2015 Sam Altman said:
Serious question: Is he a comic book supervillain? Is this world actually real? Why does this quote not garner an emotive reaction out of anybody but me?
I was surprised by this quote. On following the link, the sentence by itself seems noticeably out of context; here's the next part:
On the growing artificial intelligence market: “AI will probably most likely lead to the end of the world, but in the meantime, there’ll be great companies.”
On what Altman would do if he were President Obama: “If I were Barack Obama, I would commit maybe $100 billion to R&D of AI safety initiatives.” Altman also shared that he recently invested in a company doing "AI safety research" to investigate the potential risks of artificial intelligence.
Old internet arguments about religion and politics felt real. Yeah, the "debates" were often excuses to have a pissing competition, but a lot of people took the question of "who was right" seriously. And if you actually didn't care, you were at least motivated to pretend you did to the audience.
Nowadays people don't even seem to pretend to care about the underlying content. If someone seems like they're being too earnest, others just reply with a picture of their face. It's sad.
This is all speculative, but I think some of this might be due to differences in how algorithms pick content to show users, the aging or death of influential figures in the New Atheist movement, and changes with respect to how religion makes its way into our lives.
See also: New Atheism: The Godlessness that Failed. Relevant quote:
The rise of the Internet broadened our intellectual horizons. We got access to a whole new world of people with totally different standards, norms, and ideologies opposed to our own. When the Internet was small and confined to an optimistic group of technophile intellectuals, this spawned Early Internet Argument Culture, where we tried to iron out our differences through Reason. We hoped that the new world the Web revealed to us could be managed in the same friendly way we managed differences with our crazy uncle or the next-door neighbor.
As friendly debate started feeling more and more inadequate, and as newer and less nerdy people started taking over the Internet, this dream receded. In its place, we were left with an intolerable truth: a lot of people seem really horrible, and refuse to stop being horrible even when we ask them nicely. They seem to believe awful things. They seem to act in awful ways. When we tell them the obviously correct reasons they should be more like us, they refuse to listen to them, and instead spout insane moon gibberish about how they are right and we are wrong.
Most or even all of the a...
The "people are wonderful" bias is so pernicious and widespread I've never actually seen it articulated in detail or argued for. I think most people greatly underestimate the size of this bias, and assume opinions either way are a form of mind-projection fallacy on the part of nice/evil people. In fact, it looks to me like this skew is the deeper origin of a lot of other biases, including the just-world fallacy, and the cause of a lot of default contentment with a lot of our institutions of science, government, etc. You could call it a meta-bias that causes the Hansonian stuff to go largely unnoticed.
I would be willing to pay someone to help draft a LessWrong post for me about this; I think it's important but my writing skills are lacking.
PSA: I have realized very recently after extensive interactive online discussion with rationalists, that they are exceptionally good at arguing. Too good. Probably there's some inadvertent pre- or post- selection for skill at debating high concept stuff going on.
Wait a bit until acceding to their position in a live discussion with them where you start by disagreeing strongly for maybe intuitive reasons and then suddenly find the ground shifting beneath your feet. It took me repeated interactions where I only later realized I'd been hoodwinked by faulty reasoning to notice the pattern.
I think in general believing something before you have intuition around it is unreliable or vulnerable to manipulation, even if there seems to be a good System 2 reason to do so. Such intuition is specialized common sense, and stepping outside common sense is stepping outside your goodhart scope where ability to reliably reason might break down.
So it doesn't matter who you are arguing with, don't believe something unless you understand it intuitively. Usually believing things is unnecessary regardless, it's sufficient to understand them to make conclusions and learn more without committing to belief. And certainly it's often useful to make decisions without committing to believe the premises on which the decisions rest, because some decisions don't wait on the ratchet of epistemic rationality.
Much like how all crashes involving self-driving cars get widely publicized, regardless of rarity, for a while people will probably overhype instances of AIs destroying production databases or mismanaging accounting, even after those catastrophies become less common than human mistakes.
Sarcasm is when we make statements we don't mean, expecting the other person to infer from context that we meant the opposite. It's a way of pointing out how unlikely it would be for you to mean what you said, by saying it.
There are two ways to evoke sarcasm; first by making your statement unlikely in context, and second by using "sarcasm voice", i.e. picking tones and verbiage that explicitly signal sarcasm. The sarcasm that people consider grating is usually the kind that relies on the second category of signals, rather than the first. It becomes more funny when the joker is able to say something almost-but-not-quite plausible in a completely deadpan manner. Compare:
As a corollary, sarcasm often works more smoothly when it's between people who already know each other, not only because it's less likely to be offensive, but also because they're starting with a strong prior about what their counterparties are likely to say in normal conversation.
Interesting whitepill hidden inside Scott Alexander's SB 1047 writeup was that lying doesn't work as well as predicted in politics. It's possible that if the opposition had lied less often, or we had lied more often, the bill would not have gotten a supermajority in the senate.
Postdiction: Modern "cancel culture" was mostly a consequence of new communication systems (social media, etc.) rather than a consequence of "naturally" shifting attitudes or politics.
The European wars of religion during the 16th to early 18th century were plausibly caused or at least strongly fanned by the invention of the printing press.
"Spy" is an ambiguous term, sometimes meaning "intelligence officer" and sometimes meaning "informant". Most 'spies' in the "espionage-commiting-person" sense are untrained civilians who have chosen to pass information to officers of a foreign country, for varying reasons. So if you see someone acting suspicious, an argument like "well surely a real spy would have been coached not to do that during spy school" is locally invalid.
Many of you are probably wondering what you will do if/when you see a polar bear. There's a Party Line, uncritically parroted by the internet and wildlife experts, that while you can charge/intimidate a black bear, polar bears are Obligate Carnivores and the only thing you can do is accept your fate.
I think this is nonsense. A potential polar bear attack can be defused just like a black bear attack. There are loads of youtube videos of people chasing Polar Bears away by making themselves seem big and aggressive, and I even found some indie documentaries of people who went to the arctic with expectations of being able to do this. The main trick seems to be to resist the urge to run away, make yourself look menacing, and commit to warning charges in the bear's general direction until it leaves.
Lie detection technology is going mainstream. ClearSpeed is such an accuracy and ease of use improvement to polygraphs that various government LEO and military are starting to notice. In 2027 (edit: maybe more like 2029) it will be common knowledge that you can no longer lie to the police, and you should prepare for this eventuality if you haven't.
Hey [anonymous]. I see you deactivated your account. Hope you're okay! Happy to chat if you want on Signal at five one oh, nine nine eight, four seven seven one (also a +1 at the front for US country code).
(Follow-up: [anonymous] reached out, is doing fine.)
Does anybody here have any strong reason to believe that the ML research community norm of "not taking AGI discussion seriously" stems from a different place than the oil industry's norm of "not taking carbon dioxide emission discussion seriously"?
I'm genuinely split. I can think of one or two other reasons there'd be a consensus position of dismissiveness (preventing bikeshedding, for example), but at this point I'm not sure, and it affects how I talk to ML researchers.
Anthropic has a bug bounty for jailbreaks: https://hackerone.com/constitutional-classifiers?type=team
If you can figure out how to get the model to give detailed answers to a set of certain questions, you get a 10k prize. If you can find a universal jailbreak for all the questions, you get 20k.
One large part of the AI 2027 piece is contigent on the inability of nation state actors to steal model weights. The authors' take is that while China is going to be able to steal trade secrets, they aren't going to be able to directly pilfer the model weights more than once or twice. I haven't studied the topic as deeply as the authors but this strikes me as naive, especially when you consider side-channel[1] methods of 'distilling' the models based on API output.
@Daniel Kokotajlo @ryan_greenblatt Did you guys consider these in writing the post? Is there some reason to believe these will be ineffective, or not provide the necessary access that raw weights lifted off the GPUs would give?
I would normally consider "side channel attacks work" to be the naive position, but in the AI 2027 post, the thesis is that there exists a determined, well resourced attacker (China) that already has insiders who can inform them on relevant details about OpenAI infrastructure, and already understands how the models were developed in the first place.
I agree that its very plausible that China would steal the weights of Agent-3 or Agent-4 after stealing Agent-2. This was a toss up when writing the story, we ultimately went with just stealing Agent-2 for a combination of reasons. From memory the most compelling were something like:
Something that is a bit more under-explored is the potential sabotage between the projects. We have the uncertain assumption that the efforts on both sides would roughly cancel out, but we were quite uncertain on the offense-defense balance. I think a more offense favored reality could change the story quite a bit, basically with the idea of MAIM slowing both sides down a bunch.
Now is the time to write to your congressman and (may allah forgive me for uttering this term) "signal boost" about actually effective AI regulation strategies - retroactive funding for hitting interpretability milestones, good liability rules surrounding accidents, funding for long term safety research. Use whatever contacts you have, this week. Congress is writing these rules now and we may not have another chance to affect them.