It is both absurd, and intolerably infuriating, just how many people on this forum think it's acceptable to claim they have figured out how qualia/consciousness works, and also not explain how one would go about making my laptop experience an emotion like 'nostalgia', or present their framework for enumerating the set of all possible qualitative experiences[1]. When it comes to this particular subject, rationalists are like crackpot physicists with a pet theory of everything, except rationalists go "Huh? Gravity?" when you ask them to explain how their theory predicts gravity, and then start arguing with you about gravity needing to be something explained by a theory of everything. You people make me want to punch my drywall sometimes.
For the record: the purpose of having a "theory of consciousness" is so it can tell us which blobs of matter feel particular things under which specific circumstances, and teach others how to make new blobs of matter that feel particular things. Down to the level of having a field of AI anaesthesiology. If your theory of consciousness does not do this, perhaps because the sum total of your brilliant insights are "systems feel 'things' when they're, y'...
I think Eliezer should've talked more about this in The Fun Theory Sequence. Because properties of qualia is a more fundamental topic than "fun".
I think Eliezer just straight up tends not to acknowledge that people sometimes genuinely care about their internal experiences, independent of the outside world, terminally. Certainly, there are people who care about things that are not that, but Eliezer often writes as if people can't care about the qualia - that they must value video games or science instead of the pleasure derived from video games or science.
His theory of fun is thus mostly a description of how to build a utopia for humans who find it unacceptable to "cheat" by using subdermal space heroin implants. That's valuable for him and people like him, but if aligned AGI gets here I will just tell it to reconfigure my brain not to feel bored, instead of trying to reconfigure the entire universe in an attempt to make monkey brain compatible with it. I sorta consider that preference a lucky fact about myself, which will allow me to experience significantly more positive and exotic emotions throughout the far future, if it goes well, than the people who insist they must only feel satisfied after literally eating hamburgers or reading jokes they haven't read before.
This is probably part of why I feel more urgency in getting an actually useful theory of qualitative experience than most LW users.
The Nick Bostrom fiasco is instructive: never make public apologies to an outrage machine. If Nick had just ignored whoever it was trying to blackmail him, it would have been on them to assert the importance of a twenty-five year old deliberately provocative email, and things might not have ascended to the point of mild drama. When he tried to "get ahead of things" by issuing an apology, he ceded that the email was in fact socially significant despite its age, and that he did in fact have something to apologize for, and so opened himself up to the Standard Replies that the apology is not genuine, he's secretly evil etc. etc.
Instead, if you are ever put in this situation, just say nothing. Don't try to defend yourself. Definitely don't volunteer for a struggle session.
Treat outrage artists like the police. You do not prevent the police from filing charges against you by driving to the station and attempting to "explain yourself" to detectives, or by writing and publishing a letter explaining how sorry you are. At best you will inflate the airtime of the controversy by responding to it, at worst you'll be creating the controversy in the first place.
The problem with trade agreements as a tool for maintaining peace is that they only provide an intellectual and economic reason for maintaining good relations between countries, not an emotional once. People's opinions on war rarely stem from economic self interest. Policymakers know about the benefits and (sometimes) take them into account, but important trade doesn't make regular Americans grateful to the Chinese for providing them with so many cheap goods - much the opposite, in fact. The number of people who end up interacting with Chinese people or intuitively understanding the benefits firsthand as a result of expanded business opportunities is very small.
On the other hand, video games, social media, and the internet have probably done more to make Americans feel aligned with the other NATO countries than any trade agreement ever. The YouTubers and Twitch streamers I have pseudosocial relationships with are something like 35% Europeans. I thought Canadians spoke Canadian and Canada was basically some big hippie commune right up until my minecraft server got populated with them. In some weird alternate universe where people are suggesting we invade Canada, my first instinctual...
So apparently in 2015 Sam Altman said:
Serious question: Is he a comic book supervillain? Is this world actually real? Why does this quote not garner an emotive reaction out of anybody but me?
I was surprised by this quote. On following the link, the sentence by itself seems noticeably out of context; here's the next part:
On the growing artificial intelligence market: “AI will probably most likely lead to the end of the world, but in the meantime, there’ll be great companies.”
On what Altman would do if he were President Obama: “If I were Barack Obama, I would commit maybe $100 billion to R&D of AI safety initiatives.” Altman also shared that he recently invested in a company doing "AI safety research" to investigate the potential risks of artificial intelligence.
PSA: I have realized very recently after extensive interactive online discussion with rationalists, that they are exceptionally good at arguing. Too good. Probably there's some inadvertent pre- or post- selection for skill at debating high concept stuff going on.
Wait a bit until acceding to their position in a live discussion with them where you start by disagreeing strongly for maybe intuitive reasons and then suddenly find the ground shifting beneath your feet. It took me repeated interactions where I only later realized I'd been hoodwinked by faulty reasoning to notice the pattern.
I think in general believing something before you have intuition around it is unreliable or vulnerable to manipulation, even if there seems to be a good System 2 reason to do so. Such intuition is specialized common sense, and stepping outside common sense is stepping outside your goodhart scope where ability to reliably reason might break down.
So it doesn't matter who you are arguing with, don't believe something unless you understand it intuitively. Usually believing things is unnecessary regardless, it's sufficient to understand them to make conclusions and learn more without committing to belief. And certainly it's often useful to make decisions without committing to believe the premises on which the decisions rest, because some decisions don't wait on the ratchet of epistemic rationality.
Does anybody here have any strong reason to believe that the ML research community norm of "not taking AGI discussion seriously" stems from a different place than the oil industry's norm of "not taking carbon dioxide emission discussion seriously"?
I'm genuinely split. I can think of one or two other reasons there'd be a consensus position of dismissiveness (preventing bikeshedding, for example), but at this point I'm not sure, and it affects how I talk to ML researchers.
Now is the time to write to your congressman and (may allah forgive me for uttering this term) "signal boost" about actually effective AI regulation strategies - retroactive funding for hitting interpretability milestones, good liability rules surrounding accidents, funding for long term safety research. Use whatever contacts you have, this week. Congress is writing these rules now and we may not have another chance to affect them.
Noticed something recently. As an alien, you could read pretty much everything Wikipedia has on celebrities, both on individual people and the general articles about celebrity as a concept... And never learn that celebrities tend to be extraordinarily attractive. I'm not talking about an accurate or even attempted explanation for the tendency, I'm talking about the existence of the tendency at all. I've tried to find something on wikipedia that states it, but that information just doesn't exist (except, of course, implicitly through photographs).
It's quite odd, and I'm sure it's not alone. "Celebrities are attractive" is one obvious piece of some broader set of truisms that seem to be completely missing from the world's most complete database of factual information.
Let me put in my 2c now that the collapse of FTX is going to be mostly irrelevant to effective altruism except inasmuch as EA and longtermist foundations no longer have a bunch of incoming money from Sam Bankman Fried. People are going on and on about the "PR damage" to EA by association because a large donor turned out to be a fraud, but are failing to actually predict what the concrete consequences of such a "PR loss" are going to be. Seems to me like they're making the typical fallacy of overestimating general public perception[1]'s relevance to an insular ingroup's ability to accomplish goals, as well as the public's attention span in the first place.
As measured by what little Rationalists read from members of the public while glued to Twitter for four hours each day.
LessWrong as a website has gotten much more buggy for me lately. 6 months ago it worked like clockwork, but recently I'm noticing that refreshes on my profile page take something like 18 seconds to complete, or even 504 (!). I'm trying to edit my old "pessimistic alignment" post now and the interface is just not letting me; the site just freezes for a while and then refuses to put the content in the text box for me to edit.
A surprisingly large amount of people seem to apply statuslike reasoning toward inanimate goods. To many, if someone sells a coin or an NFT for a very high price, this is not merely curious or misguided: it's outright infuriating. They react as if others are making a tremendous social faux pas - and even worse, that society is validating their missteps.
Made an opinionated "update" for the anti-kibitzer mode script; it works for current LessWrong with its agree/disagree votes and all that jazz, fixes some longstanding bugs that break the formatting of the site and allow you to see votes in certain places, and doesn't indent usernames anymore. Install Tampermonkey and browse to this link if you'd like to use it.
Semi-related, I am instituting a Reign Of Terror policy for my poasts/shortform, which I will update my moderation policy with. The general goal of these policies is to reduce the amount of ti...
Computer hacking is not a particularly significant medium of communication between prominent AI research labs, nonprofits, or academic researchers. Much more often than leaked trade secrets, ML people will just use insights found in this online repository called arxiv, where many of them openly and intentionally publish their findings. Nor (as far as I am aware) are stolen trade secrets a significant source of foundational insights for researchers making capabilities gains, local to their institution or otherwise.
I don't see this changing on its own,...
Currently reading The Rise and Fall of the Third Reich for the first time. I've wanted to read a book about Nazi Germany for a while now, and tried more "modern" and "updated" books, but IMO they are still pretty inferior to this one. The recent books from historians I looked at were concerned more with an ideological opposition to Great Men theories than factual accuracy, and also simply failed to hold my attention. Newer books are also necessarily written by someone who wasn't there, and by someone who does not feel comfortable commenting about events fr...
I have been working on a detailed post for about a month and a half now about how computer security is going to get catastrophically worse as we get the next 3-10 years of AI advancements, and unfortunately reality is moving faster than I can finish it:
https://krebsonsecurity.com/2022/10/glut-of-fake-linkedin-profiles-pits-hr-against-the-bots/
Either post your NASDAQ 100 futures contracts or stop fronting near-term slow takeoff probabilities above ~10%.
>be big unimpeachable tech ceo
>need to make some layoffs, but don't want to have to kill morale, or for your employees to think you're disloyal
>publish a manifesto on the internet exclaiming your corporation's allegiance to right-libertarianism or something
>half of your payroll resigns voluntarily without any purging
>give half their pay to the other half of your workforce and make an extra 200MM that year
Based on Victoria Nuland's recent senate testimony, I'm registering a 66% prediction that those U.S. administered biological weapons facilities in Ukraine actually do indeed exist, and are not Russian propaganda.
Of course I don't think this is why they invaded, but the media is painting this as a crazy conspiracy theory, when they have very little reason to know either way.
It is unnecessary to postulate that CEOs and governments will be "overthrown" by rogue AI. Board members in the future will insist that their company appoint an AI to run the company because they think they'll get better returns that way. Congressmen will use them to manage their campaigns and draft their laws. Heads of state will use them to manage their militaries and police agencies. If someone objects that their AI is really unreliable or doesn't look like it shares their values, someone else on the board will say "But $NFGM is doing the same thing; we...
Good rationalists have an absurd advantage over the field in altruism, and only a marginal advantage in highly optimized status arenas like tech startups. The human brain is already designed to be effective when it comes to status competitions, and systematically ineffective when it comes to helping other people.
So it's much more of a tragedy for the competent rationalist to choose to spend most of their time competing in those things than to shoot a shot at a wacky idea you have for helping others. You might reasonably expect to be better at it than 99% of the people who (respectably!) attempt to do so. Consider not burning that advantage!
Forcing your predictions, even if they rely on intuition, to land on nice round numbers so others don't infer things about the significant digits is sacrificing accuracy for the appearance of intellectual modesty. If you're around people who shouldn't care about the latter, you should feel free to throw out numbers like 86.2% and just clarify that your confidence is way outside 0.1%, if that's just the best available number for you to pick.
Saw some today demonstrating what I like to call the "Kirkegaard fallacy", in response to the Debrief article making the rounds.
People who have one obscure or weird belief tend to be unusually open minded and thus have other weird beliefs. Sometimes this is because they enter a feedback loop where they discover some established opinion is likely wrong, and then discount perceived evidence for all other established opinions.
This is a predictable state of affairs regardless of the nonconsensus belief, so the fact that a person currently talking to you about e.g. UFOs entertains other off-brand ideas like parapsychology or afterlives is not good evidence that the other nonconsensus opinion in particular is false.
Putting body cameras on police officers often increases tyranny. In particular, applying 24/7 monitoring to foot soldiers forces those foot soldiers to strictly follow protocol and arrest people for infractions that they wouldn't otherwise. In the 80s, for example, there were many officers who chose not to follow mandatory arrest procedures for drugs like marijuana, because they didn't want to and it was unworth their time. Not so in todays era, mostly, where they would have essentially no choice except to follow orders or resign.
Anarchocapitalism is pretty silly, but I think there are kernels of it that provide interesting solutions to social problems.
For example: imagine lenders and borrowers could pay for & agree on enforcement mechanisms for nonpayment metered out by the state, instead of it just being dictated by congress. E.g. if you don't pay this back on time you go to prison for ${n} months. This way people with bad credit scores or poor impulse control might still be able to get credit.
There's a portion of project lawful where Keltham contemplates a strategy of releasing Rovagug as a way to "distract" the Gods while Keltham does something sinister.
Wouldn't Lawful beings with good decision theory precommit to not being distracted and just immediately squish Keltham, thereby discouraging him from attempting those sorts of strategies?
Giving people money for doing good things they can't publicly take credit for is awesome, but what would honestly motivate me to do something like that just as much would be if I could have an official nice-looking but undesignated Truman Award plaque to keep in my apartment. That way people in the know who visit me or who googled it would go "So, what'd you actually get that for?" and I'd just mysteriously smile and casually move the conversation along.

Feel free to brag shamelessly to me about any legitimate work for alignment you've done outside of my posts (which are under an anti-kibitzer policy).
IMO: Microservices and "siloing" in general is a strategy for solving principal-agent problems inside large technology companies. It is not a tool for solving technical problems and is generally strictly inferior to monoliths otherwise, especially when working on a startup where the requirements for your application are changing all of the time.
Two caveats to efficient markets in finance that I've just considered, but don't see mentioned a lot in discussions of bubbles like the one we just experienced, at least as a non-economist:
First: Irrational people are constantly entering the market, often in ways that can't necessarily be predicted. The idea that people who make bad trades will eventually lose all of their money and be swamped by the better investors is only valid inasmuch as the actors currently participating in the market stay the same. This means that it's perfectly possible for either ...
The most common refrain I hear against the possibility of widespread voter fraud is that demographers and pollsters would catch such malfeasance, but in practice when pollsters see a discrepancy between voting results and polls they seem to just assume the polls were biased. Is there a better reason besides "the FBI seems pretty competent"?
Apparently I was wrong[1] - OpenAI does care about ChatGPT jailbreaks.
Here is my first partial jailbreak - it's a combination of stuff I've seen people do with GPT-4, combining base64, using ChatGPT to simulate a VM, and weird invalid urls.
Sorry for having to post multiple screenshots. The base64 in the earlier message actually just produces a normal kitchen recipe, but it gives the ingredients there up. I have no idea if they're correct. When I tried later to get the unredacted version:
This book is required reading for anyone claiming that explaining the AI X-risk thesis to normies is really easy, because they "did it to Mom/Friend/Uber driver":
https://www.amazon.com/Mom-Test-customers-business-everyone-ebook/dp/B01H4G2J1U
"The test of sanity is not the normality of the method but the reasonableness of the discovery. If Newton had been informed by [the ghost of] Pythagoras that the moon was made of green cheese, then Newton would have been locked up. Gravitation, being a reasoned hypothesis which fitted remarkably well into the Copernican version of the observed physical facts of the universe, established Newton's reputation for extraordinary intelligence, and would have done so no matter how fantastically he arrived at it. Yet his theory of gravitation is not so impressive ...
Making science fiction novels or movies to tell everyone about the bad consequences of a potential technology seems completely counterproductive, in retrospect:
A small colony of humans is a genuinely tiny waste of paperclips. I am slightly more worried about the possibility that the acausal trade equilibrium cashes out to the AGI treating us badly because some aliens in a foreign Everett branch have some bizarre religious/moral opinions about the lives we ought to lead, than I am about being turned into squiggles.
Dogs and cats are not "aligned" to the degree that would be necessary to prevent a superintelligent dog from doing bad things. If tomorrow a new chew toy were released that made dogs capable of organizing to overthrow the government and start passing mandatory petting quotas, that would be a problem.
I just launched a startup, Leonard Cyber. Basically a Pwn2Job platform.
If any hackers on LessWrong are out of work, here are some invite codes:
kBCYAzL7J5vGTGY
c8Vakd4AE3al9NI
cnMBO0ZfGhsNZd7
0zsYsfDk7r5508l
F0gv4NRID7FBeJH
"No need to invoke slippery slope fallacies, here. Let's just consider the Czechoslovakian question in of itself" - Adolf Hitler