Stampy's AI Safety Info is a little like that in that it has 1) pre-written answers, 2) a chatbot under very active development, and 3) a link to a Discord with people who are often willing to explain things. But it could probably be more like that in some ways, e.g. if more people who were willing to explain things were habitually in the Discord.
Also, I plan to post the new monthly basic AI safety questions open thread today (edit: here), which is also a little like that.
Anonymous #7 asks:
I am familiar with the concept of a utility function, which assigns numbers to possible world states and considers larger numbers to be better. However, I am unsure how to apply this function in order to make decisions that take time into account. For example, we may be able to achieve a world with higher utility over a longer period of time, or a world with lower utility but in a shorter amount of time.
Anonymous #6 asks:
Why hasn't an alien superintelligence within our light cone already killed us?
Anonymous #5 asks:
How can programers build something and dont understand inner workings of it? Are they closer to biologists-cross-breeders than to car designers?
Anonymous #4 asks:
How large space of possible minds? How its size was calculated? Why is EY thinks that human-like minds are not fill most of this space? What are the evidence for it? What are the possible evidence against "giant Mind Design Space and human-like minds are tiny dot there"?
Anonymous #3 asks:
Can AIs be anything but utility maximisers? Most of the existing programs are something like finite-steps-executors (like Witcher 3 and calculator). So what's the difference?
Anonymous #2 asks:
A footnote in 'Planning for AGI and beyond' says "Many of us think the safest quadrant in this two-by-two matrix is short timelines and slow takeoff speeds; shorter timelines seem more amenable to coordination" - why do shorter timelines seem more amenable to coordination?
Anonymous #1 asks:
...This one is not technical: now that we live in a world in which people have access to systems like ChatGPT, how should I consider any of my career choices, primarily in the context of a computer technician? I'm not a hard-worker, and I consider that my intelligence is just a little above average, so I'm not going to pretend that I'm going to become a systems analyst or software engineer, but now code programming and content creation are starting to be automated more and more, so how should I update my decisions based on that?
Sure, this qu
From 38:58 of the podcast:
...So I do think that over time I have come to expect a bit more that things will hang around in a near human place and weird shit will happen as a result. And my failure review where I look back and ask — was that a predictable sort of mistake? I feel like it was to some extent maybe a case of — you’re always going to get capabilities in some order and it was much easier to visualize the endpoint where you have all the capabilities than where you have some of the capabilities. And therefore my visualizations were not dwelling enough
trevor has already mentioned the Stampy project, which is trying to do something very similar to what's described here and wishes to join forces.
Right now, Stampy just uses language models for semantic search, but the medium-term plan is to use them for text generation as well: people will be able to go to chat.stampy.ai or chat.aisafety.info, type in questions, and have a conversational agent respond. This would probably use a language model fine-tuned by the authors of Cyborgism (probably starting with a weak model as a trial, then increasingly strong on...
There's another issue where "P(doom)" can be read either as the probability that a bad outcome will happen, or the probability that a bad outcome is inevitable. I think the former is usually what's meant, but if "P(doom)" means "the probability that we're doomed", then that suggests the latter as a distracting alternative interpretation.
In terms of "and those people who care will be broad and varied and trying their hands at making movies and doing varied kinds of science and engineering research and learning all about the world while keeping their eyes open for clues about the AI risk conundrum, and being ready to act when a hopeful possibility comes up" we're doing less well compared to my 2008 hopes. I want to know why and how to unblock it.
I think to the extent that people are failing to be interesting in all the ways you'd hoped they would be, it's because being interesting in th...
As far as I know, this is the standard position. See also this FAQ entry. A lot of people sloppily say "the universe" when they mean the observable part of the universe, and that's what's causing the confusion.
I have also talked with folks who’ve thought a lot about safety and who honestly think that existential risk is lower if we have AI soon (before humanity can harm itself in other ways), for example.
It seems hard to make the numbers come out that way. E.g. suppose human-level AGI in 2030 would cause a 60% chance of existential disaster and a 40% chance of existential disaster becoming impossible, and human-level AGI in 2050 would cause a 50% chance of existential disaster and a 50% chance of existential disaster becoming impossible. Then to be indifferen...
"Safewashing" would be more directly parallel to "greenwashing" and sounds less awkward to my ears than "safetywashing", but on the other hand the relevant ideas are more often called "AI safety" than "safe AI", so I'm not sure if it's a better or worse term.
Yes, my experience of "nobody listened 20 years ago when the case for caring about AI risk was already overwhelmingly strong and urgent" doesn't put strong bounds on how much I should anticipate that people will care about AI risk in the future, and this is important; but it puts stronger bounds on how much I should anticipate that people will care about counterintuitive aspects of AI risk that haven't yet undergone a slow process of climbing in mainstream respectability, even if the case for caring about those aspects is overwhelmingly strong and urgent (...
Note that the full 2021 MIRI conversations are also available (in robot voice) in the Nonlinear Library archive.
As I see it, "rationalist" already refers to a person who thinks rationality is particularly important, not necessarily a person who is rational, like how "libertarian" refers to a person who thinks freedom is particularly important, not necessarily a person who is free. Then literally speaking "aspiring rationalist" refers to a person who aspires to think rationality is particularly important, not to a person who aspires to be rational. Using "aspiring rationalist" to refer to people who aspire to attain rationality encourages people to misinterpret self-...
Great report. I found the high decision-worthiness vignette especially interesting.
I haven't read it closely yet, so people should feel free to be like "just read the report more closely and the answers are in there", but here are some confusions and questions that have been on my mind when trying to understand these things:
Has anyone thought about this in terms of a "consequence indication assumption" that's like the self-indication assumption but normalizes by the probability of producing paths from selves to cared-about consequences instead of the proba...
My impression (based on using Metaculus a lot) is that, while questions like this may give you a reasonable ballpark estimate and it's great that they exist, they're nowhere close to being efficient enough for it to mean much when they fail to move. As a proxy for the amount of mental effort that goes into it, there's only been three comments on the linked question in the last month. I've been complaining about people calling Metaculus a "prediction market" because if people think it's a prediction market then they'll assume there's a point to be made like...
Metaculus (unlike Manifold) is not a market and does not use play money except in the same sense that Tetris score is play money.
I don't understand why people are calling Metaculus a prediction market. There's no buying or selling going on, even in play money. There's a score, but score doesn't affect the community estimate, which is just a median of all user predictions weighted by recency. I think it ends up doing pretty well, but calling it a market (which it doesn't call itself) will give readers a mistaken impression of how it works.
It took a minute to "click" for me that the green up marks and red down marks corresponded to each other in four opposed pairs, and that the Truth/Aim/Clarity numbers also corresponded to these axes. Possibly this is because I went straight to the thread after quickly skimming the OP, but most threads won't have the OP to explain things anyway. So my impression is it should be less opaque somehow. I do like having votes convey a lot more information than up/down. I wonder if it would be best to hide the new features under some sort of "advanced options" in...
Are there online spaces that talk about the same stuff LW talks about (AI futurism, technical rationality, and so on), with reasonably high quality standards, but more conversational-oriented and less soapbox-oriented, and maybe with less respectability signaling? I often find myself wanting to talk about things discussed here but feeling overconstrained by things like knowing that comments are permanent and having to anticipate objections instead of taking them as they come.
I tend to want to split "value drift" into "change in the mapping from (possible beliefs about logical and empirical questions) to (implied values)" and "change in beliefs about logical and empirical questions", instead of lumping both into "change in values".
This seems to be missing what I see as the strongest argument for "utopia": most of what we think of as "bad values" in humans comes from objective mistakes in reasoning about the world and about moral philosophy, rather than from a part of us that is orthogonal to such reasoning in a paperclip-maximizer-like way, and future reflection can be expected to correct those mistakes.
future reflection can be expected to correct those mistakes.
I'm pretty worried that this won't happen, because these aren't "innocent" mistakes. Copying from a comment elsewhere:
...Why did the Malagasy people have such a silly belief? Why do many people have very silly beliefs today? (Among the least politically risky ones to cite, someone I’ve known for years who otherwise is intelligent and successful, currently believes, or at least believed in the recent past, that 2⁄3 of everyone will die as a result of taking the COVID vaccines.) I think the unfort
"Problematic dynamics happened at Leverage" and "Leverage influenced EA Summit/Global" don't imply "Problematic dynamics at Leverage influenced EA Summit/Global" if EA Summit/Global had their own filters against problematic influences. (If such filters failed, it should be possible to point out where.)
Your posts seem to be about what happens if you filter out considerations that don't go your way. Obviously, yes, that way you can get distortion without saying anything false. But the proposal here is to avoid certain topics and be fully honest about which topics are being avoided. This doesn't create even a single bit of distortion. A blank canvas is not a distorted map. People can get their maps elsewhere, as they already do on many subjects, and as they will keep having to do regardless, simply because some filtering is inevitable beneath the eye of Sa...
due to the mechanisms described in "Entangled Truths, Contagious Lies" and "Dark Side Epistemology"
I'm not advocating lying. I'm advocating locally preferring to avoid subjects that force people to either lie or alienate people into preferring lies, or both. In the possible world where The Bell Curve is mostly true, not talking about it on LessWrong will not create a trail of false claims that have to be rationalized. It will create a trail of no claims. LessWrongers might fill their opinion vacuum with false claims from elsewhere, or with true claims, ...
I'm not advocating lying.
I understand that. I cited a Sequences post that has the word "lies" in the title, but I'm claiming that the mechanism described in the cited posts—that distortions on one topic can spread to both adjacent topics, and to people's understanding of what reasoning looks like—can apply more generally to distortions that aren't direct lies.
Omitting information can be a distortion when the information would otherwise be relevant. In "A Rational Argument", Yudkowsky gives the example of an election campaign manager publishing survey re...
"Offensive things" isn't a category determined primarily by the interaction of LessWrong and people of the sneer. These groups exist in a wider society that they're signaling to. It sounds like your reasoning is "if we don't post about the Bell Curve, they'll just start taking offense to technological forecasting, and we'll be back where we started but with a more restricted topic space". But doing so would make the sneerers look stupid, because society, for better or worse, considers The Bell Curve to be offensive and does not consider technological forecasting to be offensive.
You'd have to use a broad sense of "political" to make this true (maybe amounting to "controversial"). Nobody is advocating blanket avoidance of controversial opinions, only blanket avoidance of narrow-sense politics, and even then with a strong exception of "if you can make a case that it's genuinely important to the fate of humanity in the way that AI alignment is important to the fate of humanity, go ahead". At no point could anyone have used the proposed norms to prevent discussion of AI alignment.
Another way this matters: Offense takers largely get their intuitions about "will taking offense achieve my goals" from experience in a wide variety of settings and not from LessWrong specifically. Yes, theoretically, the optimal strategy is for them to estimate "will taking offense specifically against LessWrong achieve my goals", but most actors simply aren't paying enough attention to form a target-by-target estimate. Viewing this as a simple game theory textbook problem might lead you to think that adjusting our behavior to avoid punishment would lead ...
I agree that offense-takers are calibrated against Society-in-general, not particular targets.
As a less-political problem with similar structure, consider ransomware attacks. If an attacker encrypts your business's files and will sell you the encryption key for 10 Bitcoins, do you pay (in order to get your files back, as common sense and causal decision theory agree), or do you not-pay (as a galaxy-brained updateless-decision-theory play to timelessly make writing ransomware less profitable, even though that doesn't help the copy of you in this timeline)?
I...
I think simplifying all this to a game with one setting and two players with human psychologies obscures a lot of what's actually going on. If you look at people of the sneer, it's not at all clear that saying offensive things thwarts their goals. They're pretty happy to see offensive things being said, because it gives them opportunities to define themselves against the offensive things and look like vigilant guardians against evil. Being less offensive, while paying other costs to avoid having beliefs be distorted by political pressure (e.g. taking it elsewhere, taking pains to remember that politically pressured inferences aren't reliable), arguably de-energizes such people more than it emboldens them.
My claim was:
if this model is partially true, then something more nuanced than an absolutist "don't give them an inch" approach is warranted
It's obvious to everyone in the discussion that the model is partially false and there's also a strategic component to people's emotions, so repeating this is not responsive.
I think an important cause of our disagreement is you model the relevant actors as rational strategic consequentialists trying to prevent certain kinds of speech, whereas I think they're at least as much like a Godzilla that reflexively rages in pain and flattens some buildings whenever he's presented with an idea that's noxious to him. You can keep irritating Godzilla until he learns that flattening buildings doesn't help him achieve his goals, but he'll flatten buildings anyway because that's just the kind of monster he is, and in this way, you and Godzi...
The relevant actors aren't consciously being strategic about it, but I think their emotions are sensitive to whether the threat of being offended seems to be working. That's what the emotions are for, evolutionarily speaking. People are innately very good at this! When I babysit a friend's unruly 6-year-old child who doesn't want to put on her shoes, or talk to my mother who wishes I would call more often, or introspect on my own rage at the abject cowardice of so-called "rationalists", the functionality of emotions as a negotiating tactic is very clear to...
standing up to all kinds of political entryism seems to me obviously desirable for its own sake
I agree it's desirable for its own sake, but meant to give an additional argument why even those people who don't agree it's desirable for its own sake should be on board with it.
if for some reason left-wing political entryism is fundamentally worse than right-wing political entryism then surely that makes it not necessarily hypocritical to take a stronger stand against the former than against the latter
Not necessarily objectively hypocritical, but hypocritical in the eyes of a lot of relevant "neutral" observers.
"Stand up to X by not doing anything X would be offended by" is not what I proposed. I was temporarily defining "right wing" as "the political side that the left wing is offended by" so I could refer to posts like the OP as "right wing" without setting off a debate about how actually the OP thinks of it more as centrist that's irrelevant to the point I was making, which is that "don't make LessWrong either about left wing politics or about right wing politics" is a pretty easy to understand criterion and that invoking this criterion to keep LW from being a...
Some more points I want to make:
It would be really nice to be able to stand up to left wing political entryism, and the only principled way to do this is to be very conscientious about standing up to right wing political entryism, where in this case “right wing” means any politics sufficiently offensive to the left wing, regardless of whether it thinks of itself as right wing.
"Stand up to X by not doing anything X would be offended by" is obviously an unworkable strategy, it's taking a negotiating stance that is maximally yielding in the ultimatum game, so should expect to receive as ...
I agree that LW shouldn't be a zero-risk space, that some people will always hate us, and that this is unavoidable and only finitely bad. I'm not persuaded by reasons 2 and 3 from your comment at all in the particular case of whether people should talk about Murray. A norm of "don't bring up highly inflammatory topics unless they're crucial to the site's core interests" wouldn't stop Hanson from posting about ems, or grabby aliens, or farmers and foragers, or construal level theory, or Aumann's theorem, and anyway, having him post on his own blog works fin...
My observation tells me that our culture is currently in need of marginally more risky spaces, even if the number of safe spaces remains the same.
Our culture is desperately in need of spaces that are correct about the most important technical issues, and insisting that the few such spaces that exist have to also become politically risky spaces jeopardizes their ability to function for no good reason given that the internet lets you build as many separate spaces as you want elsewhere.
Our culture is desperately in need of spaces that are correct about the most important technical issues
I also care a lot about this; I think there are three important things to track.
First is that people might have reputations to protect or purity to maintain, and so want to be careful about what they associate with. (This is one of the reasons behind the separate Alignment Forum URL; users who wouldn't want to post something to Less Wrong can post someplace classier.)
Second is that people might not be willing to pay costs to follow taboos. The more a spac...
I’m going to be a little nitpicky here. LW is not “becoming,” but rather already is a politically risky space, and has been for a long time. There are several good reasons, which I and others have discussed elsewhere here. They may not be persuasive to you, and that’s OK, but they do exist as reasons. Finally, the internet may let you build a separate forum elsewhere and try to attract participants, but that is a non-trivial ask.
My position is that accepting intellectual risk is part and parcel of creating an intellectual environment capable of maintaining...
And so you need to make a pitch not just "this pays for itself now" but instead something like "this will pay for itself for the whole trajectory that we care about, or it will be obvious when we should change our policy and it no longer pays for itself."
I don't think it will be obvious, but I think we'll be able to make an imperfect estimate of when to change the policy that's still better than giving up on future evaluation of such tradeoffs and committing reputational murder-suicide immediately. (I for one like free speech and will be happy to advocate for it on LW when conditions change enough to make it seem anything other than pointlessly self-destructive.)
Since somebody was wondering if it's still possible to participate without having signed up through alignmentjam.com:
Yes, people are definitely still welcome to participate today and tomorrow, and are invited to head over to Discord to get up to speed.