Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.
This is a repository for miscellaneous short things I want to post. Other people are welcome to make top-level comments here if they want. (E.g., questions for me you'd rather discuss publicly than via PM; links you think will be interesting to people in this comment section but not to LW as a whole; etc.)
Shared with permission, a google doc exchange confirming Eliezer still finds the arguments for alignment optimism, slower takeoffs, etc. unconvincing:
... (read more)FWIW, I think Yudkowsky is basically right here and would be happy to explain why if anyone wants to discuss. I'd likewise be interested in hearing contrary perspectives.
Facebook comment I wrote in February, in response to the question 'Why might having beauty in the world matter?':
I assume you're asking about why it might be better for beautiful objects in the world to exist (even if no one experiences them), and not asking about why it might be better for experiences of beauty to exist.
[... S]ome reasons I think this:
1. If it cost me literally nothing, I feel like I'd rather there exist a planet that's beautiful, ornate, and complex than one that's dull and simple -- even if the planet can never be seen or visited by anyone, and has no other impact on anyone's life. This feels like a weak preference, but it helps get a foot in the door for beauty.
(The obvious counterargument here is that my brain might be bad at simulating the scenario where there's literally zero chance I'll ever interact with a thing; or I may be otherwise confused about my values.)
2. Another weak foot-in-the-door argument: People seem to value beauty, and some people claim to value it terminally. Since human value is complicated and messy and idiosyncratic (compare person-specific ASMR triggers or nostalgia triggers or culinary pref... (read more)
Somewhat more meta level: Heuristically speaking, it seems wrong and dangerous for the answer to "which expressed human preferences are valid?" to be anything other than "all of them". There's a common pattern in metaethics which looks like:
1. People seem to have preference X
2. X is instrumentally valuable as a source of Y and Z. The instrumental-value relation explains how the preference for X was originally acquired.
3. [Fallacious] Therefore preference X can be ignored without losing value, so long as Y and Z are optimized.
In the human brain algorithm, if you optimize something instrumentally for awhile, you start to value it terminally. I think this is the source of a surprisingly large fraction of our values.
Rolf Degen, summarizing part of Barbara Finlay's "The neuroscience of vision and pain":
From the paper:
... (read more)[Epistemic status: Thinking out loud]
If the evolutionary logic here is right, I'd naively also expect non-human animals to suffer more to the extent they're (a) more social, and (b) better at communicating specific, achievable needs and desires.
There are reasons the logic might not generalize, though. Humans have fine-grained language that lets us express very complicated propositions about our internal states. That puts a lot of pressure on individual humans to have a totally ironclad, consistent "story" they can express to others. I'd expect there to be a lot more evolutionary pressure to actually experience suffering, since a human will be better at spotting holes in the narratives of a human who fakes it (compared to, e.g., a bonobo trying to detect whether another bonobo is really in that much pain).
It seems like there should be an arms race across many social species to give increasingly costly signals of distress, up until the costs outweigh the amount of help they can hope to get. But if you don't have the language to actually express concrete propositions like "Bob took care of me the last time I got sick, six months ago, and he can attest that I had a hard time walking that time too", then those costly signals might be mostly or entirely things like "shriek louder in response to percept X", rather than things like "internally represent a hard-to-endure pain-state so I can more convincingly stick to a verbal narrative going forward about how hard-to-endure this was".
[Epistemic status: Piecemeal wild speculation; not the kind of reasoning you should gamble the future on.]
Some things that make me think suffering (or 'pain-style suffering' specifically) might be surprisingly neurologically conditional and/or complex, and therefore more likely to be rare in non-human animals (and in subsystems of human brains, in AGI subsystems that aren't highly optimized to function as high-fidelity models of humans, etc.):
1. Degen and Finlay's social account of suffering above.
2. Which things we suffer from seems to depend heavily on mental narratives and mindset. See, e.g., Julia Galef's Reflections on Pain, from the Burn Unit.
Pain management is one of the main things hypnosis appears to be useful for. Ability to cognitively regulate suffering is also one of the main claims of meditators, and seems related to existential psychotherapy's claim that narratives are more important for well-being than material circumstances.
Even if suffering isn't highly social (pace Degen and Finlay), its dependence on higher cognition suggests that it is much more complex and conditional than it might appear on initial introspection, which on its own reduces the probability of it
... (read more)Collecting all of the quantitative AI predictions I know of MIRI leadership making on Arbital (let me know if I missed any):
Some caveats:
- Arbital predictions range from 1% to 99%.
- I assume these are generally ~5 years old. Views may have shifted.
- By default, I assume that the standard caveats for probabilities like these apply: I treat these as off-the-cuff ass numbers unless stated otherwise, products of 'thinking about the problem on and off for years and then querying my gut about what it expects to actually see', more so than of building Guesstimate models or trying to hard to make sure all the probabilities are perfectly coher
... (read more)On my model, the point of ass numbers isn't to demand perfection of your gut (e.g., of the sort that would be needed to avoid multiple-stage fallacies when trying to conditionalize a lot), but to:
It may still be a terrible idea to spend too much time generating ass numbers, since "real numbers" are not the native format human brains compute probability with, and spending a lot of time working in a non-native format may skew your reasoning.
(Maybe there's some individual variation here?)
But they're at least a good tool to use sometimes, for the sake of crisper communication, calibration practice (so you can generate non-awful future probabilities when you need to), etc.
Suppose most people think there's a shrew in the basement, and Richard Feynman thinks there's a beaver. If you're pretty sure it's not a shrew, two possible reactions include:
- 'Ah, the truth is probably somewhere in between these competing perspectives. So maybe it's an intermediate-sized rodent, like a squirrel.'
- 'Ah, Feynman has an absurdly good epistemic track record, and early data does indicate that the animal's probably bigger than a shrew. So I'll go with his guess and say it's probably a beaver.'
But a third possible response is:
- 'Ah, if Feynman's right, then a lot of people are massively underestimating the rodent's size. Feynman is a person too, and might be making the same error (just to a lesser degree); so my modal guess will be that it's something bigger than a beaver, like a capybara.'
In particular, you may want to go more extreme than Feynman if you think there's something systematically causing people to underestimate a quantity (e.g., a cognitive bias -- the person who speaks out first against a bias might still be affected by it, just to a lesser degree), or systematically causing people to make weaker claims than they really believe (e.g., maybe people don't want to sound extreme or out-of-step with the mainstream view).
From Twitter:
I replied:
... (read more)Chana Messinger, replying to Brandon Bradford:
From an April 2019 Facebook discussion:
Rob Bensinger: avacyn:
... I love how EA does veganism / animal welfare things. It's really good.
(From the comment section on https://forum.effectivealtruism.org/posts/TyLxMrByKuCmzZx6b/reasons-to-eat-meat)
[... Note that in posting this I'm not intending] to advocate for a specific intervention; it's more that it makes me happy to see thorough and outside-the-box reasoning from folks who are trying to help others, whether or not they have the same backgr
... (read more)While your comment was clearly written in good faith, it seems to me like you're missing some context. You recommend that EY recommend that the detractors read books. EY doesn't just recommend people read books. He wrote the equivalent of like three books on the subjects relevant to this conversation in particular which he gives away for free. Also, most of the people in this conversation are already big into reading books.
It is my impression he also helped establish the Center for Applied Rationality, which has the explicit mission of training skills. (I'm not sure if he technically did but he was part of the community which did and he helped promote it in its early days.)
From an April 2019 Facebook discussion:
Rob Bensinger:
... (read more)Twitter thread collecting examples of alignment research MIRI has said relatively positive things about.
Copied from some conversations on Twitter:
· · · · · · · · · · · · · · ·
Eric Rogstad: I think "illusionism" is a really misleading term. As far as I can tell, illusionists believe that consciousness is real, but has some diff properties than others believe.
It's like if you called Einstein an "illusionist" w.r.t. space or time.
See my comments here:
https://www.lesswrong.com/posts/biKchmLrkatdBbiH8/book-review-rethinking-consciousness
Rob Bensinger: I mostly disagree. It's possible to define a theory-neutral notion of 'consciousness', but I think it's just true that 'there's no such thing as subjective awareness / qualia / etc.', and I think this cuts real dang deep into the heart of what most people mean by consciousness.
Before the name illusionism caught on, I had to use the term 'eliminativism', but I had to do a lot of work to clarify that I'm not like old-school eliminativists who think consciousness is obviously or analytically fake. Glad to have a clearer term now.
I think people get caught up in knots about the hard problem of consciousness because they try to gesture at 'the fact that they have subjective awareness', without realizing they're gesturing... (read more)
It's apparently not true that 90% of startups fail. From Ben Kuhn:
... (read more)I don't have a cite handy as it's memories from 2014 but when I looked into it I recall the 7 year failure rate excluding the obvious dumb stuff like restaurants was something like 70% but importantly the 70% number included acquisitions, so the actual failure rate was something like 60 ish.
A blurb for the book "The Feeling of Value":
... (read more)Ben Weinstein-Raun wrote on social media:
... (read more)What do you think of Brian Tomasik's flavor of panpsychism, which he says is compatible with (and, indeed, follows from) type-A materialism? As he puts it,
[Epistemic status: Thinking out loud, just for fun, without having done any scholarship on the topic at all.]
It seems like a lot of horror games/movies are converging on things like 'old people', 'diseased-looking people', 'psychologically ill people', 'women', 'children', 'dolls', etc. as particularly scary.
Why would that be, from an evolutionary perspective? If horror is about fear, and fear is about protecting the fearful from threats, why would weird / uncanny / out-of-evolutionary-distribution threats have a bigger impact than e.g. 'lots of human warr
... (read more)The wiki glossary for the sequences / Rationality: A-Z ( https://wiki.lesswrong.com/wiki/RAZ_Glossary ) is updated now with the glossary entries from the print edition of vol. 1-2.
New entries from Map and Territory:
... (read more)https://namespace.obormot.net/Jargon/Jargon
Jeffrey Ladish asked on Twitter:
I replied:
... (read more)From Facebook:
Mark Norris Lance: [...] There is a long history of differential evaluation of actions taken by grassroots groups and similar actions taken by elites or those in power. This is evident when we discuss violence. If a low-power group places someone under their control it is kidnapping. If they assess their crimes or punish them for it, it is mob justice or vigilanteism. [...]
John Maxwell: Does the low power group in question have a democratic process for appointing judges who then issue arrest warrants?
That's a key issue for me... "Mob rule" is
... (read more)From https://twitter.com/JonHaidt/status/1166318786959609856:
... (read more)Yeah, I'm an EA: an Estimated-as-Effective-in-Expectation (in Excess of Endeavors with Equivalent Ends I've Evaluated) Agent with an Audaciously Altruistic Agenda.
How would you feel about the creation of a Sequence of Shortform Feeds? (Including this one?) (Not a mod.)
For being too indistinguishable from GPT-3.