Second version, updated for the 2018 Review. See change notes.
There's a concept which many LessWrong essays have pointed at it (indeed, I think the entire sequences are exploring). But I don't think there's a single post really spelling it out explicitly:
You might want to become a more robust, coherent agent.
By default, humans are a kludgy bundle of impulses. But we have the ability to reflect upon our decision making, and the implications thereof, and derive better overall policies.
Some people find this naturally motivating –it's aesthetically appealing to be a coherent agent. But if you don't find naturally appealing, the reason I think it’s worth considering is robustness – being able to succeed at novel challenges in complex domains.
This is related to being instrumentally rational, but I don’t...
Most AI research focuses on reproducing human abilities: to learn, infer, and reason; to perceive, plan, and predict. There is a complementary problem which (understandably) receives much less attention: if you had these abilities, what would you do with them?
The steering problem: Using black-box access to human-level cognitive abilities, can we write a program that is as useful as a well-motivated human with those abilities?
This post explains what the steering problem is and why I think it’s worth spending time on.
A capable, well-motivated human can be extremely useful: they can work without oversight, produce results that need not be double-checked, and work towards goals that aren’t precisely defined. These capabilities are critical in domains where decisions cannot be easily supervised, whether because they are too...
We’ll start by defining “as useful for X as Hugh,” and then we will informally say that a program is “as useful” as Hugh if it’s as useful for the tasks we care most about.
If a program is useful accomplishing the tasks we care most about, while being horrible for the things we care less about, would the program still be considered useful? For example, suppose I care a lot about music, and just a little about comedy. If an AI was useful for making the music I listen to slightly better, but completely destroyed my ability to get comedy, I'm not sure it's a good idea to call such a thing "useful".
Epistemic Status: Simple point, supported by anecdotes and a straightforward model, not yet validated in any rigorous sense I know of, but IMO worth a quick reflection to see if it might be helpful to you.
A curious thing I've noticed: among the friends whose inner monologues I get to hear, the most self-sacrificing ones are frequently worried they are being too selfish, the loudest ones are constantly afraid they are not being heard, the most introverted ones are regularly terrified that they're claiming more than their share of the conversation, the most assertive ones are always suspicious they are being taken advantage of, and so on. It's not just that people are sometimes miscalibrated about themselves- it's as if the loudest alarm in their heads, the one...
Interesting! Different experiences.
I do want to make it clear that people who are X often acknowledge that they are X, but don't intensely worry about it. E.g. a friend who knows he's abrasive, knows his life would be better if he were less abrasive on the margin, but doesn't have the emotional reaction "oh god, am I being abrasive?" in the middle of social interactions.
Follow-up to: The Intelligent Social Web
Related to: Fake Frameworks
Yesterday I described a framework for viewing culture as a kind of distributed intelligence, and ourselves as nodes in this distributed network.
Today I’d like to share a way of using this framework intentionally that doesn’t require Looking. My main intent here is concreteness: I’d like to illustrate what an application of accounting for the Omega-web can look like. But I also hope this is something some of y’all can benefit from.
I’ll warn up front: this is playing with epistemic fire. I think the skill of clearly labeling when you’re entering and leaving a fake framework is especially important here for retaining epistemic integrity. If you aren’t sure how to do that, or if the prospect of needing to unnerves...
The reference to the Book of the Law was intentional. The reference to chaos magic was not, as that concept had yet to be formulated when I wrote the essay - at least, not out where I could see it.
I myself do not use psychoactives for magical purposes; I've never found it necessary and consider them a rather blunt and chancy instrument. I do occasionally take armodafinil for the nootropic effect, but that is very recent and long postdates the essay.
Voting theory, also called social choice theory, is the study of the design and evaulation of democratic voting methods (that's the activists' word; game theorists call them "voting mechanisms", engineers call them "electoral algorithms", and political scientists say "electoral formulas"). In other words, for a given list of candidates and voters, a voting method specifies a set of valid ways to fill out a ballot, and, given a valid ballot from each voter, produces an outcome.
(An "electoral system" includes a voting method, but also other implementation details, such as how the candidates and voters are validated, how often elections happen and for what offices, etc. "Voting system" is an ambiguous term that can refer to a full electoral system, just to the voting method,...
Hi Jameson, brilliant post. I have some questions regarding the candidates in an election :
1. What are the implicit and explicit assumptions we make about candidates?
2. Would it be possible to incentivize preferable behavior from voters by making the candidates play an anonymous game prior to the election? For example lets say we have 100 candidates that we want to narrow to 10. If we, for example, made the 100 candidates answer 5 pertinent questions in a digital form and had 1000 voters rank order(or other voting function of choice) the responses of 10 random candidates, we could take the top 10 performing candidates and then have a non-anonymous round of voting.
Epistemic status: Fake Framework
When you walk into an improv scene, you usually have no idea what role you’re playing. All you have is some initial prompt — something like:
“You three are in a garden. The scene has to involve a stuffed bear somehow. Go!”
So now you’re looking to the other people there. Then someone jumps forward and adds to the scene: “Oh, there it is! I’m glad we finally found it!” Now you know a little bit about your character, and about the character of the person who spoke, but not enough to fully define anyone’s role.
You can then expand the scene by adding something: “It’s about time! We’re almost late now.” Now you’ve specified more about what’s going on, who you are, and who the other...
Characters often want change as part of their role. And just as importantly, their role often requires that they can't achieve that change. The tension between craving and deprivation gives birth to the character's dramatic raison d'être. The "wife" can't be as clingy and anxious if the "husband" opens up, so "she" enacts behavior that "she" knows will make "him" close down. "She" can't really choose to change this because "her" thwarted desire for change is part of "her" role.
I'm conflicted about drawing this kind of conclusions from people behaviour, it ... (read more)
I asked why humanity took so long to do anything at the start, and the Internet gave me its thoughts. Here is my expanded list of hypotheses, summarizing from comments on the post, here, and here.
Inventing is harder than it looks
For an in depth argument that could taken to support this point, I highly recommend Humankind: A Hopeful History by Rutger Bregman.
(Cross-posted from Facebook.)
Now and then people have asked me if I think that other people should also avoid high school or college if they want to develop new ideas. This always felt to me like a wrong way to look at the question, but I didn't know a right one.
Recently I thought of a scary new viewpoint on that subject.
This started with a conversation with Arthur where he mentioned an idea by Yoshua Bengio about the software for general intelligence having been developed memetically. I remarked that I didn't think duplicating this culturally transmitted software would be a significant part of the problem for AGI development. (Roughly: low-fidelity software tends to be algorithmically shallow. Further discussion moved to comment below.)
But this conversation did get me thinking about...
Further comment on re-reading the essay:
I do think EY's central point is a long-observed one. Essentially that over decades the media has increasingly dumbed down in order to appeal to mass audiences; and this perpetuates the dumbness of audiences.
The second half is not so often observed as the first, with regard to the media, though it is in other spheres. For example, in the UK over the last few decades high school and university curricula have dumbed down (due to competition for students between exam boards and new universities), with the obvious effect of producing dumb students.
Cross-posting from 250bpm.com
People who helped Jews during WWII are intriguing. They appear to be some kind of moral supermen. Observe how they had almost nothing to gain and everything to lose. Jewish property was confiscated early on and any portable assets Jews may have had have evaporated quickly. Helping Jews, after a short initial period, wasn't a way to get rich. Hoping for compensation after the war didn't work either. At the time it was not obvious that Nazis will lose. Until last couple of years of WWII it was even probable that they will win. And even if they had lost, hoping for some kind of future appreciation from their anti-Semitic compatriots would be naive. On the other hand, by helping Jews, one put oneself...
The rescuers were just random people hindered by bystander effect like everybody else.
You do not seem to go into the possibility that many people who were asked to help refused to do so, in contrast to the rescuers. Since it was a literal matter of life and death, I believe that many or even most Jewish people did try to ask for help but were declined.
But otherwise a great post, and I was happy to see it included in the Curiosity Book.
"Thou shalt not strike terms from others' expressive vocabulary without suitable replacement." - me
Suppose your friend says: "I don't buy that brand of dip. It's full of chemicals."
Reasonable answer: "I'm skeptical that any of them are harmful in these quantities; we don't have much reason to believe that."
Reasonable answer: "Yellow 5? Are you allergic?"
Reasonable answer: "Okay, let's get the kind with four easily recognizable ingredients."
No: "Technically, everything is chemicals. Dihydrogen monoxide!"
Pedantry is seldom a way to make friends and influence people, but this example particularly gets my goat because there doesn't seem to actually exist a word in English for the thing you know perfectly well people mean when they say "chemicals". When I tried to find one on Twitter, the closest...
sometimes people think of things as being either X or Y, and then learn an argument for why this dichotomy doesn't make sense. As a result, they might reject the dichotomy entirely
This reminds me of the Fallacy of Gray.
Epistemic status: pretty confident. Based on several years of meditation experience combined with various pieces of Buddhist theory as popularized in various sources, including but not limited to books like The Mind Illuminated, Mastering the Core Teachings of the Buddha, and The Seeing That Frees; also discussions with other people who have practiced meditation, and scatterings of cognitive psychology papers that relate to the topic. The part that I’m the least confident of is the long-term nature of enlightenment; I’m speculating on what comes next based on what I’ve experienced, but have not actually had a full enlightenment. I also suspect that different kinds of traditions and practices may produce different kinds of enlightenment states.
While I liked Valentine’s recent post on kensho and its follow-ups a lot,...
Pain as information about option value is a nice compact frame. Thanks.
Epistemic status: political, opinionated, personal, all the typical caveats for controversial posts.
I was talking with a libertarian friend of mine the other day about my growing discomfort with the political culture in the Bay Area, and he asked why I didn't just move.
It's a good question. Peter Thiel just moved to L.A., citing the left-wing San Francisco culture as his reason.
But I like living in the Bay, and I don't plan to go anywhere in the near future. I could have said that I'm here for the tech industry, or here because my friends are, or any number of superficially "practical" reasons, but they didn't feel like my real motivation.
What I actually gave as the reason I stay was... aesthetics.
Wait, what?
Let's Talk About Design
I'm not a designer, so...
Late to the game, I read the book...
‘But: imagine if we could talk about why things seem beautiful and appealing, or ugly and unappealing. Where do these preferences come from, in a causal sense? Do we still endorse them when we know their origins?’
There is! These deeper answers lie in the work of architect Christopher Alexander. I found out about him some years ago on Hacker News so he is known in this part of the memosphere. His most general work seems to be the four volume ‘The Nature of Order’. I think he was even a core influence to the de... (read more)
(Cross-posted from Facebook.)
Tl;dr: There's a similarity between these three concepts:
A small note, which would probably have been better before it get published. For someone not following US politics, the story of Doug Jones is hard to follow as I don't have any context about it. My first reading suggest that some people would have wanted to expel a senator, leaving the senate with one less member on the democratic side. But it does not seems to make sens, unless some party have the power to expell a senator from the whole senate
This is part 30 of 30 in the Hammertime Sequence. Click here for the intro.
One of the overarching themes from CFAR, related to The Strategic Level, is that what you learn at CFAR is not a specific technique or set of techniques, but the cognitive strategy that produced those techniques. It follows that if I learned the right lessons from CFAR, then I would be able to produce qualitatively similar – if not as well empirically tested – new principles and approaches to instrumental rationality.
After CFAR, I wanted to design a test to see if I had learned the right lessons. Hammertime was that sort of test for me. Now here’s that same test for you.
I will give three essay prompts and three difficulty levels. Original ideas...
Commenting here to complete my 30 day streak. I'll write up my final exam sometime this week and edit this with a link. I really appreciated the sequence! A lot has happened over the last month and it was nice to have a Hammertime post to return to every day.
edit: Looks like I didn't keep my promise!
Mathematicians answer clean questions that can be settled with formal argument. Scientists answer empirical questions that can be settled with experimentation.
Collective epistemology is hard in domains where it's hard to settle disputes with either formal argument or experimentation (or a combination), like policy or futurism.
I think that's where rationalists could add value, but first we have to grapple with a basic question: if you can't settle the question with logic, and you can't check your intuitions against reality to see how accurate they are, then what are you even doing?
In this post I'll explain how I think about that question. For those who are paying close attention, it's similar to one or two of my previous posts (e.g. 1 2 3 4 5...).
An economist might...
Can you recommend some other posts in that reference class?
In the post 'Four layers of Intellectual Conversation', Eliezer says that both the writer of an idea, and the person writing a critique of that idea, need to expect to have to publicly defend what they say at least one time. Otherwise they can write something stupid and never lose status because they don't have to respond to the criticism.
I was wondering about where this sort of dialogue happens in academia. I have been told by many people that current journals are quite terrible, but I've also heard a romantic notion that science (especially physics and math) used to be more effectively pursued in the early 20th century (Einstein, Turing, Shannon, etc). So Oliver and I thought we'd look at the journals to see if they had...
Something I've seen annoyingly often when digging into the literature is (often critical) reviews of the document I'm after. This is annoying because everything is behind a paywall, and the title often makes it look like I'm getting the document I'm really after, when in fact I'm just getting a 2-page review written by a different person (which I only discover after I've paid for it). I very rarely pay for documents behind paywalls; when I do, it's because the document in question just can't be obtained another way. According to my unreliable memory, at le... (read more)
Wei Dai, one of the first people Satoshi Nakamoto contacted about Bitcoin, was a frequent Less Wrong contributor. So was Hal Finney, the first person besides Satoshi to make a Bitcoin transaction.
The first mention of Bitcoin on Less Wrong, a post called Making Money With Bitcoin, was in early 2011 - when it was worth 91 cents. Gwern predicted that it could someday be worth "upwards of $10,000 a bitcoin". He also quoted Moldbug, who advised that:
If Bitcoin becomes the new global monetary system, one bitcoin purchased today (for 90 cents, last time I checked) will make you a very wealthy individual...Even if the probability of Bitcoin succeeding is epsilon, a million to one, it's still worthwhile for anyone to buy at least a few bitcoins now...I...
This is a very important post to read concerning Bitcoin "billionaires", from one of the guys who helped code its software:
Not as rich as you think… - http://gavinandresen.ninja/not-as-many-as-you-think
It goes like this:
People assume that the people who worked on Bitcoin in the early years are fabulously wealthy.
That’s a bad assumption, for lots of reasons:
Circling is a practice, much like meditation is a practice.
There are many forms of it (again, like there are many forms of meditation). There are even life philosophies built around it. There are lots of intellectual, heady discussions of its theoretical underpinnings, often centered in Ken Wilber's Integral Theory. Subcultures have risen from it. It is mostly practiced in the US and Europe. It attracts lots of New Age-y, hippie, self-help-guru types. My guess is that the median age of practicers is in the 30's. I sometimes refer to practicers of Circling as relationalists (or just Circlers).
In recent years, Circling has caught the eye of rationalists, and that's why this post is showing up here, on LessWrong. I can hopefully direct people here who have...
There may be a small minority of facilitators who do not have this problem. I do not think I, you, or anyone else can, before something goes wrong, pick them out from the crowd of seems-pretty-good facilitators who do have the problem. Especially since charismatic people are better at seeming trustworthy than trustworthy but uncharismatic people are. Individual evaluation, absent an actual record of past behavior to examine, is pretty worthless. And if they are following reasonable counselative ethics*, there will be no record; allowing such a record to be... (read more)
[I am not a sleep specialist. Please consult with one before making any drastic changes or trying to treat anything serious.]
Van Geijlswijk et al describe supplemental melatonin as “a chronobiotic drug with hypnotic properties”. Using it as a pure hypnotic – a sleeping pill – is like using an AK-47 as a club to bash your enemies’ heads in. It might work, but you’re failing to appreciate the full power and subtlety available to you.
Melatonin is a neurohormone produced by the pineal gland. In a normal circadian cycle, it’s lowest (undetectable, less than 1 pg/ml of blood) around the time you wake up, and stays low throughout the day. Around fifteen hours after waking, your melatonin suddenly shoots up to 10 pg/ml – a process called “dim...
The science is solid for naltrexon low dose therapy which is used to up-regulate opiate receptors. The idea is to use small doses of antagonist to make the exiting receptors more sensible after some period of time. The same principle could be applied to other depressants, including melatonin, which start to simulate because of withdrawal effects.
(Cross-posted from Facebook.)
What's this about Inadequate Equilibria's publication?
I actually think it works better as a guide for newbies than as a filter. The people I want to filter on, I typically am able to have long protracted conversations about agency with them anyway, and this blog post isn't the primary way that they get filtered.