Latest Posts
Recent Discussion
Is there an intuitive way to explain how much better superforecasters are than regular forecasters? (I can look at the tables in https://www.researchgate.net/publication/277087515_Identifying_and_Cultivating_Superforecasters_as_a_Method_of_Improving_Probabilistic_Predictions but I don't have an intuitive understanding of what brier scores mean, so I'm not sure what to think about it).
I was told I should start linking my essays on Less Wrong, so here is today's post. This is the first essay in a series about the history of rationality and general semantics.
There is presently no list of posts by date, however the sidebar should have all of the essays (aside from the two most recent ones I haven't added yet).
It's worth noting as well, a lot of my older posts aren't very good and a lot of them I don't fully endorse anymore. Sometime over the next few months I intend to go through and do a review of all my old content, find everything I no longer endorse and indicate that I've updated away from it. For now though, just be aware that everything from before the summer of 2018 is not going to be as good as the stuff that comes later, since that was prior to my major identity death and rebirth event.
[Epistemic status: Argument by analogy to historical cases. Best case scenario it's just one argument among many. Edit: Also, thanks to feedback from others, especially Paul, I intend to write a significantly improved version of this post in the next two weeks.]
I have on several occasions heard people say things like this:
The original Bostrom/Yudkowsky paradigm envisioned a single AI built by a single AI project, undergoing intelligence explosion all by itself and attaining a decisive strategic advantage as a result. However, this is very unrealistic. Discontinuous jumps in technologica... (Read more)
I like your point #2; I should think more about how the 30 year number changes with size. Obviously it's smaller for bigger entities and bigger for smaller entities, but how much? E.g. if we teleported 2020 Estonia back into 1920, would it be able to take over the world? Probably. What about 1970 though? Less clear.
Military power isn't what I'm getting at either, at least not if measured in the way that would result in AI companies having little of it. Cortez had, maybe, 1/10,000th of the military power of Mexico when he got started. At leas... (read more)
I begin by thanking Holden Karnofsky of Givewell for his rare gift of his detailed, engaged, and helpfully-meant critical article Thoughts on the Singularity Institute (SI). In this reply I will engage with only one of the many subjects raised therein, the topic of, as I would term them, non-self-modifying planning Oracles, a.k.a. 'Google Maps AGI' a.k.a. 'tool AI', this being the topic that requires me personally to answer. I hope that my reply will be accepted as addressing the most important central points, though I did not have time to explore every avenue. I certainly do ... (Read more)
[Eli's personal notes for personal understanding. Feel free to ignore or engage.]
Eli's proposed AGI planning-oracle design:
The AGI has four parts:
- A human model
- A NLP "request parser"
- A "reality simulator" / planning module, that can generate plans conditioning on certain outcomes.
- A UX system that outputs plans and outcomes
Here's how it works:
1. A user makes a request of the system, by giving some goal that the user would like to achieve, like "cure cancer". This request is phrased in natural language, and can in... (read more)
Introduction
When you're trying to learn martial arts, usually you go to a physical location and get instructed by a real live person in real time. However, there are also books on martial arts. It is unknown to me whether or not anyone can actually learn anything useful about martial arts just by reading a book, even if they practice everything they have read.
When you're learning something in meat space, the information is transferred from one mind into words and then into another mind. The other mind can confirm or deny understanding, paraphrase the content, ask for examples, etc. The first
... (Read more)(There's some relation to the sunk cost fallacy here, in the sense that theoretically you should search equally hard for understanding after you've already paid no matter how much it costed. However, human brains don't actually work like that, so I think that this extension of the concept is warranted.)
I've seen this argument, and while I acknowledge it might be true for some people, I have no reason to believe that this isn't mistaken correlation* - if you pay more for something you probably care more about it. (Though the Ikea ef... (read more)
The We Want MoR podcast, a chapter by chapter read-through and discussion of Harry Potter and the Methods of Rationality by a Rationalist and a newbie, has just finished up Book One (Ch. 1-21) of HPMOR. (Apple Podcasts link.)
Less Wrong used to be a major locus of discussion for HPMOR, so I thought it made sense to share this here, especially now that the show has passed this milestone. If folks don't necessarily want to keep HPMOR content on Less Wrong, there are ongoing lively discussions of the weekly episodes on the r/hpmor subreddit.
Disclosure, I was the guest on this episode.
The Effective Altruism Foundation (EAF) is focused on reducing risks of astronomical suffering, or s-risks, from transformative artificial intelligence (TAI). S-risks are defined as as events that would bring about suffering on an astronomical scale, vastly exceeding all suffering that has existed on earth so far[1]. As has been discussed elsewhere, s-risks might arise by malevolence, by accident, or in the course of conflict.
We believe that s-risks arising from conflict are among the most important, tractable, and neglected of these. In particular, strategic threats by powerful AI agents or A
... (Read more)There don't seem to be many plausible paths to s-risks: by default, we shouldn't expect them, because it would be quite surprising for an amoral AI system to think it was particularly useful or good for humans to _suffer_, as opposed to not exist at all, and there doesn't seem to be much reason to expect an immoral AI system.
I think this is probably false, but it's because I'm using the strict definition of s-risk.
I expect to the extent that there's any human-like stuff, or animal-like stuff in the future, the fact that there ... (read more)
If it’s worth saying, but not worth its own post, here's a place to put it. (You can also make a shortform post)
And, if you are new to LessWrong, here's the place to introduce yourself. Personal stories, anecdotes, or just general comments on how you found us and what you hope to get from the site and community are welcome.
If you want to explore the community more, I recommend reading the Library, checking recent Curated posts, seeing if there are any meetups in your area, and checking out the Getting Started section of the LessWrong FAQ.
The Open Thread sequence is here.
Scott Anderson
Can I infer via nominative determinism that Scott Anderson is a friend of the rationalist community?
(This is not your typical factual-answer question, but I think it makes sense to format this as a question rather than a post.)
TLDR: Recommend some posts for a "practice of rationality" sequence I want to curate! Proposing posts that should exist but don't is also cool.
I've been thinking recently that it would be nice if rationality were more associated with a practice -- a set of skills which you can keep grinding and leveling up. Testable rationality skills (like accuracy or calibration in forecasting) are obviously a plus, but I'm not referring exclusively to this -... (Read more)
I really like this framing, and it resonates a lot with how I personally think about my orientation to the world.
There are still two elephants in the room that I must address before concluding. Then I will discuss paths forward.
Moloch’s Army
The first elephant is Moloch’s Army. I still can’t find a way into this without sounding crazy. The result of this is that the sequence talks about maze behaviors and mazes as if their creation and operation are motivated by self-interest. That’s far from the whole picture.
There is mindset that instinctively and unselfishly opposes everything of value. This mindset is not only not doing calculations to see what it would prefer or might accomplish. It does not even ... (Read more)
Also feels related: The Return of Conservative Economics (a)
This post was written for Convergence Analysis. Cross-posted to the EA Forum.
The concept of information hazards is highly relevant to many efforts to do good in the world (particularly, but not only, from the perspective of reducing existential risks). I’m thus glad that many effective altruists and rationalists seem to know of, and refer to, this concept. However, it also seems that:
- People referring to the concept often don’t clearly define/explain it
- Many people (quite understandably) haven’t read Nick Bostrom’s original (34 page) paper on the subject
- Some people misunderstand or misuse th
Do people generally consider wireheading and Goodheart's law as information hazards? They're both "errors" caused by access to true data, but that is easy to misuse.
To the tune of "Big Yellow Taxi", with apologies to Joni Mitchell
we'll pave paradise and put up a parking lot
if that's what it takes to make all the wild suffering stop
the pain it only grows and grows
until we boil it up in our pot
we'll pave paradise and put up a parking lot
we'll take all the creatures and put 'em in a simulation
where there's no pain or death or starvation
oh! the pain it only grows and grows
until we boil it up in our pot
we'll pave paradise and put up a parking lot
hey all you people put away decision theory
i don't care about max utils if we can't end suffering, please!
(I feel some obligation to note that I don't endorse the ideas in the song, but do find it amusing)
TL;DR: You are invited to join us online on Saturday the 29th, to write that blog post you've been thinking about writing but never got around to. Comment if you accept this invitation, so I can gauge interest.
The Problem:
Like me, you are too scared and/or lazy to write up this idea you've had. What if it's not good? I started a draft but... Etc.
The Solution:
1. Higher motivation via Time Crunch and Peer Encouragement
We'll set an official goal of having the post put up by midnight. Also, we'll meet up in a special-purpose discord channel to chat, encourage each other, s... (Read more)
I'll participate (either on that day or possibly an adjaecent one if I end up with scheduling conflicts)
I'm reading Superforecasters and one of the things that differentiates good from bad forecasters is ideology: those who try to fit the world in their left/right wing view are less effective at forecasting.
Does Ideology still has a place in a rational world? Why (not)?
My limited perspective tells me in theory ideology would give you ideas to try and would bias potential solutions.
TL:DR;
My limited perspective tells me in theory ideology would give you ideas to try and would bias potential solutions.
Theories also give you ideas to try. Is biasing potential solutions a good thing or a bad thing?
Long:
What is ideology?
I'll try to offer an answer here. (For the purposes of this comment "Ideology" is used negatively.)
Here's a frame:
A) Ideology: The way the world is + Things to do. Immutable 'Theory' that in the simplest case flows from one's self, multiple sources leads to complications and can involve i... (read more)
Is there software for goal factoring (the CFAR technique)? I want to use it to create a directed acyclic graph, which is not necessarily a tree (this requirement disqualifies mind mapping software). Nodes are goals, they have attached text. There's an edge from goal x to goal y iff fulfillment of x directly contributes to fulfillment of y. There should be an easy way to see the graph and modify it, preferrably in a visual way.
To my knowledge, Yed is the least worst existing graphing software.
I use it, as does Andrew Critch, and people at Leverage / Paradigm, generally.
"Allow me to make sure I have this straight," the hero said. "I've been untimely ripped from my home world to fight unspeakable horrors, and you say I'm here because I'm lucky?"
Aerhien dipped her eyelashes in elegant acknowledgment; and quietly to herself, she thought: Thirty-seven. Thirty-seven heroes who'd said just that, more or less, on arrival.
Not a sign of the thought showed on her outward face, where the hero could see, or the other council members of the Eerionnath take note. Over the centuries since her accidental immortality she'd built a reputation for sereni... (Read more)
I would love to know what you would be trying.
I always wash my hands but typically not very well; I never worked in healthcare and never got trained. Given the current situation I decided to actually research how to do it properly.
Different sources differ slightly in their recommendations but the basic gist is the same. Some takeaways:
- Washing your hands should take ~ 20-30 seconds. If it's taking you much less you're not doing it right.
- Make sure you actually get all your hand surfaces, including your nails and between your fingers. The WHO guide has a good protocol that's probably worth memorising.
- If possible, try to dry your hands wi
Droplets don't usually float around very long. It seems most transmission would still require something going into a person's mouth. A small portion of these droplets hang around in the air and can be breathed in, but poor hygiene seems like a far bigger issue given how often people touch their face, mouth, and food.
In the last few months, I've gotten increasingly alarmed by leftist politics in the US, and the epistemic conditions that it operates under and is imposing wherever it gains power. (Quite possibly the conditions are just as dire on the right, but they are not as visible or salient to me, because most of the places I can easily see, either directly or through news stories, i.e., local politics in my area, academia, journalism, large corporations, seem to have been taken over by the left.)
I'm worried that my alarmism is itself based on confirmation bias, tribalism, catastrophizing, or any number
... (Read more)Probably it makes more sense to prepare for scenarios where ideological fanaticism is widespread but isn't wielding government power.
I find discussions about AI takeoff to be very confusing. Often, people will argue for "slow takeoff" or "fast takeoff" and then when I ask them to operationalize what those terms mean, they end up saying something quite different than what I thought those terms meant.
To help alleviate this problem, I aim to compile the definitions of AI takeoff that I'm currently aware of, with an emphasis on definitions that have clear specifications. I will continue updating the post as long as I think it serves as a useful reference for others.
In this post, an AI takeoff can be ro... (Read more)
This is a fantastic set of definitions, and it is definitely useful. That said, I want to add something to what you said near the end. I think the penultimate point needs further elaboration. I've spoken about "multi-agent Goodhart" in other contexts, and discussed why I think it's a fundamentally hard problem, but I don't think I've really clarified how I think this relates to alignment and takeoff. I'll try to do that below.
Essentially, I think that the question of multipolarity versus individual or collective takeoff ... (read more)
As a sophomore undergraduate student, my most valuable rewards from the college experience have come from personal growth, rather than the classroom. However, one problem that I can't seem to shake is dealing with all the subcategories of my total personality.
On the one hand, I am hyper intellectual, sometimes annoyingly so, because I have an overwhelming number of ideas--all under the vague category of "philosophy." But this side of me has produced the purest, most profound joy that I have ever experienced, and it offers the most promise for a successful career.
On the other ha... (Read more)
I used to be terribly distracted by video games.
I can't pinpoint the exact thing that happened that let me really cut down on that, but some things I did that all seemed to lead to my current state of playing video games 0-3 hours per week.
1. Uninstalled all games and game distribution services.
2. Downgraded my internet connection to something that makes downloading games take a long time. (I've since upgraded my internet connection but haven't had a "relapse")
3. Unsubscribed from video gaming RSS feeds.
4. Gotten older!
This AI impacts article includes three intuitive ways to think about the findings.