Latest Posts
Recent Discussion
As always, cross-posted from Putanumonit.
From Tokyo to TriBeCa, people are increasingly alone. People go on fewer dates, marry less and later, have smaller families if at all. People are having less sex, especially young people. The common complaint: it’s just too hard. Dating is hard, intimacy is hard, relationships are hard. I’m not ready to play on hard mode yet, I’ll do the relationship thing when I level up.
And simultaneously, a cottage industry sprung up extolling the virtue of loneliness. Self-care, self-development, self-love. Travel solo, live solo, you do you. Wait, doesn’t that la
... (Read more)Cross-posted to the EA Forum. For an epistemic status statement and an outline of the purpose of this sequence of posts, please see the top of my prior post. There are also some explanations and caveats in that post which I won’t repeat - or will repeat only briefly - in this post.
Purpose of this post
In my prior post, I wrote:
... (Read more)We are often forced to make decisions under conditions of uncertainty. This uncertainty can be empirical (e.g., what is the likelihood that nuclear war would cause human extinction?) or moral (e.g., does the wellbeing of future generations matter morally?). The issue
Will look and see if I can add some useful thought there. Thanks.
Many approaches to AI alignment require making assumptions about what humans want. On a first pass, it might appear that inner alignment is a sub-component of AI alignment that doesn't require making these assumptions. This is because if we define the problem of inner alignment to be the problem of how to train an AI to be aligned with arbitrary reward functions, then a solution would presumably have no dependence on any particular reward function. We could imagine an alien civilization solving the same problem, despite using very different reward functions to train their AIs.
Unfortunatel... (Read more)
It seems like if we want to come up with a way to avoid these types of behavior, we simply must use some dependence on human values. I can't see how to consistently separate acceptable failures from non-acceptable ones except by inferring our values.
I think people should generally be a little more careful about saying "this requires value-laden information". First, while a certain definition may seem to require it, there may be other ways of getting the desired behavior, perhaps through reframing. Building an AI which only does small things should not r
... (read more)Continuing the experiment from August, let's try another open thread for AI Alignment discussion. The goal is to be a place where researchers and upcoming research can ask small questions they are confused about, share early stage ideas and have lower-key discussions.
Ahh. To be honest, I read that, but then responded to something different. I assumed you were just expressing general pessimism, since there’s no guarantee that we would converge on good values upon a long reflection (and you recently viscerally realized that values are very arbitrary).
I guess I was also expressing a more general update towards more pessimism, where even if nothing happens during the Long Reflection that causes it to prematurely build an AGI, other new technologies that will be available/deployed during the Long Reflection could also in
... (read more)One thing I've noticed recently is that when someone complains about how a certain issue "just keeps happening" or they "keep having to deal with it", it often seems to indicate an unsolved problem that people may not be aware of. Some examples:
- Players of a game repeatedly ask the same rules questions to the judges at an event. This doesn't mean everyone is bad at reading -- it likely indicates an area of the rules that is unclear or misleadingly written.
- People keep trying to open a door the wrong way, either pulling on a door that's supposed to be pushed or
I'm not sure that's at odds with what mscottveach is saying. To put it in different words, while the amount of feedback might vary, I don't think the ratio of positive vs. negative feedback varies. It's the very rare situation where the number of messages that say, "This was good, everything went as planned or intended," outnumbers the messages that talk about how something went wrong.
Oh, I quite disagree. I've often found it normal for people to give positive feedback only or where positive feedback far outweighs negati... (read more)
Welcome!
This is part 1 of a series of posts I initially planned to organize as a massive post last summer on principal-agent problems. As that task quickly became overwhelming, I decided to break it down into smaller posts that ensure I cover each of the cases and mechanisms that I intended to.
Overall, I think the trade-off between the alignment of agents and the competence of agents can explain a lot of problems to which people often think there are simple answers. The less capable an agent is (whether the agent is a person, a bureaucracy, or an algorithm) the easier it is for a principal to assess t... (Read more)
I think those two cases are pretty compatible. The simple rules seem to get formed due to the pressures created by large groups, but there are still smaller sub-groups within large groups than can benefit from getting around the inefficiency caused by the rules, so they coordinate to bend the rules.
Hanson also has an interesting post on group size and conformity: http://www.overcomingbias.com/2010/10/towns-norm-best.html
In the vegan case, it is easier to explain things to a small number of people than a large number of people, even though it may still n... (read more)
Why has there never been a "political Roko's basilisk", i.e. a bill or law that promises to punish any member of parliament who voted against it (or more generally any individual with government power, e.g. judge or bureaucrat, who did not do everything in their capacity to make it law)?
Even if unconstitutionality is an issue, it seems like the "more general" condition would prevent judges from overturning it, etc. And surely there are countries with all-powerful parliaments.
A lot of these examples are distinct from Roko's idea, in that they are self-reinforcing, but generally through other mechanisms than distinguishing supporters from non-supporters and targeting those groups specifically.
There's a pretty strong governance norm (and in many cases constitutional protection) against this kind of segregation and targeting, at least in nominally-free democratic societies. A politician who puts opponents in jail JUST because they are opposed (or proposes a law that punishes ONLY those who oppose it) won't last lon... (read more)
The following is a fictional dialogue building off of AI Alignment: Why It’s Hard, and Where to Start.
(Somewhere in a not-very-near neighboring world, where science took a very different course…)
ALFONSO: Hello, Beth. I’ve noticed a lot of speculations lately about “spaceplanes” being used to attack cities, or possibly becoming infused with malevolent spirits that inhabit the celestial realms so that they turn on their own engineers.
I’m rather skeptical of these speculations. Indeed, I’m a bit skeptical that airplanes will be able to even rise as high as stratospheric weather balloons anytime... (Read more)
For what it's worth, I was just learning about the basics of MIRI's research when this came out, and reading it made me less convinced of the value of MIRI's research agenda. That's not necessarily a major problem, since the expected change in belief after encountering a given post should be 0, and I already had a lot of trust in MIRI. However, I found this post by Jessica Taylor vastly clearer and more persuasive (it was written before "Rocket Alignment", but I read "Rocket Alignment" first). In particular, I would ... (read more)
All else being equal, arms races are a waste of resources and often an example of the defection equilibrium in the prisoner’s dilemma. However, in some cases, such capacity races may actually be the globally optimal strategy. Below I try to explain this with some examples.
1: If the U.S. kept racing in its military capacity after WW2, the U.S. may have been able to use its negotiating leverage to stop the Soviet Union from becoming a nuclear power: halting proliferation and preventing the build up of world threatening numbers of high yield weapons. Basically, the earlier you win an arms race... (Read more)
I think capacity races to deter deployment races are the best from this perspective: develop decisive capability advantages, and all sorts of useful technology, credibly signal that your could use it for coercive purposes and could deploy it at scale, then don't abuse the advantage, signal good intent, and just deploy useful non-coercive applications. The development and deployment process basically becomes an escalation ladder itself, where you can choose to stop at any point (though you still need to keep people employed/trained to sustain credibili... (read more)
Cross-posted from Putanumonit.
A couple of weeks ago Venkatesh challenged his followers to brainstorm at least 100 tweets on a topic via live responses. Since I’m not an expert on anything in particular, I decided to simply see if I can come up with 100 discrete pieces of life advice in a day.
This off-the-cuff game turned into perhaps the most successful creative project I’ve ever done. The thread was viewed by tens of thousands of people, received thousands of likes, and gained me hundreds of Twitter followers. I didn’t know there was such thirst for random life-advice, no... (Read more)
As a result of this, I put a post on Nextdoor offering to walk people's dogs for free. I'm hoping someone takes me up on it. Thanks for the brilliant suggestion!
LessWrong seems to be a big fan of spaced-repetition flashcard programs like Anki, Supermemo, or Mnemosyne. I used to be. After using them religiously for 3 years in medical school, I now categorically advise against using them for large volumes of memorization.
[A caveat before people get upset: I think they appropriate in certain situations, and I have not tried to use them to learn a language, which seems its most popular use. More at the bottom.]
A bit more history: I and 30 other students tried using Mnemosyne (and some used Anki) for multiple tests. At my school, we have a test approximate
... (Read more)In what program can I create my own unique flashcard repetition algorithm? How?
Previous post: How Escape From Immoral Mazes
Sequence begins here: Moloch Hasn’t Won
The previous posts mostly took mazes as given.
As an individual, one’s ability to fight any large system is limited.
That does not mean our individual decisions do not matter. They do matter. They add up.
Mostly our choice is a basic one. Lend our strength to that which we wish to be free from. Or not do so.
Even that is difficult. The methods of doing so are unclear. Mazes are ubiquitous. Not lending our strength to mazes, together with the goal of keeping one’s metaphorical soul intact and still putting food o... (Read more)
Install new windows for the HQ office, and I'll recommend your company for the contract"
This still feels corrupt to me.
I am a bit confused and thought I'd rather ask and discuss here before thinking about it for long. As usual I am trying to compartmentalize, structure, make distinctions.
My confusion was triggered thinking about the evaluation function (heuristic to rate the certainty to win/loose) in chess. Clearly what it takes is all there on the board, actually the game is already decided based on that state and assuming both player play to force the best possible outcome.
Why do we need to process data when the information is obviously already in the input? (Yes, I know one can make wordy the distinct... (Read more)
Didn't mean to condescend, I was mostly pointing out that complexity is in the iteration of simple rules with a fairly wide branching. I will still argue that all the heuristics and evaluation mechanisms used by standard engines are effectively search predictions, useful only because the full search is infeasible, and because the full search results have not been memoized (in the not-so-giant-by-today's-standards lookup table of position->value).
Epistemic Status: Confident
This idea is actually due to my husband, Andrew Rettek, but since he doesn’t blog, and I want to be able to refer to it later, I thought I’d write it up here.
In many games, such as Magic: The Gathering, Hearthstone, or Dungeons and Dragons, there’s a two-phase process. First, the player constructs a deck or character from a very large sample space of possibilities. This is a particular combination of strengths and weaknesses and capabilities for action, which the player thinks can be successful against other decks/characters or at winning in the game universe. The ... (Read more)
I didn't feel like I fully understood this post at the time when it was written, but in retrospect it feels like it's talking about essentially the same thing as Coherence Therapy does, just framed differently.
Any given symptom is coherently produced, in other words, by either (1) how the individual strives, without conscious awareness, to carry out strategies for safety or well-being; or (2) how the individual responds to having suffered violations of safety or well-being. This model of symptom production is squarely in accord with the construct... (read more)
I've been wanting to get a better example of CDT (causal decision theory) misbehaving, where the behaviour is more clearly suboptimal than it is in the Newcomb problem (which many people don't seem to accept as CDT being suboptimal), and simpler to grasp than Death in Damascus.
The "predictors exist" problem
So consider this simple example: the player is playing against Omega, who will predict their actions[1]. The player can take three actions: "zero", "one", or "leave".
If ever they do "leave", then the experiment is over and they leave. If they choose "zero" or "one", then Omega will predict
... (Read more)Can you clarify what you mean by "successfully formalised"? I'm not sure if I can answer that question but I can say the following:
Stanford's encyclopedia has a discussion of ratifiability dating back to the 1960s and (by the 1980s) it has been applied to both EDT and CDT (which I'd expect, given that constraints on having an accurate world model should be independent of decision theory). This gives me confidence that it's not just a random Less Wrong thing.
Abram Dempski from MIRI has a whole sequence on when CDT=EDT which lev... (read more)
Inspired by my post on problems with causal decision theory (CDT), here is a hacked version of CDT that seems to be able to imitate timeless decision theory (TDT) and functional decision theory[1] (FDT), as well as updateless decision theory (UDT) under certain circumstances.
Call this ACDT, for (a)causal decision theory. It is, essentially, CDT which can draw extra, acausal arrows on the causal graphs, and which attempts to figure out which graph represents the world it's in. The drawback is its lack of elegance; the advantage, if it works, is that it's simple to specify and focuses attention
... (Read more)Thanks! I changed it to:
If the predictor is near-perfect, but the agent models its actions as independent of the predictor (since the prediction was made in the past), then the agent will have some belief about the prediction and will choose the less likely action for expected utility at least 1, and will continually lose.
The problem with the previous agent is that it never learns that it has the wrong causal model. If the agent is able to learn a better causal model from experience, then it can learn that the predictor can actually predict the agent successfully, and so will no longer expect a 50% chance of winning, and it will stop playing the game.
There appears to be something of a Sensemaking community developing on the internet, which could roughly be described as a spirituality-inspired attempt at epistemology. This includes Rebel Wisdom, Future Thinkers, Emerge and maybe you could even count post-rationality. While there are undoubtedly lots of critiques that could be made of their epistemics, I'd suggest watching this space as I think some interesting ideas will emerge out of it.
This may be of interest to people interested in AI Safety. This event is part of the 2020 IEEE Symposium on Security and Privacy and is being sponsored by the Johns Hopkins University Institute for Assured Autonomy.
Previously in sequence and most on point: What is Success in an Immoral Maze?, How to Identify an Immoral Maze
This post deals with the goal of avoiding or escaping being trapped in an immoral maze, accepting that for now we are trapped in a society that contains powerful mazes.
We will not discuss methods of improving conditions (or preventing the worsening of conditions) within a maze, beyond a brief note on what a CEO might do. For a middle manager anything beyond not making the problem worse is exceedingly difficult. Even for the CEO this is an extraordinarily difficult task.
To rescue so... (Read more)
In both cases, I think 'start one's own business' should be at the top of the list. This can be a start-up designed to make a lot of money - and that's by far the highest EV play if you can take a real shot and afford to fail. But it does not need to be something so risky. If you have a trade where you can open a store, or put yourself and perhaps a small number of others out for hire, or even become a consultant of some kind, consider doing one of those before anything else.
Doctor -> private practice. Lawyer -> small law firm as... (read more)
This advice seems conditional on some minimum threshold of attractiveness (social status and physical attractiveness), enough to find hundreds of people willing to go on dates with you within your social circle or on dating apps. This seems like quite a high threshold, or am I missing something? For example the Atlantic article says:
... (read more)