The standard formulation of Newcomb's problem has always bothered me, because it seemed like a weird hypothetical designed to make people give the wrong answer. When I first saw it, my immediate response was that I would two-box, because really, I just don't believe in this "perfect predictor" Omega. And while it may be true that Newcomblike problems are the norm, most real situations are not so clear cut. It can be quite hard to demonstrate why causal decision theory is inadequate, let alone build up an intuition about it. In fact, the closest I've seen to a real-worl... (Read more)

A few months ago, Olivier Bailleux, a Professor of computer science and reader of my book on Bayesianism, sent me an email. He suggested to apply some of the ideas of the book to examine students. He proposed Bayesian examination.

I believe it to be a brilliant idea, which could have an important impact on how many people think. At least, I think that this is surely worth sharing here.

tl;dr Bayesian examinations seem very important to deploy because they incentivize both probabilistic thinking and intellectual honesty. Yet, as argued by Julia Galef in this talk, incentives seem critical to cha... (Read more)

Applying economic models to physiology seems really obvious. For instance:

• Surely the body uses price signals to match production to consumption of various metabolites. Insulin as a price signal for glucose is one example.
• Presumably such price signals coordinate between spatially-separated organs with specialized roles in various physiological "supply chains". That should lead to general equilibrium models, and questions of convexity and stability.
• Can we back out an implied discount rate for the body's long-term energy stores?

Yet when I run a google search for the obvious phrase ... (Read more)

This is a brief review of On the Chatham House Rule by Scott Garrabrant.

I tend to be very open about my thoughts and beliefs. However, I naturally am still discrete about a lot of things - things my friends told me privately, personal things about myself, and so on.

This has never been a big deal, figuring out the norms around secrecy. For most of my life it's seemed pretty straightforward, and I've not had problems with it. We all have friends who tell us things in private, and we're true to our word. We've all discovered a fact about someone that's maybe a bit embarrassing or personal, where

The dominant model about status in LW seems to be one of relative influence. Necessarily, it's zero-sum. So we throw up our hands and accept that half the community is just going to run a deficit.

Here's a different take: status in the sense of worth. Here's a set of things we like, or here's a set of problems for you to solve, and if you do, you will pass the bar and we will grant you personhood and take you seriously and allow you onto the ark when the world comes crumbling. Worth is positive-sum.

I think both models are useful, but only one of these models underlies the em... (Read more)

In 2008, Steve Omohundro's foundational Basic AI Drives made important conjectures about what superintelligent goal-directed AIs might do, including gaining as much power as possible to best achieve their goals. Toy models have been constructed in which Omohundro's conjectures bear out, and the supporting philosophical arguments are intuitive. The conjectures have recently been the center of debate between well-known AI researchers.

Instrumental convergence has been heuristically understood as an anticipated risk, but not as a formal phenomenon with a well-understood cause. The goal of this pos

Of course, I'm not expecting you to support the idea in the answers, but simply mentioning its conclusion:)

Suppose a general-population survey shows that people who exercise less, weigh more. You don't have any known direction of time in the data - you don't know which came first, the increased weight or the diminished exercise. And you didn't randomly assign half the population to exercise less; you just surveyed an existing population.

The statisticians who discovered causality were trying to find a way to distinguish, within survey data, the direction of cause and effect - whether, as common sense would have it, more obese people exercise less because they find physical activity less rewarding; o... (Read more)

What it says on the tin.

Lately I've come to think of human civilization as largely built on the backs of intelligence and virtue signaling. In other words, civilization depends very much on the positive side effects of (not necessarily conscious) intelligence and virtue signaling, as channeled by various institutions. As evolutionary psychologist Geoffrey Miller says, "it’s all signaling all the way down."

A question I'm trying to figure out now is, what determines the relative proportions of intelligence vs virtue signaling? (Miller argued that intelligence signaling can be considered a kind of virtue signaling, but

What is voting theory?

Voting theory, also called social choice theory, is the study of the design and evaulation of democratic voting methods (that's the activists' word; game theorists call them "voting mechanisms", engineers call them "electoral algorithms", and political scientists say "electoral formulas"). In other words, for a given list of candidates and voters, a voting method specifies a set of valid ways to fill out a ballot, and, given a valid ballot from each voter, produces an outcome.

(An "electoral system" includes a voting method... (Read more)

This post is eventually about partial agency. However, it's been a somewhat tricky point for me to convey; I take the long route. Epistemic status: slightly crazy.

I've occasionally said that everything boils down to credit assignment problems.

One big area which is "basically credit assignment" is mechanism design. Mechanism design is largely about splitting gains from trade in a way which rewards cooperative behavior and punishes uncooperative behavior. Many problems are partly about mechanism design:

• Building functional organizations;
• Designing markets to solve problems (suc

The Cybersecurity Dilemma: Hacking, Trust, and Fear Between Nations organizes its arguments in a fairly no-nonsense, premise –> premise –> conclusion manner that I thought would be good to summarize with a blog post. You can buy the book here.

The argument:

Security dilemmas are dangerous because in seeking their own security, states often build capabilities and take actions that can directly threaten the security of other states: often creating impressions of imminent offensive intent and prompting escalation. This is especially true for cybersecurity for several main reason... (Read more)

Cross-posted from Putanumonit where the images show up way bigger. I don't know how to make them bigger on LW.

Basketballism

Imagine that tomorrow everyone on the planet forgets the concept of training basketball skills.

The next day everyone is as good at basketball as they were the previous day, but this talent is assumed to be fixed. No one expects their performance to change over time. No one teaches basketball, although many people continue to play the game for fun.

Snapshots of a Steph Curry jump shot. Image credit: ESPN.

Geneticists explain that some people are born with better hand-eye... (Read more)