The below is the draft of a blog post I have about why I like AI doom liability. My dream is that people read it and decide "ah yes this is the main policy we will support" or "oh this is bad for a reason Daniel hasn't noticed and I'll tell him why". I think usually you're supposed to flesh out posts, but I'm not sure that adds a ton of information in this case.

Why I like AI doom liability

• AI doom liability is my favourite approach to AI regulation. I want to sell you all on it.

• the basic idea

  • general approach to problems: sue people for the negative impacts
    • internalizes externalities
    • means that the people figuring out how to avoid are informed and aligned (rather than bureaucrats less aware of on-the-ground conditions / trying to look good / seeking power)
    • less fucked than criminal law, regulatory law
      • look at what hits the supreme court, which stuff ends up violating people's rights the worst, what's been more persistent over human history, what causes massive protests, etc.
  • first-pass approach to AI: sue for liabilities after AI takes over
    • can't do that
  • so sue for intermediate disasters, get punitive damages for how close you were to AI takeover
    • intuition: pulling

A complaint about AI pause: if we pause AI and then unpause, progress will then be really quick, because there's a backlog of improvements in compute and algorithmic efficiency that can be immediately applied.

One definition of what an RSP is: if a lab makes observation O, then they pause scaling until they implement protection P.

Doesn't this sort of RSP have the same problem with fast progress after pausing? Why have I never heard anyone make this complaint about RSPs? Possibilities:

• They do and I just haven't seen it
• People expect "AI pause" to produce longer / more serious pauses than RSPs (but this seems incidental to the core structure of RSPs)

I continue to think that agent foundations research is kind of underrated. Like, we're supposed to do mechinterp to understand the algorithm models implement - but how do we know what algorithms are good?

On the most recent episode of the podcast Rationally Speaking, David Shor discusses how members of the USA's Democratic Party could perform better electorally by not talking about their unpopular extreme views, but notes that many individual Democrats have better lives by talking about their unpopular extreme views that are popular with left-wing activists (e.g. because they become more prominent and get to feel good about themselves), which cause some voters to associate those unpopular extreme views with the Democratic Party and not vote for them.

This is discussed as a sad irrationality that constitutes a coordination failure among Democrats, but I found that an odd tone. Part of the model in the episode is that Democratic politicians in fact have these unpopular extreme views, but it would hurt their electoral chances if that became known. From a non-partisan perspective, you'd expect it to be a good thing to know what elected officials actually think. Now, you might think that elected officials shouldn't enact the unpopular policies that they in fact believe in, but it's odd to me that they apparently can't credibly communicate that they won't enact those policies. At any rate, I'm a bit bothered by the idea of coordinated silence to ensure that people don't know what powerful people actually think being portrayed as good.

Models and considerations.

There are two typical ways of deciding whether on net something is worth doing. The first is to come up with a model of the relevant part of the world, look at all the consequences of doing the thing in the model, and determine if those consequences are net positive. When this is done right, the consequences should be easy to evaluate and weigh off against each other. The second way is to think of a bunch of considerations in favour of and against doing something, and decide whether the balance of considerations supports doing the thing or not.

I prefer model-building to consideration-listing, for the following reasons:

• By building a model, you're forcing yourself to explicitly think about how important various consequences are, which is often elided in consideration-listing. Or rather, I don't know how to quantitatively compare importances of considerations without doing something very close to model-building.
• Building a model lets you check which possible consequences are actually likely. This is an improvement on considerations, which are often of the form "such-and-such consequence might occur".
• Building a model lets you notice consequences which you m

A rough and dirty estimate of the COVID externality of visiting your family in the USA for Christmas when you don't feel ill [EDIT: this calculation low-balls the externality, see below]:

You incur some number of μCOVIDs[*] a week, let's call it x. Since the incubation time is about 5 days, let's say that your chance of having COVID is about 5x/7,000,000 when you arrive at the home of your family with n other people. In-house attack rate is about 1/3, I estimate based off hazy recollections, so in expectation you infect 5xn/21,000,000 people, which is about... (read more)

Ted Kaczynski as a relatively apolitical test case for cancellation norms:

Ted Kaczynski was a mathematics professor who decided that industrial society was terrible, and waged a terroristic bombing campaign to foment a revolution against technology. As part of this campaign, he wrote a manifesto titled "Industrial Society and Its Future" and said that if a major newspaper printed it verbatim he would desist from terrorism. He is currently serving eight life sentences in a "super-max" security prison in Colorado.

My understanding is that his manifesto (which... (read more)

Here's a script I wrote to analyze how good Manifold Markets is at predicting Ukraine stuff. Basically: it's about as good as you would be if you were calibrated at 80% accuracy if you average market prices over the life of the market, and if you take the probabilities at the mid-point of the market, it's about as good as you would be if you were calibrated at 72% accuracy.

Some puzzles:

• rubber ducking is really effective
• it's very difficult to write things clearly, even if you understand them clearly

These seem like they should be related, but I don't quite know how. Maybe if someone thought about it for an hour they could figure it out.

Here are two EA-themed podcasts that I think someone could make. Maybe that someone is you!

1. More or Less, but EA (or for forecasting)

More or Less is a BBC Radio program. They take some number that's circulating around the news, and provide context like "Is that literally true? How could someone know that? What is that actually measuring? Is that a big number? Does that mean what you think it means?" - stuff like that. They spend about 10 minutes on each number, and usually include interviews with experts in the field. IMO, someone could do this for numb... (read more)

A sad fact is that good methods to elicit accurate probabilities of the outcome of some future process, e.g. who will win the next election, give you an incentive to influence that outcome, e.g. by campaigning and voting for the candidate you said was more likely to win. But with mind uploading and the 'right' theory of personal identity, we can fix this!

First, suppose that you think of all psychological descendants of your current self as 'you', but you don't think of descendants of your past self as 'you'. So, if you were to make a copy of yourself tomor... (read more)

Avoid false dichotomies when reciting the litany of Tarski.

Suppose I were arguing about whether it's morally permissible to eat vegetables. I might stop in the middle and say:

If it is morally permissible to eat vegetables, I desire to believe that it is morally permissible to eat vegetables. If it is morally impermissible to eat vegetables, I desire to believe that it is morally impermissible to eat vegetables. Let me not become attached to beliefs I may not want.

But this ignores the possibility that it's neither morally permissible nor morally impermi... (read more)

An interesting tension: it's kind of obvious from a micro-econ view that group houses should have Pigouvian taxes on uCOVIDs[*] (where I pay housemates for the chance I get them sick) rather than caps on how many uCOVIDs everyone can incur per week - and of course both of these are better than "just sort of be reasonable" or having no system. But uCOVID caps are nice in that they make it significantly easier to coordinate with other houses - it's much easier to figure out how risky interacting with somebody is when they can just tell you their cap, rather ... (read more)

Research project idea: formalize a set-up with two reinforcement learners, each training the other. I think this is what's going on in baby care. Specifically: a baby is learning in part by reinforcement learning: they have various rewards they like getting (food, comfort, control over environment, being around people). Some of those rewards are dispensed by you: food, and whether you're around them, smiling and/or mimicking them. Also, you are learning via RL: you want the baby to be happy, nourished, rested, and not cry (among other things). And the baby... (read more)

Blog posts I could write up in the next few days:

• My EDC as of late Dec 2022 [EDIT: here]
• thoughts on media I consumed in 2022 (would include Kindle books + stuff I watched on Netflix and Amazon Prime Video)

An argument for stock-picking:

I'm not sure whether I can pick stocks better than the market. But if I can, then money is more valuable to me in that world, since I have better-than-market opportunities in that world but only par-with-market opportunities in the EMH world. So I should buy stocks that look good to me, at least for a while, to check whether I'm in the world where I can do that, because it's a transfer from a world where money is less valuable to me to one where money is more valuable.

I think this argument goes thru if you assume market returns are equal in both worlds, which I think I think.

Excerpts from a FB comment I made about the principle of charity. Quote blocks are a person that I'm responding to, not me. Some editing for coherence has been made. tl;dr: it's fine to conclude that people are acting selfishly, and even to think that it's likely that they're acting selfishly on priors regarding the type of situation they're in.

The essence of charitable discourse is assuming that even your opponents have internally coherent and non-selfish reasons for what they do.

If this were true, then one shouldn't engage in charitable discourse. P... (read more)

A failure of an argument against sola scriptura (cross-posted from Superstimulus)

Recently, Catholic apologist Joe Heschmeyer has produced a couple of videos arguing against the Protestant view of the Bible - specifically, the claims of Sola Scriptura and Perspicuity (capitalized because I'll want to refer to them as premises later). "Sola Scriptura" has been operationalized a few different ways, but one way that most Protestants would agree on is (taken from the Westminster confession):

The whole counsel of God, concerning all things necessary for [...] m