All Comments

Very Unnatural Tasks?

I don't understand the question. Maximizing paperclips at the expense of everything else strikes me as "distinctly un-human", isn't it?

Josephine's Shortform

From what I remember, Selfish Reasons to Have More Kids contains only a very short mention of how to inspire good behavior in children - essentially the advice to punish consistently, especially including funny or endearing offenses. The more I think about this, the more it seems to me that discipline itself is the practice of training the same response to an increasing range of stimuli. 

Progress in Vipassana, and in meditation in general, comes from engaging with fewer and fewer distractions. Progress in habits comes from decreasing the number of times that you allow some override or excuse to change your plans. Progress in security comes from shrinking the number of people you make exceptions for, maybe because you don't want to be mean to them or there are extenuating circumstances. Progress in being a fair parent comes from providing the same consequences for the same actions, even if one of your children's actions gets a warmer reaction from you. 

This isn't to say that discipline is the only virtue, or that you ought to seek discipline for its own sake; some excuses do rise above the threshold of acceptability. But the point of training discipline is, in a nutshell, to have consistent reactions to stimuli that invite inconsistent ones. 

Working With Monsters

Should a country where cryonic preservation is routine try to take over one where it is forbidden?

Or a country where anti-aging medicine delivered in international aid is being stolen and wasted to prevent out-groups from receiving treatment?


It's a moderately interesting question though only because our current moral frameworks privilege "do nothing and let something bad happen" over "do something and cause something bad but less bad to happen".  

It's just the Trolley problem restated.  The solve I have for the trolley problem is viewing the agent in front of the lever as a robotic control system.  Every timestep, the control system must output a control packet, on a CAN or rs-485 bus.  There is nothing special or privileged between a packet that says "keep the actuators in their current position" and "move to flip the lever".  

Therefore the trolley problem vanishes from a moral sense.  From a legal sense, a court of law might try to blame the robot, however.

Working With Monsters

Which is a meta comment on present day I think, where the blue red divide is such that one of those sides clearly aligns with facts and physical reality while another side relies on made up stories.  

Except of course present day, the reason for one of these sides seems almost to be a self identity thing, where they don't really believe in their color's precepts, they just identify with the people in it more so that's the color they fly.

Black ravens and red herrings

There are choices of hypotheses and assumptions about probability distributions.

Good's choice was the hypothesis family "there are i non-black ravens in the universe", uniform prior over these, and an assumption that there are N objects in the universe and observations were drawn uniformly at random from these.

For these assumptions, anything that isn't a non-black raven does carry the same weight for updating the posterior distribution. But the assumptions are obviously false and the hypothesis family doesn't seem very efficient. I wouldn't use these by default.

Delta Strain: Fact Dump and Some Policy Takeaways

According to this specific calculation. He does say his all things considered view is like 5x the cost.

Why Subagents?


there is no way to get from to the “New York (from Philly)” node directly from the “New York (from DC)” node.

Jews and Nazis: a version of dust specks vs torture

This situation is more like "they eat babies, but they don't eat that many, to the extent that it produces net utility given their preferences for continuing to do it."

Torture vs Specks: Sadist version

I'm referring to the fact that utility functions are equivalent under positive affine transformations (if you add a constant and multiply it by a positive constant, the UF remains the same in the sense that it has the same preference in every situation)

Assuming we are computing the utility of an outcome by assigning a utility to each person and then summing them, adding a constant value to any person's utility doesn't change the comparison between outcomes, because the net effect is just to add a constant to the utility of each outcome(as long as the person we are adding a constant value to exists in every outcome).

Therefore, we can convert the situation to negative utilitarian without functionally changing it, by subtracting the maximum utility from each person, ensuring that everyone's utility will be negative in every outcome. We can also convert it to positive utilitarian by subtracting the minimum in a similar way.

This analysis assumes that there is a maximum and a minimum utility, and that every outcome has the same set of people in it, so if these assumptions break there may be relevant differences.

What are some triggers that prompt you to do a Fermi estimate, or to pull up a spreadsheet and make a simple/rough quantitative model?

Hrm, that is a good point. I suppose if I try to be more strict, it's when there is a question of what to do, there are two or more approaches, and there is some difference in quantifiable risk and/or reward between the options, and I haven't already pre-determined a best approach in advance that applies to the situation.

Delta Strain: Fact Dump and Some Policy Takeaways

So Adam is saying microcovids are cheaper than the OP does? Connor writes

1 hour of your life lost every 1k-5k uCOVIDs

(Pardon me commenting while not getting into the details on this important topic, I am busy but am trying to track at least whether there's disagreement and in what direction.)

When writing triggers memory reconsolidation

I'm tempted to write on why one should follow this post's advice so that it actually has an impact on my behavior.

Who decides what should be worked on?

You're right - the reason they review it at all is to retain the power to reject unreasonable proposals (and sometimes to make slight tweaks based on information only they possess).  Sometimes they make the goals more ambitious, too.

90% of the time, though (and 100% of the time in my company's short existence), the proposals coming from below are almost exactly what they wanted - the communication outside of strategic planning exercises is sufficient to ensure that.

Who decides what should be worked on?

The plans are made at the lower levels of the company, and basically rubber-stamped by senior leadership

While this might be true in a healthy company, it's not true in every company and companies where the senior leadership makes decisions while ignoring the lower levels are unpleasant to work in.

Who decides what should be worked on?

I completely agree that execution and a culture of effective execution make a huge difference, but the "visionary" strategic stuff gets written about. 

But is there a distinction to be made here between high-level strategy/direction vs. mid-level planning and low-level execution? It's hard for me to imagine that senior leadership of a company w/ > 50 people just rubber stamps ideas from below. 

For example, senior leadership might say, "this year we're going to grow fruit" and they may accept proposals from below for bananas or oranges or apples. But they wouldn't accept proposals to manufacture cars.

For sure, different companies have different gradations of freedom for mid to lower level employees to engage in decision-making and planning, but it's always a slider and upper level leadership would always hold tightly to the top end of the slider, no? 

Come Build Affordable Housing!

Right, but that impacts whether it's actually profitable to build them.

Cognitive Impacts of Cocaine Use

I very well could be wrong, but I believe that slower response time indicates less impulsivity, which is "better" sustained attention. Here's how the review article I'm summarizing describes the findings:

Cognitive Impacts of Cocaine Use

Better controlled studies found that cocaine dependent participants had mild cognitive impairment and structural differences; however, this was less than the cognitive impairment of alcohol dependent participants. Structural differences were less than psychopathological disorders such as schizophrenia.

This sounds pretty bad? If you really want to take a stimulant there's many others out there that don't have studies which show that they cause cognitive impairment. "Better than being an alcoholic or schizophrenic" doesn't sound like much of an endorsement to me.

Josephine's Shortform

I'm impressed by how accurately this describes learning complex skills.

I'm practicing writing and I feel the same way most of the points describe: as if I'm exploring a system of caves without a map, finding bits and pieces of other explorers (sometimes even meeting them), but it's all walking a complicated, 3d structure and constantly bumping into unknown unknowns. Let me illustrate it this way: about 3 years ago, when I started on this journey, I thought I would read 1-2 books about writing and I'll be good. Now, I'm standing in sub-cave system #416, taking a hard look at "creativity"/"new ideas" and chuckling at my younger self who thought that sub-cave system #18 "good sentences" will lead him to the exit.

And even though I haven't practices Brazilian Jiu Jitsu since the pandemic began, I see a lot of similarities there. At first, I thought I just have to practice a move. Then I noticed that there are many small variations depending on my energy level, the opponents size and weight, etc. Then I noticed that I could fake moves to lure my opponent into making mistakes, but I should avoid mistakes myself. Then I noticed that my opponents were better in at some moves than others. Then I noticed that my own build gave me certain advantages and disadvantages. Then I noticed...

At the end, just before the lockdowns, I learned a lot about humility and began to discard all the "factual knowledge" I got from youtube videos or books and instead began focusing on sets of small details to explore how they worked in different situations. Then, just practiced it over and over until I saw "the thing".

Delta Strain: Fact Dump and Some Policy Takeaways

The calculation would be pretty straightforward: 

3.5 days mean, let's assume another 2 days or so in terms of sickness, so 5.5 days lost in total for 1M microcovids. 

5.5 days = 130 hours

Which implies that ~7600 (1MM / 130) microcovids is ~1 hour of life lost.

Brief thoughts on inefficient writing systems

Sign languages have no written or verbal component. They are much sparser and more contextual than English is. You may be using English concepts but you won't be using them like you would spoken or written English. 

The problem for machines here is easily demonstrated: imagine asking a small child to imitate a cat. They will instantly know what to do, and you'll instantly recognise that they are being a cat. That is an incredibly complex capture problem for machines, then a gestural interpretation one, and then finally a conceptual one. Increase the complexity by several orders of magnitude and you have sign language.

Importance of Ideas and People We Disagree With

A spectrum is a bad way to think about diversity of ideas. To map ideas into a spectrum you have to put them into a box. 

The GMO is example is very fitting of how the box limits thinking well about the issue. It's a tool that allows to change a species in a stronger way then is possible without the tool and with that power comes the ability to screw up. Sensible GMO policy is about risk management and what kind of risk management is reasonable instead of having a "GMO is good" or "GMO is bad" position.

If we all go to the right, and there is no one going to the left, then in very unlikely event that going right way proves to be fatal (black swans are still a thing),

Black Swans per definition are not on the spectrum.

Who decides what should be worked on?

You're precisely right - I've asked it myself in order to see if the company operates in a reasonable manner or not :)

Your second point is spot on as well - it mostly comes from more junior candidates (which, numerically, is the bulk of the people we hire), and from the follow-ups I get, it does often seem to be asked in earnest from that population.

Delta Strain: Fact Dump and Some Policy Takeaways

(I'd personally appreciate you saying how many microcovids you think is equivalent to an hour's time; that's the main number I've been using to figure out whether various costs are worth it.)

Who decides what should be worked on?

However, I always get a bit frustrated when trying to answer it, because I know that it is really not the right question to be asking.


When I ask these questions it is to interview you (or, really, the company). What is the company implicitly and explicitly teaching it's employees about how decisions get made?

In other words, I already know the answer, I'm seeing if you do!

That being said, I would guess that the more inexperienced you are, the more earnest the question asker is about the question.

Another (outer) alignment failure story

I think the AI systems in this story have a clear understanding of the the difference between the measurement and the thing itself.

Are humans similarly like drug addicts, because we'd prefer experience play and love and friendship and so on even though we understand those things are mediocre approximations to "how many descendants we have"?

Another (outer) alignment failure story

realise the difference between measuring something and the thing measured

What does this cash out to, concretely, in terms of a system's behavior? If I were to put a system in front of you that does "realize the difference between measuring something and the thing measured", what would that system's behavior look like? And once you've answered that, can you describe what mechanic in the system's design would lead to that (aspect of its) behavior?

Answer by DagonAug 01, 202110

There will be no clear signs - ambiguity is required to keep humans under control (or rather, so the humans control themselves by incorrect beliefs).

Another (outer) alignment failure story

These stories always assume that an AI would be dumb enough to not realise the difference between measuring something and the thing measured.

Every AGI is a drug addict, unaware that it's high is a false one.

Why? Just for drama?

Cognitive Impacts of Cocaine Use

Are you sure about the imcreased sustained attention among cocaine users? The abstract in your link seems to suggest the opposite.

The World is Continuous, Not Discrete

I fear you're mixing ambiguity with uncertainty with un-quantized states.  A sensible definition of dead would be "never again going to take action".  There is a bright line where this is the case, though there's uncertainty whether it's been crossed.  Hearts can start beating again, though I don't know of any cases of post-cremation resurrection.

Healthy vs sick is similar regarding uncertainty and ambiguity.  Outside of movies, doctors don't pronounce you absolutely healthy.  They can say relatively healthy - better shape or less risk than some comparison group.  

Another (outer) alignment failure story

I like how it never actually attributes consciousness or self-awareness or anything "AGI like" to the automated systems, they could still just be very fast local optimizers, like something Peter Watts or Karl Schroeder would come up with.

Torture vs Specks: Sadist version

You might be correct, but I'm not convinced that all negative utilitarians would agree with you. I think that some formulations (e.g. potentially NHU as described here) would describe the person not getting tortured as producing a reduction in pleasure for the sadists, and thus not ascribe any moral value to the sadists' preferences not getting fulfilled.

I'd be curious to read more about your comment that "Positive and negative utilitarianism are equivalent whenever UFs are bounded and there are no births or deaths as a result of the decision." Do you have some resources you could link for me to read? 

Delta Strain: Fact Dump and Some Policy Takeaways

I did my own back-of-the-envelope calculation and came up with a similar but slightly higher estimated cost of 1.4 to 5.5 quality-adjusted days lost to long-term sequalea conditional on getting symptomatic COVID case. FWIW, I originally thought the OPs numbers seemed way too low, and was going to write a take-down post -- but unfortunately the data did not cooperate with this agenda. I certainly don't fully trust these numbers: it's based on a single study, and there were a bunch of places I didn't keep track of uncertainty, so the true credible interval should definitely be a lot wider. Given that and the right-tailed nature of the distribution, my all-things-considered mean is closer to 30 because of this, but figured I'd share the BOTEC anyway in case it's helpful to anyone.

My model is pretty simple:

  1. What % of symptoms are there at some short-term follow up period (e.g. 4 to 12 weeks)? This we actually have data on.

  2. How bad are these symptoms? This is fairly subjective.

  3. How much do we expect these symptoms to decay long-term? This is going off priors.

For 1. I used Al-Aly et al (2021) as a starting point, which was based on comparing medical records between a COVID-positive and non-COVID demographically matched control group in the US Department of Veterans Affairs database. Anna Ore felt this was one of the more rigorous ones, and I agree. Medical notes seem more reliable than self-report (though far from infallible), they seem to have actually done a Bonferroni correction, and they tested their methodology didn't pick up any false positives via both a negative-outcome and negative-exposure controls. Caveat: many other studies have scarier headline figures, and it's certainly possible relying on medical records skews this low (e.g. doctors might be reluctant to give a diagnosis, many patients won't go to the doctor for mild symptoms, etc).

They report outcomes that occurred between 30 and 180 days after COVID exposure, although infuriatingly don't seem to break it down any further by date. Figure 2 shows all statistically significant symptoms, in terms of the excess burden (i.e. increase above control) of the reported symptom per 1000 patients. There were 38 in total, ranging from 2.8% (respiratory signs and symptoms) to 0.15% (pleurisy). In total the excess burden was 26%.

I went through and rated each symptom with a very rough and subjective high / medium / low severity. 2% excess burden of high severity symptoms, 19% medium severity, 5% low severity. I then ballparked that high severity (e.g. heart disease, diabates, heart failure) wiped out 30% of your QALYs, medium severity (e.g. respiratory signs, anxiety disorders, asthma) as 5% and low (e.g. skin rash) as 1%. Caveat: there's a lot of uncertainty in these numbers. Although I suspect I've gone for higher costs than most people would, since I tend to think health has a pretty big impact on productivity.

Using my weightings, we get a 1.6% reduction in QALYs conditional on symptomatic COVID case. I think this is misleading for three reasons:

  1. Figure 3 shows that excess burden is much higher for people who were hospitalized, and if anything the gap seems bigger for more severe symptoms (e.g. about 10x less heart failure in people positive but not hospitalized, whereas rates of skin rash were only 2x less). This is good news as vaccines seem significantly more effective at preventing hospitalizations, and if you are fortunate enough to be a young healthy person your chance of being hospitalized was pretty low to begin with. I'm applying a 10x reduction for this.

  2. This excess burden is per diagnosis, not per patient. Sick people tend to receive multiple diagnoses. I'm not sure how to handle this. In some cases, badness-of-symptoms does seem roughly additive: if I had a headache, I'd probably pay a similar amount not to also develop a skin rash then if my head didn't hurt. But it seems odd to say that someone who drops dead from cardiac arrest was more fortunate than another patient with the same cause of death, who also had the misfortune of being diagnosed with heart failure a week earlier. So there's definitely some double-counting with the diagnosis, which I think justifies a 2-5x decrease.

  3. This study was presumably predominantly the original COVID strain (based on a cohort between March 2020 and 30 November 2020). Delta seems, per the OP, about 2-3x worse: so let's increase it by that factor.

Overall we decrease 1.6% by a factor of 6.5 (10*2/3) to 25 (10*5/2), to get a short-term QALY reduction of 0.064% to 0.24%.

However, El-Aly et al include any symptom reported between 30 to 180 days. What we really care about is chance of lifelong symptoms if someone is experiencing a symptom after 6 months there seems like a considerable chance it'll be lifelong, but if only 30 days has elapsed the chance of recovery seems much higher. A meta-review by Thompson et al (2021) seems to show a drop of around 2x between symptoms in a 4-12 week period vs 12+ weeks (Table 2), although with some fairly wild variation between studies so I do not trust this that much. In an extremely dubious extrapolation from this, we could say that perhaps symptoms half again from 12 weeks to 6 months, again from 6 months to a year, and after that persist as a permanent injury. In this case, we'd divide the "symptom after 30 days figure" from Al-Aly et al by a factor of 8 to get the permanent injury figure, which seems plausible to me (but again, you could totally argue for a much lower number).

With this final fudge, we get a lifelong QALY reduction of 0.008% to 0.03%. Assuming a 50-year life expectancy, this amounts to 1.4 to 5.5 days of cost from long-term sequelae. Of course, there are also short-term costs (and risk of morbidity!) that is omitted from this analysis, so the total costs will be higher than this.

DataPacRat's Shortform

Just published the first chapter of a month-long novel-writing experiment, which contains enough LW-compatible tropes that it might be of interest: Hustling Through the Dark

Torture vs Specks: Sadist version

Ah you’re right, sorry. Edited.

Torture vs Specks: Sadist version

I'm not sure how negative utilitarianism changes things. Positive and negative utilitarianism are equivalent whenever UFs are bounded and there are no births or deaths as a result of the decision.

Negative utilitarianism interprets this situation as the sadists suffering from boredom which can be slightly alleviated by knowing that the guy they hate is suffering.

Destroying Insecurity and Boosting Confidence Through Your Interests and Values

Thanks for this post. Yes, that is mostly the idea that I am discussing here, which I think can have beyond practical and work related benefits. Mainly, that it makes one more secure and confident in their life as a whole. 

Another thought related to that is that it probably makes not only for more secure personality and confidence, but also more secure career in case one field becomes more obsolete.

Black ravens and red herrings

I think by default, anything that isn't a non-black raven would carry the same weight of "I made an observation and it wasn't a contradictory example"

I guess unless maybe you had a somewhat contrived prior, saying that there must be a certain number of ravens, and also that there can only be a certain number of black things in total. Then seeing a black non-raven would deplete the number of remaining black things and raise the odds that some of the number of ravens would spill outside that category.

Torture vs Specks: Sadist version

Do you mean negative utilitarianism would get them to choose torture, rather than dust specks? I would have considered both to be forms of suffering.

Incorrect hypotheses point to correct observations

No worries. :) Getting it into the curation e-mail was probably good.

Sherrinford's Shortform

Did you consider looking at it rather from "options" than "goals" perspective? Rather than defining goals and looking for the optimal path to get there, you can look at /brainstorm exploitable options that you have available and seem to have high returns. And then prioritize them. I recently spent half a day writing down cool ideas for things to do, then collected them in todoist, and since then, whenever I have time I go through them. And add something new.

Covid 7/29: You Play to Win the Game

I am confused by the quick turn-around in the UK, here are 3 scenarios I considered and why I am not convinced by either.

a) Herd immunity reached as the last peak infected (and made immune) the remaining people. Problems: Intuitively this would have to have been very targeted infection to cause such a quick turn / high change in immunity, as we have millions, 30% population [1] (10% adults [2]), without antibodies. We had a few hundred thousand infections maybe, so I would not expect stark changes in effective R from this.

b) Last peak & immunity in clusters of unvaccinated, or otherwise more-spreading ("party"), people? So similar to the above scenario but much more inhomogeneous, and if we just make the part of the population that is "driving" the pandemic (assuming such a part ot the population exists) immune that might lead to the decline. E.g. (i) in age-groups (young), or (ii) social circles of unvaccinated people (surely there is a correlation between your friends being vaccinated and you being vaccinated), which could be small enough so that the peak indeed corresponds to running out of people to infect? This should be possible to check, by (i) age-distribution or (ii) vaccination status of infected although I have not found that data

c) Could a significant decrease in testing make the cases appear to be lower? The positivity rate [3] dropped from ~ 11.8% (19th) to 9.6% (24th, most recent) while daily cases dropped 41.8k (19th) to 37.1k (24th) [4] --- the positivity rate dropped actually more than cases but the peak of testing (19th) was after the cases peak (43.5k on 21st), which confuses me, though the ratios between the respective peaks and 24th match. Testing rates went down but the data is rather noisy [7].

PS: Metaculus [6] prediction for large 3rd wave this year (>250 deaths/day) went down to 22% from ~ 40% 10 days ago.

Note: I checked the data & wrote most of this 3 days ago, might be slightly out of date now. Decline in UK seems to flatten now.


Brief thoughts on inefficient writing systems
  1. The same way we recognize those words by ear: through context.
  2. Comparing English and Chinese is difficult because of the different ways the two languages use the melody of speech. I note that you have included the tonal markers on all your "shi"s, which make those words distinct (and, to somebody who has learned to read the language that way, easy to read). Moreover, that representation makes the playfulness of the title obvious, which adds value.
Brief thoughts on inefficient writing systems

To do an English spelling reform you would need to decide which dialect of English is the correct one to map and which dialects are wrong.

Disagree: a new writing system can be chosen to accommodate certain dialectical variations (the pen/pin and father/bother issues, for example) and simply represent others. (The name John is pronounced Jawn in some regions of the USA. It would be very easy to spell it with that vowel if we expected spelling to match the sounds.) And it can all be done without applying right/wrong labels to anybody's dialect. It's just a matter of having enough letters (or variations of letters) to accurately represent the differences where they are important. Since we (broadly speaking) currently teach our children to read and write with 26 letters times 4 variations of each (upper- and lower-case in manuscript and cursive) for a total of 104 characters, I see no reason to expect they can't learn letters that map to the 40-ish sounds that make up the English language (plus a few marks to specify which version of the vowel is being indicated.)

If you're concerned about one group "getting" the base vowels (with no diacritics), we could teach that a vowel with no markings represents a family of similar sounds and that adding a marking simply selects for a particular sound. (It's not really all that different than trying to cram 8-12 vowel sounds into 5 letters like we're doing now, just more precise.) You could spell any word with all unmarked vowels with no more ambiguity than that provided by homographs under the current system.

And we could finally represent puns in text! Maybe some of our jokes would actually make sense to archeologists in a few thousand years.

the Balkan is full of wars between people who believe they have a different national identity because they consider their dialect to be a full language and the dialect that other people speak to be another language.

That's region-wide arguing by definition combined with a strong cultural emphasis on national identity. Those problems don't mean that the local spelling system is problematic in any way. Language is a technology and spelling is a tool. That folks are using it to hurt each other doesn't make a tool "bad" somehow, or we'd ban hammers and axes for being involved in murders. Granted, some governments do ban some objects for just that reason. But the practice is extremely uneven, and usually limited to objects that appear to have been designed to hurt people in the first place.

Covid 7/29: You Play to Win the Game

The big mystery remains why Delta suddenly peaked and turned around, first in India, and now in the UK and the Netherlands. These turnarounds are excellent news, and I presume we will see a similar turnaround at a similar point, but what’s causing them to happen so quickly? I don’t know.


Regarding the covid situation in the Netherlands, it was a bit of a perfect storm of many things coming together.
- Dance clubs, events and such opened up with few restrictions - primarily entry proof for vaccination/false-tests
- Getting a single jab of J&J would instantly count as entry proof, and was heavily promoted for youths - it was also promoted to take it the same day / day before going clubbing as an alternative to testing.
- Delta variant appeared
- There was some fraud with the QR-code app for entry-proofs, unsure about how prevalent this was.
- Opening up of clubbing and such happened *before* it was even possibly for any substantial part of the 15-30 year olds to have gotten the vaccine the "regular way" at time.

Closing clubs has stopped the wave, and now a month later the majority of youths are vaccinated. And clubs are still closed. Hospital/IC admissions are now starting to go up as the initial wave has spread to mostly leftover unvaccinated 50+ year olds (I'd imagine (grand)parents of youth that went clubbing).

What follows is an outline of what happened time by time and some sources.

General positive test timing data: [1]
- Lowest point around 27-29th June
- Big inflection point up around 5th of July
- Peak already at 10th of July
A strong majority of positive tests was in ages 10-29, of which most again were in range 20-29.

Timeline of events leading up to the peak
Late June in general - Ages 20-29 that plan for the "regular" vaccine (both the RNA ones; pfizer/moderna) can start getting their first jab. This does not count as being vaccinated for things where you need vaccination-proof / false-test-proof. The second jab is late July to early August.
18th June - [2] Announcement that by the 26th of June nearly everything is open; events and such that can't be done at 1.5 meters will require vaccination/false-test proofs.
23th June - [3] 200K J&J vaccine jabs are available for taking; Youth is encouraged to get it with the slogan "Dansen met Janssen" - "Dancing with J&J". This vaccine only requires one jab so the idea was that it would be easier to get a high vaccination rate under youths using it; you only have to go out of your way once and you're done. A quick easy way to be "covid proof" for when dance clubs, festivals and holidays would open up in the weeks after it, without having to wait 6 weeks or such for a 2nd jab before it officially counts. Phone lines were absolutely swamped by demand. Would save a lot of testing for people who want to go out clubbing every weekend.
24th June - [4] CoronaCheck app updated; being vaccinated counts via this app as entry proof for events. It counts immediately after getting the vaccine, no delay.
(weekend 1; 26th, 27th of June)
26th June - [5] 175K extra J&J vaccines. Everything opens up. About the lowest point in positive test cases.
26th June - [8] At one club that was open on this very first "freedom night", 195/600 youths have tested positive in the following days.
28th June - [9] Reports on "CoronaCheck fraud"; there's a 'print' functionality for the QR-code, which makes one that doesn't time out after a few minutes. Send it to all your mates, they show it at the door. Many clubs aren't going to do the manual secondary ID card check to see if it matches, as that's frustrating customers and it already shows as "safe".
(weekend 2; 3rd, 4th of July)
6th July - [6] A mistake was corrected - Being vaccinated doesn't count as entry proof for the first 2 weeks after vaccination.
10th July - Peak day - [7] New covid rules; Dance/night clubs closed, live entertainment in catering banned, large multi-day festivals get cancelled; work at home if possible.

Vaccination related data;
[10] Official sources say that for 81% of positive tests between the 1st & 27th of July the vaccination status was known. Of those with known status, 11% was fully vaccinated, 16% partially, 73% not.
[11] Questioning of the majority of hospitals by a group of doctors got some results on vaccination status for covid hospital patients in the week of 12-18th July. About 75% was not vaccinated. About another 5% had a known problem that would reduce vaccination efficiency.
[12] People with "at least one jab", per age group: 20-35 is about 60%, 35-45 about 75%, 45-55 about 80%, 55+ is about 90%

[1] Official dutch covid dashboard;
[2] Official gov site
[3] Dutch public broadcasting news
[4] Official gov site
[8] random news site;
[9] random news site;
[10] Dutch CDC equivalent, official
[11] Public broadcasting news
[12] you can get a table from the graph at the bottom

Delta Strain: Fact Dump and Some Policy Takeaways

In another comment, I discuss what seems like a big limitation of the paper. 

Incorrect hypotheses point to correct observations

You're doing good work with the curation and it's very effective at bringing important posts back into the reader's eye so thanks for that! I would probably have never seen this post otherwise. I'm glad you're working on the system to iron out the kinks.

Josh Jacobson's Shortform

~2 weeks ago, the FDA added a warning to the J&J Covid shot regarding increased risk of developing Guillain-Barré Syndrome.

Perhaps unsurprisingly, given the history with blood clots, my quick check of prevalence finds that reports of developing GBS following J&J vaccination are actually less than would be expected otherwise.

My very basic analysis:

Numbers from:

Torture vs Specks: Sadist version

I think negative utilitarianism is the most common ethical framework that would cause someone to choose the torture in the specks vs. torture case and no torture in this case. That's because in the specks vs. torture case involves people being harmed in both cases, whereas this case involves people gaining positive utility vs. someone being harmed. Some formulations of negative utilitarianism, like that advocated for by Brian Tomasik, would say that avoiding extreme suffering is the most important moral principle and would therefore argue in favor of avoiding torture in both cases. But a very simple negative utilitarian calculus might favor torture in the first case but not in the second. 

I would guess that few people in the rationalist/EA community (and perhaps in the broader world as well) are likely to think that kind of simplistic negative utilitarian calculation is the morally correct one. My guess is that most people would either think that preventing extreme suffering is the most important or that a more standard utilitarian calculus is correct. For a well-reasoned argument against the negative utilitarian formulation, Toby Ord has a discussion of his point of view that's worth checking out. 

Torture vs Specks: Sadist version

Although under strict preference utilitarianism, wouldn't change in values/moral progress be considered bad, for the same reason a paperclip maximizer would consider it bad?

Torture vs Specks: Sadist version

I should say we assume that we're deciding which one a stable, incorruptible AI should choose. I'm pretty sure any moral system which chose torture in situations like this would not lead to good outcomes if applied in a practical circumstance, but that's not what I'm wondering about, I'm just trying to figure out which outcome is better. In short, I'm asking an axiological question, not a moral one.

My intuition strongly says that the torture is worse here even though I choose torture in the original, but I don't have an argument for this because my normal axiological system, preference utilitarianism, seems to unavoidably say torture is better.

Torture vs Specks: Sadist version

If consequences are completely ignored, I lean towards the torture, but if consequences are considered I would choose no torture out of hope it accelerates moral progress (at least if they had never seen someone who "aught to be tortured" get away, the first one might spark change. which might be good?). In the speck case, I choose torture.

Brief thoughts on inefficient writing systems

Language isn’t just about efficiency; cultural aesthetic is a terminal value.


Comment on decision theory

There have been various academics who have discussed decision theory before MIRI even came into existence. I don't know if they're actually working on improving things, though. My sense is that they've mostly been just been sticking with causal decision theory and evidential decision theory. But there's probably at least some work done improving.

Uncertainty can Defuse Logical Explosions

Can you give the probabilities that the agent assigns to B1 through D4 in the "sandboxed" counterfactual?

Delta Strain: Fact Dump and Some Policy Takeaways

It's not a great study.  It's observational, and the best it can show is correlation.  It's just as likely that people with that score lower on cognitive tests are more likely to get covid, which doesn't seem unrealistic.  A bit of armchair analysis (with only a little bit of ax-grinding):

Josephine's Shortform

Sounds correct. I was thinking how this applies to computer games:

Several subskills - technical perfection, new idea, interesting story, graphics, music... Different games become popular for different aspects (Tetris vs Mass Effect vs Cookie Clicker).

A frequent beginner mistake is making a game with multiple levels which feel like copies of each other. That's because you code e.g. five or ten different interactive elements, and then you use all of them in every level. It makes the first level needlessly difficult, and every following level boring. Instead, you should introduce them gradually, so each level contains a little surprise, and perhaps you should never use all of them in the same level, but always use different subsets, so each level has a different flavor instead of merely being more difficult.

Another beginner mistake is to focus on the algorithm and ignore the non-functional aspects. If one level has a sunset in background, and another level uses a night sky with moon, it makes the game nicer, even if the background does not change anything about functionality.

Yet another mistake is to make the game insanely difficult, because as a developer you know everything about it and you played the first level for hundred times, so even the insanely difficult feels easy to you. If most new players cannot complete the tutorial, your audience is effectively just you alone.

Some people may be successful and yet you don't want to be like them, e.g. because they optimize the product to be addictive, while you aim for a different type of experience; or their approach is "explore the market, and make a clone of whatever sells best", while you have a specific vision.

You should do a very simple game first, because you are probably never going to finish a complicated one if it's your first attempt. I know a few people who ignored this advice, spent a lot of time designing something complex, in one case even rented a studio... but never finished anything. (Epistemic check: possible base-rate fallacy; most people never write a complete computer game, this might include even most of those who started small.) And the more time you wasted trying to make a complicated game, the less likely you are to give up and start anew.

Successful game authors often recycle good ideas from their previous, less successful games.

The audience is famously toxic. Whatever game you make, some people will say horrible things about the game and about you in general. It is probably wise to ignore them. (Epistemic check: so you're saying that you should only listen to those who liked your game? Yeah... from the profit perspective, the important thing is how many fans you have, not what is their ratio to haters. A game with 1000 fans and 10000 haters is more successful than a game with 10 fans and 1 hater.)

Being good at designing logical puzzles does not translate into being good at designing 3D shooters, and vice versa.

Open and Welcome Thread – July 2021

This is known as "agent simulates predictor". There has been plenty of discussion of this problem. I'm currently feeling too lazy to try to summarize or link all the approaches, but here are some thoughts I had about it via my infra-Bayesian theory.

How to teach things well

In my experience teachers tend to only give examples of typical members of a category. I wish they'd also give examples along the category border, both positive and negative. Something like: "this seems to have nothing to do with quadratic equations, but it actually does, this is why" and "this problem looks like it can be solved using quadratic equations but this is misleading because XYZ". This is obvious in subjects like geography, (when you want to describe where China is, don't give a bunch of points around Beijing as examples, but instead draw the border and maybe tell about ongoing territorial conflicts) but for some reason less obvious in concept-heavy subjects like mathematics.

Another point on my wishlist: create sufficient room for ambition. Give bonus points for optional but hard exercises. Tell about some problems that even world's top experts don't know how to solve. 

Delta Strain: Fact Dump and Some Policy Takeaways

Curated. It seems really important to figure out how to deal with the Delta variant, and orienting around how to interface with covid variants longterm. I appreciated this post for:

  1. Striking a good balance of presenting current facts on the ground
  2. Noting where assumptions lie
  3. Being clear about it's epistemic status, while
  4. Aiming to be useful in the immediate term.
Working in Virtual Reality: A Review

Thanks for sharing. I think that the Windows options seem pretty strictly superior to the Apple options at this point. 

I've used Virtual Desktop with the Quest, but it doesn't allow you to use a wired connection (using a mac). With my wifi setup, it was pretty slow.

Sorry to hear about the myopia! I didn't really notice that. I hope future units improve here.

We have some evidence that masks work


My current belief state is that cloth masks will reduce case load by ~15% and surgical masks by ~20%.

Without altering the bet I'm curious as to what your belief state is.

Brief thoughts on inefficient writing systems

I think these are all good examples of language reforms. I guess my issue is that I was over-fixating on english.

Covid 7/29: You Play to Win the Game

The Hill has published some more information:

The state health department identified 469 COVID-19 cases among Massachusetts residents who went to Provincetown, a popular vacation destination in Barnstable County, in the month of July, including 346 fully vaccinated people.

Some 127 COVID-19 samples from the fully vaccinated, including recipients of all three U.S.-authorized vaccines, showed a similar viral load to the samples from the 84 unvaccinated people.

The report noted that microbiological studies are needed to confirm that similarity in the viral load to determine whether fully vaccinated people can transmit the virus.

I still have the impression that this data could be systematically biased: it makes sense that the viral load would be high among identified cases, but randomized testing of the broader population is needed to understand the base rates.

Incorrect hypotheses point to correct observations

As mentioned elsethread, I checked with authors the first couple of times. Having met no objections, I applied induction and tested whether maybe the overhead wasn't necessary.

Incorrect hypotheses point to correct observations


I have been unsure how long it could persist for.  There's an argument for it being present all the time it's at the top of the curated list, and if you imagine author's being proud of it, for even longer than that. But perhaps (while we're doing the low-tech thing), it should only be for sending out the email.

Incorrect hypotheses point to correct observations

Could definitely do that. I felt more weird about putting my own name right at the top.

Incorrect hypotheses point to correct observations

I contacted the authors the first couple of times while trying this new thing and neither objected, nor had anyone else yet, so I didn't check with Kaj (sorry Kaj).

It's good now that people have spoken about and we'll think about how to get the good without the bad. As Habryka mentioned, the thing we're trying to achieve is giving people context on the curation. I've wanted to avoid putting something prominently in front of people's faces without them knowing why. Not all curated posts require this, but sometimes we curated weird ones, or ones where it's being curated for the comments rather than the post itself, or ones that are irregular (e.g. important Covid updates). In all those cases, it feels important to provide an explanation that gets seen.

As Habryka mentioned, if we keep doing this, we'll build a way to do that doesn't involve editing. This was just the quick test version.

Thanks again though for the feedback!

Actually possible: thoughts on Utopia

By the way - your Omelas link is broken.

Tao Lin's Shortform

More likely that this just doesn’t occur to people and/or they have no idea how to make and install favicons, and possibly don’t even have any concept of how favicons work and where they come from.

Don’t underestimate the technical cluelessness of people on the internet, even ‘rationalists’.

Covid 7/29: You Play to Win the Game

I also am skeptical that this effect could fail to partly fade with time or as symptoms fully go away, whereas they are claiming to not see such effects. 

I'm also skeptical because effects from time in ICU for other respiratory diseases and other conditions do partly fade if you wait long enough (e.g. 6-12 months). Trying to make sense of the supplementary figures, it seems to me that nearly all subjects did the cognitive test less than 3 months after the onset of Covid (despite what the figure actually shows). Here's the figure (downloaded from this page):


The top graph suggests a non-trivial proportion completing the assessment 3 months after onset. However, this is self-report and lots of people erroneously believed they had Covid in the early days of the epidemic (when there was almost zero testing in the UK for mild cases). The bottom graph suggests that the cognitive assessment is mostly over by the end of May. So people with onset >3 months earlier had Covid before the start of March. Yet the UK had very few cases before March: the first wave peak was after April 15. On March 13, there had been a total 10 deaths (corresponding to 1000 cases on a 1% IFR). So I think their inferred "illness onset" plot on the bottom graph is seriously flawed. I haven't run the numbers, but I'm guessing that the time from onset of Covid to assessment is (i) a narrower distribution than the top figure (due to truncating at 3 months), and (ii) has a mode shifted left of 2 months. 

If I'm right in my analysis, this suggests the following:
1. The researchers were sloppy.
2. The study cannot tell us that much about Long Covid because the time since onset is too short. 

Incorrect hypotheses point to correct observations

(I now edited out the curation notice, since the e-mail was already sent a while back.)

What weird treatments should people who lost their taste from COVID try?

n=1 isn't nearly enough data to answer that question unfortunately.

Covid 7/29: You Play to Win the Game

Or they may be so fatigued and therefore behind on lots of life-admin tasks that signing up for a study is the last thing they'd consider.

Also, if there is a 0.5 IQ points loss in expectation, it seems to matter how it's distributed. Would everyone get the same effect or is it mostly no effect + some people lose 5 IQ points? The latter is arguably a lot worse because you can easily compensate for a small loss (drink extra caffeine when you need to be extra sharp) but not for a big one. 


Something that distinguishes child-rearing and immersive language learning from children in school is the seriousness that both tasks force. The parents want to create a new, well-functioning human being (when they don’t want this, it doesn’t go well), and you need the language in order to have any interaction with the people around you. At least, you do if you’re a newborn, and it still takes a good many years. (Is immersion actually the best way for adults? It’s the only way for children, but because of that there’s no other way to compare it with.)

That quality of seriousness, it seems to me, is what produces progress. The school child is rarely serious. Someone attending a course just for the socialising is not serious. There are teenagers who have won gold medals at the Olympics. Those ones are serious.


1. You can think of a learning mind as a tool for mapping cause and effect relationships by being affected by them.
If you live among speakers of a language you experience a lot of interactions (e.g. people refer to an object with the same sound) ---> these change your mind (associations created via neurons) ---> better knowledge of language (better mapping between variables in this domain)

2. However, when you move a mind to a space with more data of a subject, other variables can also change, which might have a negative impact and can disable the mind.
If you want to find out how to float on water, jumping into a deep pool and experimenting with motions will help in finding the ones that achieve this (again, the mind is creating connection between movements done with the limbs and how they affect movement in the water). However, if you can't find the correct motions, you will sink to the bottom. ---> your brain will not receive oxygen ---> your mind is disabled (death)

3. So you want to be careful about which space you move a mind to. It is not enough to have more data, you should control other variables to make sure the mind is safe.
Learn the basics of swimming in shallow water.

4. As minds change through learning, variables may have different effect on them than before.
If you know how to swim, jumping into a deep pool won't kill you anymore in most circumstances.

We have some evidence that masks work

Sounds fine. Just noticed they have a cloth and a surgical treatment. Take the mean?

Brief thoughts on inefficient writing systems

There are two problems with phonetic writing. 

  1. Omonims. How to recognise, what is "rouz" (it may be rows, rose, rhos, ...).
  2. Compactness. 施氏食狮史 is much shorter then "The Story of Mr. Shi Eating Lions" and much easier to understand then Shī-shì shí shī shǐ (again omonims).

Try to think, why we write 1, 2, 3, ... but (usually) not one, two, three. We learn a lot of characters, such as #$%@<>..., road signs, product signs, math signs, emoji, ...

Chinese is hard to learn, but easier to read.

Covid 7/29: You Play to Win the Game

Very much agree with this - we have adverse event data from tens of millions of vaccinations, so it seems odd to suddenly ignore that on the basis of this study (doesn't look at deaths as far as I can see, and only looks at ~400,000 AZ vaccinations, mostly in people aged over 60).

Answering questions honestly instead of predicting human answers: lots of problems and some solutions

Yep, that's what I mean.

Then I'm confused what you meant by

I'm not sure what you mean by this part— and  are just different heads, not entirely different models, so I'm not sure what you mean by “the parameters in .”

Seems like if the different heads do not share weights then "the parameters in " is perfectly well-defined?

Note that conditioning on the part-which-shares-weights is definitely not what the prior is doing

Yeah, sorry, by "conditioning" there I meant "assuming that the algorithm correctly chose the right world model in the end", I wasn't trying to describe a particular step in the algorithm. But in any case I don't think we need to talk about that 

They are certainly different functions over the space of all possible deduced statements—but once you put a correct world model in them, they should produce equivalent  maps.

Okay, so iiuc you're relying on an assumption (fact? desire?) that the world model will never produce deduced statements that distinguish between  and  ? My understanding of  and  comes from here:

Specifically,  is the “honest embedding” which directly converts between logical statements and their equivalent natural language, thus answering questions by embedding  as a logical statement and unembedding its answer in . Conversely,  is the “mimicry embedding” which just searches for deductions about what a human would say in response to  and outputs that—thus,  just quotes , embedding it as just a string of characters for a human to respond to, rather than actually having to understand it in any meaningful way.

If  and  produce equivalent  maps, doesn't that mean that we've just gotten something that can only respond as well as a human? Wouldn't that be a significant limitation? (E.g. given that I don't know German, if my question to the model is "what does <german phrase> mean", does the model have to respond "I don't know"?)

In addition, since the world model will never produce deduced statements that distinguish between  and , it seems like the world model could never produce decision-relevant deduced statements that the human wouldn't have realized. This seems both (a) hard to enforce and (b) a huge capability hit.

Incorrect hypotheses point to correct observations

A minor change might make this more perspicuous (although this suggestion should be viewed in the light of the OP and the thread from Said Achmiz's comment on UX design): Begin the curation notice at the top of the OP with "Curated by (username)".

How much compute was used to train DeepMind's generally capable agents?

Huh, thanks! I guess my guesstimate is wrong then. So should I multiply everything by 8?

How much compute was used to train DeepMind's generally capable agents?

I did, sorry -- I guesstimated FLOP/step and then figured parameters is probably a bit less than 1 OOM less than that. But since this is recurrent maybe it's even less? IDK. My guesstimate is shitty and I'd love to see someone do a better one!

Brief thoughts on inefficient writing systems

Personally, I want a written language that not only is logographic, but breaks concepts down into their component parts, the way an alphabet breaks syllables into their component sounds. Think:

Latin Alphabet : Japanese syllabary :: Logobet : Chinese Logograms

The problem with phonemic orthographies is that they correspond to only one language, while logographies can correspond to any language; this provides a benefit in China, where the population speaks many mutually unintelligible languages, but everybody can read & understand the same written language. This is why logographies are superior to phonetic systems.

Prediction-based-Medicine instead of E̵v̵i̵d̵e̵n̵c̵e̵ ̵b̵a̵s̵e̵d̵ ̵M̵e̵d̵i̵c̵i̵n̵e̵ Authority-based-Medicine

My main point is that the ability exists in both the existing system and also a prediction-based one.

But when it comes to justifying poison, imagine you have a drug that gives everybody nausia after they take it. You have to tell your trial participants because of informed consent that everybody who gets verum will very likely get nausia soon after taking it. 

You can justify giving the patients in the placebo group a poison that causes nausia because otherwise all the patients know whether or not they got the placebo. 

Many studies don't specify what they are using as placebo. 

I do remember cases where someone argued that the dose of the control group was likely net negative but don't have specific links right now. 

Incorrect hypotheses point to correct observations

Shouldn't be too hard, but seemed worth testing whether it was working in a low-tech solution before we build a thing. My guess is we will add this more properly relatively soon, if we keep doing it.

Brief thoughts on inefficient writing systems

I think we should also take into account the value of English spellings that maintain common forms with other languages, even at the expense of being phonetic.

For instance, to a speaker of French or Spanish, the English word "diversification" would certainly seem less alien than a hypothetical respelling as "daiversifikeishun". Does having a common (or very similar) cross-language orthography for latinate words offer more advantages than the benefits of phonetic spellings? I'm not sure, but it should certainly be part of the discussion.

I suspect the benefit is greatest in technical fields with lots of latin-derived vocabulary (i.e. health & biology). Would international scientific cooperation become more difficult if French and Spanish speakers had to relearn spellings for words like "capillaries" [kapileirīs] or "canine" [keinain] whereas the original english was almost identical to the spelling in their native language.

Incorrect hypotheses point to correct observations

Yes I also find the curation tagline to be awkward. The first few times I saw it I assumed that the original author had edited their post to include the curation comment at the start. I only discovered this was not the case by reading the comments here. Seems that editing another user's post is bad form. Adding your own comment to the start of their post is especially obtrusive. The current curated tag should be enough for the website itself. Maybe the subscription e-mail can retain the curation notice while the original post remains unedited.

Unless the original authors are contacted and agreed to the curation notice being included at the top of their posts. Then everything is fine as it is.

Prediction-based-Medicine instead of E̵v̵i̵d̵e̵n̵c̵e̵ ̵b̵a̵s̵e̵d̵ ̵M̵e̵d̵i̵c̵i̵n̵e̵ Authority-based-Medicine

I think his point is that the same failure state Measure mentioned, doctors giving patients poison and correctly predicting outcomes, is just as likely as for the current clinical trial scheme.

How much compute was used to train DeepMind's generally capable agents?

Your link says rats have ~200 million neurons, but I think synapses are a better comparison for NN parameters. After all, both synapses and parameters roughly store how strong the connections between different neurons are.

Using synapse count, these agents are closer to guppies than to rats.

Incorrect hypotheses point to correct observations

(Separately from the violence issue, it looks like "the community" has an initials collision: I initially read "RB" as "Rob Bensinger" rather than "Ruby Bloom.")

Incorrect hypotheses point to correct observations

How hard is it to make the Curation admin tooling include a text field for curation reason, that both gets posted as a comment, and included in the email? This really seems like a nice-to-have software feature request, rather than something that merits the grotesque violence of editing the OP!

Prediction-based-Medicine instead of E̵v̵i̵d̵e̵n̵c̵e̵ ̵b̵a̵s̵e̵d̵ ̵M̵e̵d̵i̵c̵i̵n̵e̵ Authority-based-Medicine

Why would poison (or anything with a known negative effect) be used as a placebo?

Of course I understand the drug companies' incentives, but I don't get how that could be justified or look reasonably scientific. Do you have a specific example?

Load More