Counter-theses on Sleep

Alexey Guzey’s Theses on Sleep gained a lot of popularity and acclaim on LessWrong and among people I follow on social media, despite largely consisting of what I think were weak arguments and misleading claims. I found that a bit surprising, so I decided to write a post pointing out several of the mistakes I think he’s made, and reporting some of what the academic literature on sleep seems to show. 

Sleep deprivation is associated with *both* depression *and* mania

One of Guzey’s theses is that “depression triggers/amplifies oversleeping while oversleeping triggers/amplifies depression.” The first piece of evidence he uses to support that is people on /r/BipolarReddit saying that they sleep a lot when depressed, and sleep very little when manic. However, there’s a big problem with using that as evidence. 

Guzey’s /r/BipolarReddit evidence is misleading

The DSM-5 specifies subtypes of depression that have opposing relationships with sleep. Depression with melancholic features is associated with early morning awakening, whereas depression with atypical features is associated with hypersomnia.

Guzey’s evidence is misleading because people with bipolar disorder are disproportionally prone to having atypical features during their depressive episodes. So, unsurprisingly, his evidence from bipolar disorder patients is not representative of what you see in the general population: both long and short sleep duration are associated with depression (and, relevantly, suicide as well, in adults as well as adolescents). 

I’m surprised no one in the comments of this post brought up this objection – it only takes a very quick Google Scholar search to see that the relationship between sleep duration and depression is not linear. 

Chronic sleep deprivation might be associated with *decreased* BDNF expression, as shown by Guzey’s own sources

Guzey hypothesizes that 

Sleep deprivation appears to increase BDNF [and therefore neurogenesis?]

He then proceeds to link to a few papers that showed up when he Googled “sleep deprivation bdnf”. These sources agree that acute sleep deprivation increases BDNF expression, but some also say that the opposite may happen when sleep deprivation is chronic, which Guzey fails to mention in his post.

From The Brain-Derived Neurotrophic Factor: Missing Link Between Sleep Deprivation, Insomnia, and Depression:

Chronic sleep deprivation and insomnia can act as an external stressors and result in depression, characterized by hippocampal BDNF downregulation along with disrupted frontal cortical BDNF expression, as well as reduced levels and impaired diurnal alterations in serum BDNF expression.

Guzey also links to The link between sleep, stress and BDNF, which seems to be only an abstract. After a bit of searching, I found this full-text paper with some of the same authors, which appears to have an almost identical abstract (?). It concludes with the following:

[O]ur findings are in line with the hypothesis of an increased stress vulnerability due to sleep loss which may lead to a decrease in BDNF. [...] While we report a reduction of BDNF levels linked to sleep disturbance reflecting chronic stress on the one side, we and others consistently showed that prolonged wakefulness caused by SD (partial or total), which can be considered as an acute stressor for the brain, leads to a rapid increase of BDNF (1,36).

Our priors about sleep research should be high

I feel grumpy, dumb and distractable every time I sleep less than 7 or so hours. I can’t do things that require focused attention like solving physics homework problems very effectively, and I don’t get nearly as much pleasure when I attempt doing so. My memory becomes very poor: after a recent night of <6 hours of sleep, I somehow forgot the reasoning behind several Manifold Markets trades I had made the prior evening and just stared at them in utter confusion for several minutes before remembering. 

I have the impression that most people have a similar experience. Guzey has a cute explanation of how this is consistent with his thesis that sleep deprivation doesn’t make you grumpy or dumb, but the fact that I and others I know have this experience with sleep deprivation obviously makes it so that I have a high prior that it’s harmful to cognition and mood. It sounds a lot more plausible that sleep deprivation being harmful in those ways is causing both my personal experience with it to be terrible and academic research to find that it’s harmful, instead of a different factor explaining each thing. Guzey has to add quite a few epicycles to his theory to explain the evidence. 

To the extent that you and the people you know feel the same way I do after a night of sleep deprivation, your prior should be high as well. 

Most of Guzey’s arguments against trusting sleep research are bad

Guzey claims that “most sleep research is extremely unreliable and we shouldn’t conclude much on the basis of it,” but there are problems with that. Firstly because he doesn’t seem to believe that about sleep research that favors his hypotheses. Guzey, after all, uses sleep research to show that Matthew Walker’s book is terrible and fraudulent. So it seems that he wants to trust sleep research when it says that sleep deprivation is not as bad as Walker shows, but doesn’t want to trust it when it says that sleep deprivation is not harmless.  

Secondly, he bases that assertion on a claim that sleep science is “mostly just rebranded cognitive psychology” (and that it is only not facing a severe replication crisis because sleep experiments are expensive), which is very misleading. Unlike the famously unreliable cognitive science results, the finding that sleep deprivation is harmful for cognition (1) in fact gets replicated a lot (see the section below on meta-analyses of sleep restriction studies, as well as this meta-analysis of total sleep deprivation studies) and (2) is consistent with a lot of people’s experiences (for examples, see this addendum.) So it’s hard to see how Guzey’s criticisms apply. 

Moderate sleep restriction impairs cognition

Guzey says, in his post: 

I wrote large chunks of this essay having slept less than 1.5 hours over a period of 38 hours. I came up with and developed the biggest arguments of it when I slept an average of 5 hours 39 minutes per day over the preceding 14 days. At this point, I’m pretty sure that the entire “not sleeping ‘enough’ makes you stupid” is a 100% psyop. It makes you somewhat more sleepy, yes. More stupid, no. I literally did an experiment in which I tried to find changes in my cognitive ability after sleeping 4 hours a day for 12-14 days, I couldn’t find any. My friends who I was talking to a lot during the experiment simply didn’t notice anything.

I don’t think that “feeling smart after sleep deprivation” (or any of those other things) is nearly enough evidence to make you conclude that “ ‘not sleeping ‘enough’ makes you stupid’ is a 100% psyop” if you start out with even a halfway reasonable prior, and especially if you appropriately update on what the sleep literature says. 

I looked for meta-analyses that investigated the effect of experimental or quasi-experimental nighttime sleep restriction on cognition. I found three, and am quoting the relevant conclusions from them:

The neurocognitive consequences of sleep restriction: A meta-analytic review:[1] 

The current meta-analytic review revealed that restricted sleep results in significant neurocognitive deficits (g = −0.383) in a sample of 1688 participants derived from 71 different study populations. This effect was apparent across multiple cognitive domains, with the largest effects being observed on measures of sustained attention (g = −0.409) and EF (g = −0.324), and within these domains, attentional lapses and (g = −0.516) and behavioural inhibition (g = −0.464) specifically.

[...]

Meta-regression analyses indicated that age-adjusted sleep deficit (β=-.206, p=.033), cumulative days of restricted sleep (days; β=-.015, p=.019),[2] subjective sleepiness (β=-.040, p=.016), biological sex (β=.0318, p=.009), and sleep latency (β=.012, p=.013) accounted for a significant proportion of the variance in the observed effect of sleep restriction on overall cognitive abilities.

Sleep Loss and Performance in Residents and Nonphysicians: A Meta-Analytic Examination:

Chronic partial sleep loss also resulted in a significant reduction in cognitive performance, with a correct d value of -.886. 

A Meta-Analysis of the Effect of Experimental Sleep Restriction on Youth’s Attention and Hyperactivity (on people under 18 years old only):

A total of 13 samples (N = 268) examined the difference in youth’s attention performance between baseline (control) sleep and restricted sleep. The overall effect size of −0.19 was significant (95% CI: −0.34−0.03; p = .02); Q(13) = 21.98, p = .04, I 2 = 45.40% (see Figure 2). The I^2 index of 45.40% indicates a small to moderate amount of heterogeneity in effect sizes across studies (Card, 2012). However, the trim and fill method for addressing publication bias (Duval & Tweedie, 2000a; Duval & Tweedie, 2000b) revealed asymmetry in the funnel plots for the difference in attention outcomes between sleep restricted and baseline sleep. One study to the left of the mean was unmatched. The counterpart of this study was imputed to the right of the mean, resulting in a non-significant and small adjusted effect size of −0.14 (95% CI: −0.32–.04).

Six samples (N = 279) examined the difference in attention between extended sleep (sleep extension) and sleep restriction. The overall effect size of −0.37 was significant (95% CI: −0.55 to −0.19, p < .0001) and represents a small to medium effect size (Lipsey & Wilson, 2001), though this effect was heterogeneous between studies Q(5) = 15.29, p = .009, I 2 = 67.30% (see Figure 2). The I 2 index of 67.30% indicates a moderate to large amount of heterogeneity in effect sizes across studies (Card, 2012). Based on trim-and-fill analyses, asymmetry was also present for attention outcomes between restricted and extended sleep, with two studies to the left of the mean unmatched, resulting in a significant adjusted effect size of −0.26 (95% CI: −0.46 to −0.05), which is considered a small to medium effect (Lipsey & Wilson, 2001).

The average age-adjusted sleep deficit in the first meta-analysis was 3.83 hours (SD = 1.25). (This is in comparison with the median recommended amount of sleep for each age group (so 8 hours per night for non-elderly adults), not the average amount of sleep people in each age group actually get.) I couldn’t find information about the other ones, but I have a high credence that they examined studies with a roughly similar protocol; I’ve combed through a lot of individual sleep restriction studies before and they rarely seem to involve making people sleep for < 3.5 hours. 

I also found a meta-analysis investigating the effects of napping, and it finds that it’s beneficial, which is probably relevant:

Effects of a Short Daytime Nap on the Cognitive Performance: A Systematic Review and Meta-Analysis:

Overall cognitive performance did not differ at baseline (t0) between groups (effect size −0.03, 95% CI −0.14 to 0.07), and improved in the nap group following the nap (t1) (0.18, 0.09 to 0.27), especially for alertness (0.29, 0.10 to 0.48). Sensitivity analyses gave similar results comparing only randomized controlled trials, and after exclusion of outliers.

There’s also (weaker) evidence that, contrary to what Guzey hypothesizes, there is no cognitive adaptation to chronic sleep restriction. From this article:

Contrary to a popular belief that healthy adults can acclimate to sleep loss, the effects of chronic partial sleep loss appear to be cumulative.9-11 Specifically, sleepiness has been found to increase9 and performance on tests of vigilance and mathematical calculations to decline across 7 days of 5 and even 7 hours of sleep per night.10,11 Subjects often underestimate their own degree of sleep-related impairments in vigilance after 1 week of partial sleep restriction.9,12 Thus, they may mistakenly believe that they have acclimated to sleep deprivation.

In addition to reduced vigilance, verbal processing and complex problem solving13,15 are impaired with both short-term and chronic partial sleep loss.

See also footnote 2 ([2]) on the matter of cognitive adaptation. 


I do recognize that even meta-analyses of experimental studies can obviously be very problematic, and so this section is not conclusive evidence that moderate sleep restriction impairs cognition (and not only because conclusive evidence is not a thing). But it’s not as if Guzey has any better evidence to argue that “ ‘not sleeping ‘enough’ makes you stupid’ is a 100% psyop.”

Occasionally stubbing your toe is good for health and promotes more efficient toe healing

One of Guzey’s theses is that “[o]ccasional acute sleep deprivation is good for health and promotes more efficient sleep.” His argument supporting that thesis is pretty much that, because some types of acute stress (such as exercising and fasting) are good, and acute sleep deprivation causes acute stress, then acute sleep deprivation is also good. (Yes, that does seem to actually be the entirety of his argument in that section. You can read it yourself.)

The obvious problem with that argument is that the set of things that cause acute bodily stress is much larger than the set of things that cause long-term benefits. Stubbing your toe, for example, causes acute bodily stress. Guzey’s argument works equally well for showing that occasional toe-stubbing is good for health as for showing that occasional acute sleep deprivation is.

Short-term != long-term

In this section, Guzey lists lines of evidence that bring him to the conclusion that “decreasing sleep by 1-2 hours a night in the long-term has no negative health effects.” The lines of evidence are:

  1. A sleep researcher who trains sailors to sleep efficiently in order to maximize their race performance believes that 4.5-5.5 hours of sleep is fine.
  2. 70% of 84 hunter-gatherers studied in 2013 slept less than 7 hours per day, with 46% sleeping less than 6 hours.
  3. A single-point mutation can decrease the amount of required sleep by 2 hours, with no negative side-effects.
  4. A brain surgery can decrease the amount of sleep required by 3 hours, with no negative-side effects.
  5. Sleep is not required for memory consolidation.

Apart from (2), we don’t have any indication of the long-term outcomes of the groups of people he mentioned. So I don’t know how these points could be more than weak and circumstantial evidence of this section’s thesis.

(For that matter, barely any of those even rigorously measure the health effects of short sleep at all. In a sense, they confirm that these people are not immediately dying or getting very acutely sick or something, and I guess I can interpret Guzey as merely wanting to claim that, but it’s not exactly a novel or surprising claim.)

(Also, the 5th line of evidence doesn’t even seem to be related to health.)

Experimental sleep restriction seems to cause short-term detriments to metabolic health

I searched Embase for meta-analyses on the effect of experimental sleep restriction on a host of health-related variables, and this is what I found.

Effects of sleep manipulation on markers of insulin sensitivity: A systematic review and meta-analysis of randomized controlled trials:

Whole-body insulin sensitivity was also reduced after short sleep when measured by the hyperinsulinemic euglycemic clamp, but peripheral insulin sensitivity was not affected. In addition, circadian misalignment and slow wave sleep suppression negatively affected insulin sensitivity, while rapid eye movement sleep disturbance and sleep fragmentation had no effect.

Effects of sleep restriction on metabolism-related parameters in healthy adults: A comprehensive review and meta-analysis of randomized controlled trials:[3]

Participants consumed 252.8 more kcal/d (p = 0.011) under sleep restriction than under normal sleep. Partial sleep restriction resulted in a 0.34 kg weight gain (p = 0.003). Sleep restriction also decreased insulin sensitivity (standardized mean difference = −0.70, p < 0.01). Significant changes in brain activity in response to food stimuli were observed under sleep restriction, particularly regions related to cognitive control and reward.

Sleep Restriction Effects on BP: Systematic Review & Meta-analysis of RCTs:

Overall, sleep restriction did not result in significant changes in systolic blood pressure (SBP) or diastolic blood pressure (DBP) and heart rate (HR). The respective weighted mean difference (MD) was 1.0 mmHg (95%CI, -2.3-4.2; p = 0.57), -0.4 mmHg (95%CI, -3.2-2.4; p = 0.80), and 2.0 bpm (95%CI, -2.2-6.2; p = 0.34).

Sleep Disturbance, Sleep Duration, and Inflammation: A Systematic Review and Meta-Analysis of Cohort Studies and Experimental Sleep Deprivation:

Experimental sleep deprivation, either for partial or total night, was not associated with CRP [...], IL-6 [...], or TNFα [...]. Likewise, sleep restriction over several days was not associated with CRP [...], IL-6 [...], or TNFα [...].

(Eyeballing each of those meta-analyses’ lists of studies, it seems that, in most of them, time in bed was restricted to 4-5 hours, with the range being from ~3.5 to ~5.5.)

So experimental sleep restriction (presumably during short periods of time) seems to impair things like insulin sensitivity and the regulation of satiety, but not blood pressure or inflammatory markers. 

Importantly, the metabolic effects of sleep restriction found in the first two meta-analyses seem consistent with the finding that short sleep duration is associated with weight gain in observational studies (particularly in younger persons), which perhaps indicates that the adverse metabolic effects of sleep restriction don’t wane with time. 

These changes seem pretty bad. It’s unclear to what extent they translate to long-term health effects (see the section below) but those are things you perhaps should not ignore when investigating whether sleep deprivation is bad for your health. 

The evidence for a U-shaped association between sleep and mortality is actually pretty weak 

Guzey doesn’t talk about the supposedly U-shaped association between sleep and mortality on Theses on Sleep itself, but he has brought it up in a couple of other places (in the conclusion of his takedown of Walker’s Why We Sleep, and on a comment in the EA Forum) so I thought I’d tackle it here. 

For those who don’t know, a lot of epidemiological studies seem to show that the association between sleep duration and mortality is something like this:

(This specific chart comes from Shen 2016). 

But Kurina et al.’s “Sleep duration and all-cause mortality: a critical review of measurement and associations” finds that this U-shaped association is suspiciously restricted to a very specific type of study (emphasis mine): 

One interesting pattern among studies using survey sleep measures is that all of the studies reporting U-shaped associations measured sleep duration with questions about typical nighttime sleep or 24-hour sleep (Table 2). None of the studies that asked about usual bedtimes and waking times reported a U-shaped association; rather, they reported either no association [12,22,23] or only a long sleep association [6,9,34,99], or, in the case of two studies of young to middle-aged Japanese men, only a short sleep association [12,22]. That the U-shaped associations are exclusively found in studies asking about usual sleep duration may be informative and suggests the possibility of systematic response biases, with people in generally good health more likely to give a “normative” response (i.e., 7 or 8 hours) and those in worse health more likely to give a “non-normative” (shorter or longer duration) response.

It seems that you can’t take people’s reports of how many hours they sleep at face value. Further evidence of that is that studies have shown that people seem to give the same answers when asked how many hours they sleep and how many hours they spend in bed. Also, a substantial fraction of older adults (47% in this study) report sleeping >=8 hours a night, but when their sleep is actually measured, very few of them do, as pointed out in Kurina et al.’s paper. 

Kurina et al. look into studies that actually measure people’s total sleep time, and report the following:

A study employing actigraphy among women 50 to 81 years of age (n = 444) concluded that the relationship between sleep duration and mortality was U-shaped [... but] death rates by more detailed sleep categories do not show a dose response for either side of the duration distribution. The top and bottom categories (<4.5 or >7.5 hours) have relatively low mortality, albeit with small numbers, and the highest mortality risk was observed in women sleeping either 4.5—5 hours or—interestingly—between 7 and 7.5 hours.

Neither of the two studies employing polysomnography reported significant associations between sleep duration and mortality [27,36]. In both studies, sleep duration was dichotomized at fewer than 6 and 6 or more hours, precluding the possibility of finding a U-shaped effect. 

This isn’t exactly strong evidence that sleeping too much or too little is not harmful. This review only mentions three studies examining the effect of measured sleep on mortality, and two of those didn’t record sleep duration in a way that would make them able to find a U-shaped relationship. Moreover, all of these naturalistic studies probably have substantial range restriction — they mostly involve elderly or middle-aged subjects, many of whom can probably sleep as much as they want, and it’s fairly plausible that very few of those people manage to consistently sleep so much or so little that it’s harmful. Also, they measured sleep for one night only, and the study equipment might have disrupted the sleep of subjects. 

However, this review does provide evidence against six hours of sleep being associated with the lowest mortality, as Guzey has hypothesized before (as well as evidence against eight hours of sleep being associated with the lowest mortality, of course). And thus, most importantly, it also provides evidence against moderately short or moderately long sleep being harmful.

Conclusion

So here are my credences on a few relevant object-level claims:

  • Shorter-than-average sleep is associated with both depression and mania: 95%
  • Modulo weird edge cases like abolishing REM sleep exclusively, a single night with 4.5 to 6 hours of sleep has a negative impact on cognition the following day: 96%
    • Conditional on this, cognition does not return to baseline levels after several months of sleep restriction: 83%
  • Also modulo weird edge cases, consistently sleeping between 4.5 and 6 hours per night as a non-elderly adult has some long-term negative health effects: 75%
  • It’s not worth it to stress out about how much you’re sleeping, as long as you feel comfortable and productive (and you’re not (getting) manic)[4]: 63%

I’d be happy to bet on reasonable operationalizations of those statements at those odds (unless I change my mind, in which case I’ll probably edit the credences listed here).

Addenda

Mania is really, really bad, and can be triggered by sleep restriction

 I thought I’d bring this up, since it affects how harmful we should expect sleep deprivation to be in expectation. Two years after being hospitalized for a manic episode, less than half of people regain their premorbid occupational and residential status, and the hospitalization rate of mania is pretty high, so manic episodes seem to screw people over really badly for a long time. 

And sleep deprivation might risk triggering mania if you have bipolar or a high risk of getting it. A night of total sleep deprivation seems to be able to trigger full-blown mania in a substantial percentage of people with bipolar disorder (even those currently depressed) and even cause mania-like behavior in healthy subjects. Moreover, a shift towards mania or hypomania after a short night of sleep seems common in bipolar patients. 

Given how bad mania is, it might be a good idea to try to decrease even a relatively small chance of getting it, so those at a high risk of getting bipolar disorder should probably consider these effects before experimenting with sleep restriction.

 

Amusing r/NewParents anecdotes about sleep deprivation

I collected a few amusing anecdotes about sleep deprivation from r/NewParents. 

From this thread, titled “Tell me you’re sleep deprived without telling me you’re sleep deprived..”:

I just unzipped my 1 month old’s sleeper to expose his nipple so I could feed him…… 🤨

 

My husband attempted to hand me a cat to nurse.

 

"There was 120ml in the bottle, there is now 50ml in the bottle, so she's drank...

Okay, this isn't that hard, there was 120 in the bottle, there is now 50 in the bottle so she's had...

Right the bottle has 50ml, she's drank 120 so there should be.. no, she's drank 50 there's 120 left so, NO!

there was 120 in the bottle, there is now 50 and it goes 50, 60, 70, 80, 90, 100, 110, 120 so she's had 120.

NO! "

i genuinely gave up and used a calculator

 

From “Finish the sentence: I was so sleep deprived I....”:

Filled the dryer full of wet clothes and tried to turn it on via the microwave sat above it

 

...ran into a laundry basket and said "excuse me" to it.
 

 

Turned on the Keurig to make my coffee and walked away without noticing that I forgot to put my mug under it. The worst part is I've done this 3 or 4 times now.

 

Forgot to put a new diaper on after taking dirty one off. Baby went commando for a while.

From “Funniest thing you’ve said while sleep deprived in the dead of night”:

Nothing will ever top my husband waking me up saying "please take the baby I can't stay awake any longer" and gently passing me a very pissed off cat that had been asleep on his lap. Note, cat weighed twice what our newborn weighed. When I told him that was a cat he looked terrified and went "but then where's the baby?!"

 

My husband and I woke up to our 5-month old crying and he said, “Is that ours??” Yes, my good sir, that is in fact our baby.

 

It was my turn to wake up for a feed and my wife woke up first so said “babe, wake up she’s crying” and I responded “what? who the hell would be crying in our house?” It was 2 weeks after our daughter got home and she’s our first 🙃 

Errata

(Let me know if you think something should be added here that hasn't been.)

  • I used to have an additional quote in the section about BDNF, saying that insomniacs had lower levels of BDNF than sleep-healthy controls in a study. Guzey noted that insomniacs often underestimate how much they sleep; which prompted me to look up the objective sleep duration of insomniacs and not find much evidence that they sleep less than other people. So I removed the quote.
  • The last paragraph of my section about mortality used wording that seemed to imply that Guzey had hypothesized that sleeping 6 hours per night was causally optimal, in the sense that people should sleep that much if they want to have the lowest mortality. Guzey did not actually hypothesize that. He pointed that out and I reworded the paragraph. 
  1. ^

    This study looks for publication bias in the literature and says the following about what they found: "Evidence for publication bias was observed for the overall effect and the effect on measures of sustained attention; see Table 3. However, the impact of such publication bias on the effect appears to be minimal as another 75 studies with an effect size <0.0 would have to be added to result in a small overall effect size (g < −0.200)."

    Orwin’s fail-safe N was used to calculate that number.

  2. ^

    Note that the β for cumulative days of restricted sleep is negative, providing evidence against Guzey’s hypothesis that cognitive adaptation occurs after several days of restricted sleep, at least in the timespans investigated in the studies included in that meta-analysis.

  3. ^

    Importantly, although this meta-analysis included total sleep deprivation studies, the numbers reported in the sentences I quoted are for partial sleep restriction only.

  4. ^

    This is a very important caveat. People with mania will feel perfectly comfortable and productive while sleeping very little.

Counter-theses on Sleep
New Comment
131 comments, sorted by Click to highlight new comments since:
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

70% of 84 hunter-gatherers studied in 2013 slept less than 7 hours per day, with 46% sleeping less than 6 hours.

The study in question also found that the hunter gatherers spent 8-9 hours in bed each night. Sleep duration was measured using those Fitbit trackers that always tell you that you only slept four hours when you’re sure you slept about eight hours. If you move around in your sleep then the tracker assumes that you are awake.

This is an important point. When discussing sleep we have to both look at sleep as measured by devices (that do notice awake times during the night that aren't remembered) and at time in bed. 

It's plausible that 7 hours of sleep are good for the average person while 8 hours in bed are necessary to achieve that for the average person. 

8Natália
The range the study gave for time in bed was actually 6.9–8.5 hours (not 8-9). But it's a good point that subjective sleep duration reports significantly overestimate actigraph measurements. 
3Dustin
For what it's worth, in my experience there is a wide delta in accuracy between bad fitbit-style sleep trackerse and good fiftbit-style trackers.   I do not know if this study used good or bad devices.

I love the thought and research you've put into this, and I'm excited to see this dialogue continue.

When Guzey posted his "Theses on Sleep," I spent a lot of time going through meta-analyses and considering whether the underlying studies were sound. I talked about what I found in the comments there, and I wonder if you got a chance to read them?

I didn't look at the meta-analyses you cite here. The ones I did look at, though (Pilcher and Huffcutt on cognitive impairment, and Irwin, Michael R., Richard Olmstead, and Judith E. Carroll on inflammation), for me surprised me with how unconvincing they were when I looked under the hood. I think it would be valuable to read some measured reviews of individual sleep research metastudies.

Based on my experience with these two meta-analyses, simply being shown additional meta-analyses in this field finding a cognitive impairment effect doesn't update me much in the direction of there being a real issue. I would need to be confident that the studies it chose to analyze were relevant to the question at hand.

One of the distinctions Guzey made that I think was important was the difference between work-related fatigue and sleep deprivation. Many SD... (read more)

One of the distinctions Guzey made that I think was important was the difference between work-related fatigue and sleep deprivation. Many SD studies are on resident doctors at the end of 24-hour shifts. They are experiencing both work-related fatigue from a notoriously taxing job and sleep deprivation. As such, if we care about SD as opposed to fatigue, then such studies are hopelessly confounded as far as relevance for our specific research question.

I do think that that's an important distinction. Note that most of the studies included in the sleep restriction meta-analyses I quoted (all but one, in fact) are not on resident doctors, and, as far as I know, they pretty much exclusively examine the effect of sleep restriction (sleeping fewer than 6 or so hours per night) rather than the effect of staying awake for an abnormally long time.  

I didn't look at the meta-analyses you cite here. The ones I did look at, though (Pilcher and Huffcutt on cognitive impairment, and Irwin, Michael R., Richard Olmstead, and Judith E. Carroll on inflammation), for me surprised me with how unconvincing they were when I looked under the hood.

I did actually cite Irwin, but note that my conclusion... (read more)

I didn't notice that it was you I was originally responding to - I apologize for the oversight! I also want to emphasize that I agree with you on some of your responses to Guzey. I think a lot of his arguments are weak, his Reddit- and self-supplied supporting evidence shouldn't be stacked up against peer-reviewed controlled sleep studies, and some of the argumentation comes off as a conspiratorial strawman (i.e. "At this point, I’m pretty sure that the entire “not sleeping ‘enough’ makes you stupid” is a 100% psyop.").

In the first of the meta-analyses you posted (Lowe, Safati, and Hall), I see some supporting evidence for your position, and some complicating factors. From the abstract:

This effect held for executive functioning (g = −0.324, p < 0.001), sustained attention (g = −0.409, p < 0.001), and long-term memory (g = −0.192, p = 0.002). There was insufficient evidence to detect an effect within the domains of attention, multitask, impulsive decision-making or intelligence.

So first, let's acknowledge that they found significant, moderate effects in two areas that we may very well care quite a lot about! However, the long-term memory effect would be conventionally categori... (read more)

5Natália
The article specifies that it used Orwin’s fail-safe N to calculate the number of missing studies required to reach a small effect size. It’s not as good as the standard trim-and-fill method I’ve seen a lot in meta-analyses. But it makes mathematical sense and provides evidence. 
3DirectedEvolution
Ah, perhaps the "<" is a typo.
5Natália
(Quick reply, will go into more depth later) I think the small effect sizes are not as important for predicting overall cognition as the larger ones. If damage happens to a specific part of the brain and not others (as in e.g. most non-fatal strokes), that causes a lot more functional impairment than you would expect if you focused on all of the many parts of the brain that weren’t damaged. A lot of the variability in the effect sizes is caused by perfectly reasonable things like variations in sleep deficit and cumulative days of restricted sleep, as I explicitly pointed out in the post, and the type of cognitive test used is probably responsible for a substantial fraction of the variability too. The meta-analysis itself tells you the average sleep deficit across the studies compared with age-adjusted sleep recommendations, as I pointed out in my post, and that number is probably more relevant than those averages. Note that one of my meta-analyses found that (the absolute value of) the effect size of sleep restriction on attention is higher when the comparison group undergoes sleep extension rather than when it undergoes no intervention. This is at least weak evidence that the dose-response relationship between sleep duration and cognition doesn’t flatten out once you get to the average amount of time people sleep. (This is consistent with me own experience; I started taking less time on assignments and getting better grades in college after I started forcing myself to sleep longer than my previous higher-than-average baseline.)
3DirectedEvolution
If I'm understanding you correctly, you're saying that you think that a 1-2 statistically significant moderate effect sizes are more worrisome than if this research had discovered statistically significant but small effect sizes in every category. You expect that functional impairment on real-world tasks would be serious and wide-ranging as a consequence of moderate impairment in a single cognitive domain. First, I think this is a great crux, and highlights the importance of carefully framing a research question. Second, I think we ought to consider a few possible functional performance scenarios: 1. Tasks that stretch cognitive capacity to the maximum extent, often by design. Example: a chess championship or math exam. 2. Tasks that typically use limited cognitive capacity, but in which mistakes can have severe consequences and rare emergencies can occur at any time that will result in much higher cognitive demands than normal. Example: jet pilots, soldiers on patrol. 3. Tasks that use limited cognitive capacity, allow for corrections and adjustments, and are primarily bottlenecked by time, skill, and material resources. Example: most jobs, tasks of daily living, sailing competitions. I think that scenarios (1) and (2) are the best candidates for the hypothesis of greater functional impairment effects relative to limited and modest cognitive function deficits, and scenario (3) is best for the hypothesis that modest and limited cognitive function deficits aren't too concerning. Some challenges might even be mixed. For example, we could imagine that in school coursework, learning is bottlenecked by hours of study in general. However, on exams, for well-prepared students, hours of sleep might become the limiting factor on their performance. Hence, an idealized study strategy might want to routinely sacrifice some sleep in exchange for more study time, but then increase sleep on the days leading up to an exam. Under this mixed framework, a strategic sleep regi
7Natália
I am interested in this, so feel free to contact me if you want to start this.

I am also interested, especially in "what tools can help people develop data for themselves, that can be aggregated in useful ways?" 

One way I think sleep science and most everything else goes wrong is in looking for the modal human answer, ignoring how the right amount of sleep/nutrient X/exercise/etc vary between humans and for the same human over time.

2DirectedEvolution
Excellent. Will do. I'm also reaching out to Guzey. Maybe we can get a bit of a community effort going!

I wear an Oura ring and and Apple Watch with a sleep app. Both of these devices agree on when I'm underslept, and they are both correct; when my watch says I'm underslept, I feel stupid and tired and my chess.com scores plummet. My chronic pain condition is also much worse when I'm underslept. Additionally, I do not use an alarm clock, so my body will claw back the sleep it needs. If I only get 6 hours two nights in a row, I will sleep 9 hours the following night, but I habitually wake up after 7-8 hours. I can observe these patterns in my recorded sleep data, and they are robust over long stretches of time.

I say all of the above because frankly my own personal experience and data tracking is sufficient evidence for me to basically disregard any sort of thesis claiming that I need less sleep. Maybe you need less sleep, I don't know. Do the experiment, try to sleep less for a couple of days, see if you physically implode. I would put money on the "you will probably learn that you were already pretty in tune with your body's needs" outcome.

I think Natalia brings up a lot of important, true, points here, and I'm really glad she wrote this post. I also stand by what I said when I curated Alexey’s post, that while I was doubtful on any particular conclusion I thought he was pointing at real deficits in current models and I was excited to see those explored so they could be eventually filled in. My current guess (having not deeply investigated the evidence for either post) is that this post is closer to the literal truth, and Alexey’s post is pointing in a more useful direction (but we will get there in part by people pointing on the parts that are false, via posts like this).

The version of Alexey’s theses I most broadly support is “we treat sleep like one thing, when it is in fact multiple things with multiple purposes”. Natalia calls this epicycles, I agree with AllAmericanBreakfast that it’s more like scurvy before the vitamin C paradigm. There’s a real thing there, but without the paradigm it looks incredibly fake and exactly like epicycles. In general I think distinguishing ideas analogous to pre-vitamin-C scurvy from ideas like epicycles is one of the Hamming problems of science, and would cheer development in the ... (read more)

[ETA: to be clear, I’m not criticizing the thesis of SMTM’s post here, just pointing out a factual error]

The linked SMTM post is misleading. 

Here is the vitamin C content per 100g of some relevant foods, which I found after a few minutes of searching on Google:

  • Lime juice: 30mg [1]
  • Lemon juice: 38.7mg [2]
  • Raw lemons: 53mg [3]
  • Raw limes: 29mg [4]
  • Key limes in particular, which Wikipedia says are the same thing as West Indian limes: 31.3mg [5]
  • Raw caribou liver: 23.874.9 mg [6]
  • Raw ringed seal liver: 23.873.8 mg [6]
  • Raw cattle liver: 71.2mg [7]
  • Raw buffalo liver: 72.4mg [7]
  • Raw sheep liver: 77.6mg [7]
  • Raw goat liver: 76.7mg [7]
  • Fresh/raw polar bear meat: 1mg [8] [9]

I couldn’t find information on the vitamin C density of polar bear livers in particular, but from these values, it seems far from clear that polar bear livers are more similar to lemons than limes in that respect. The vitamin C contents of limes and lime juice do not stand out in that list. 

Moreover, it seems that it only takes about 10mg of vitamin C per day to prevent scurvy, and the Manual of Nutritional Therapeutics says that the same daily quantity is enou... (read more)

5DirectedEvolution
1. Copper tubing and environmental exposure does destroy vitamin C At first glance, I thought you were claiming that copper tubing and air exposure does not destroy vitamin C, but now I think I read you backwards. I think you are claiming that they do destroy vitamin C, and that copper and O2 destruction of vitamin C was the only reason that "lime juice" was failing to prevent scurvy (and that the differences between lemons vs. limes had nothing to do with it). Just to emphasize that vitamin C is susceptible to destruction by copper and environmental exposure, here's the Australian Ministry of Health: 2. Liver and vitamin C Polar bear liver contains massive amounts of vitamin A, enough to kill humans (1, 2). When the Idle Words essay on which the SMTM post is based says "Eat a bear liver every few weeks and scurvy will be the least of your problems," the proper response is "because you'll be dead of vitamin A poisoning." However, the IW post vascillates between saying that the vitamin C in the polar bear comes from "meat," "liver," and "kidneys," and I don't trust that the author carefully investigated the nutritional value of each of these organs. However, I did find a source that talks more about the Inuit and early European arctic explorer diet. Raw bear meat has a little more vitamin C (2mg/100g). So this would require eating 500g bear meat/day (about 650 calories, or 1.1lb per day) to get an adequate quantity to prevent scurvy indefinitely, or less merely to prevent it for the duration of the voyage. Scurvy takes 3 months to set in if vitamin C intake is extremely low. I think we have adequate evidence to say that bear meat could have prevented scurvy in Arctic explorers if lightly cooked. 3. Sourcing the claims on limes vs. lemons I found a source containing the claim of lemons being 4x as powerful an antiscorbutic as limes, which may be the origin of the IW/SMTM claim. In fact, it contains several of the essential points in the story conveyed by these
1Natália
Yeah, your current interpretation is correct.  It's not that different from the raw limes in USDA's data. In the titration method, that article found that musk limes had 42.3% as much vitamin C as lemons, and in the HPLC method, 53.5%. The average of those is 47.9%. My data on non-musk limes suggest they have 54.7% as much vitamin C as lemons.  I'm confused. The number I listed for lemon vitamin C was 53mg/100g. To be clear, I wasn't arguing against that. I was just pointing out that it's misleading to claim that polar bear meat is more like lemons than limes are. All 3 of those things seem able to prevent scurvy when they're fresh and consumed in reasonable quantities; even 1/4 of the vitamin C concentration of fresh lemon juice is 23.22mg per cup. It's possible that a citrus species has an abnormally low vitamin C concentration, and it's possible that 10mg of vitamin C from lime juice is not the same as 10mg of vitamin C from fresh meat. But SMTM was asserting as a known fact that Sicilian lemons are more like polar bear meat than they are like West Indian limes in their ability to treat scurvy, which in expectation causes people who read the post to be worse at predicting the data we found than they would be if they had never read it, and that is what I was objecting to. 
3DirectedEvolution
Apologies, I missed several of these points in your OP at first, and edited my post to correct it. Sorry you saw it before it was fixed!!! I agree with your characterization of SMTM's argument. It's hard to parse, given the reference to his own made-up dialog that's partly to make (supposedly) factual claims and partly to illustrate the challenges of analyzing complex scientific data. However, I read this as SMTM in their authorial voice defending the claim that key limes don't contain vitamin C and polar bear meat does. We both agree this claim is false. Here's the relevant quote: I think that SMTM got this fact wrong. However, I think that the essential point of the SMTM article holds, and by extension the analogy I am drawing between vitamin C and sleep. Thank you for your patience with my original slip-ups in noticing the full content of your preceding comment.
5Natália
There was also another relevant passage: 
3DirectedEvolution
Followup: SMTM points out that differences in fertilization and propagation (clonally or by seed) mean that we don't know whether or not historical key limes had enough vitamin C to cure scurvy. I notice in myself a certain tendency to say oh puh-leeze in response to this, but at the same time, I think the broader point of the original SMTM piece is that it's precisely this reaction that we ought to be suspicious of. After all, we don't necessarily know that the arctic explorers who found that lime juice didn't cure their scurvy were drinking it out of copper tubes, or that the vitamin C in the lime juice they were drinking was oxidized. Reality is surprisingly complex.
4Natália
Hm, that would still make the post misleading — “Sicilian lemons really ARE more like polar bear meat than they are like West Indian limes, at least for the purposes of treating scurvy” is a very different claim from “we don't know whether or not historical key limes had enough vitamin C to cure scurvy.” Thanks for contacting them, though!
2DirectedEvolution
Agreed!
3DirectedEvolution
I will see if I can get in touch with SMTM and IW, I'd be curious what they have to say about this.

The version of Alexey’s theses I most broadly support is “we treat sleep like one thing, when it is in fact multiple things with multiple purposes”. Natalia calls this epicycles

That sleep is multiple things is not something I am arguing against. (In fact, I don’t recall that being a point in Guzey’s post). What I uncharitably called “epicycles” was the additional complexity Guzey’s model has to have to explain why so many people feel dumber after sleep restriction, and why experimental studies say that sleep loss causes cognitive impairments, when ““not sleeping ‘enough’ makes you stupid” is a 100% psyop.”

It’s pretty clear to me that sleep has multiple effects, and that it might be the case that there’s something with all of sleep’s good effects and none of the bad ones that just hasn’t been discovered yet. Maybe digital people would only have to spend the subjective equivalent of a few seconds per day shut down in order to renormalize their weights or whatever, or might not need to be shut down ever at all to maintain their performance. 

But I don't think that is incompatible with the object-level claims in my post, any more than saying “humans don't live to be 200 years old” ... (read more)

2Elizabeth
Sounds like we agree a lot- I think he got a lot of stuff wrong and was glad to see you correct it. I think he wildly overextrapolated from his own data. My opinion on mainstream sleep researchers is closer to his than yours, but agree he overextrapolated from the sailing sleep coach.  I spend a lot of time going "that person's big idea is built on a foundation of factually inaccurate sand", so it's novel for me to be the one going "yes, mostly, but I still want to follow up". I feel this way due to a combination of thinking Alexey brings up a lot of legit problems with existing sleep research, and that he put serious time into generating new data (although I was extremely disappointed in his data collection, and would have found even writing down his subjective score each day an improvement over nothing). 
7Natália
  Could you elaborate on what you mean by "mainstream sleep researchers" and/or "existing sleep research," and what, in your view, the legit problems with the latter are? (Modulo Walker). These terms can mean many things and refer to very different sets of models. Researchers, the organizations that set guidelines, and clinicians in a certain field can often have very mistaken views about what the research in their field actually shows if they’re not personally involved in it. For example, some clinical psychiatrists promote the serotonin theory of depression even though the research body does not support it. Lumping together the clinicians with the research body as a monolithic "psychiatric field" obscures this. (I'm not sure that that's what's going on here but discussing what you mean specifically could be helpful.) Some of Guzey's points on Theses on Sleep and Matthew Walker's "Why We Sleep" Is Riddled with Scientific and Factual Errors are genuinely both counterintuitive to the general public (and some/most sleep researchers, guidelines-setters, and clinicians) and true. But Guzey's evidence for those points didn't come out of the ether (and most of it did not come from his own experiment). It largely came out of the academic research body on sleep, the same place in which I found evidence that sleep duration e.g. affects cognition but barely has a clear association with mortality. So it seems reasonable to me to dissociate the research body from the opinions of most people in the field.  (Also, I think it’s very unclear what view you’re ascribing to me in the first quote, but I suspect that I do not endorse it. I don’t expect sleep scientists to be that good at predicting reality in subfields or sub-subfields they’re not personally involved in.) 
2Natália
[An earlier version of this comment said “do endorse it” when I meant the opposite, sorry for the confusion.]
9DirectedEvolution
I'll take a stab at this. It seems like the key distinction is separating ideas that are heavily but still inadequately tested (epicycles) from those that are underexplored (pre-vitamin-C scurvy) and those that are thoroughly tested (the pathogen theory of disease). Researchers are bound by convention, resources, and the limits of their own imagination and enthusiasm. As such, they often wind up using experimental methods that both fail to distinguish empirical confounders and can never yield a better underlying mechanistic account of the phenomena at hand. Instead, they just put out another study as meta-analysis fodder. Take this sleep research. We can find dozens of studies showing that X hours of sleep deprivation results in Y amount of cognitive deficit. But have the scientists designed studies to test for weird confounds? Maybe being sleepy doesn't make you cognitively slow, but instead makes you socially anxious. Being around a bunch of strange scientists subjecting you to unusual tests in an unfamiliar environment could trigger anxiety. This in turn could explain some of the cognitive deficits in the sleep research. Sleepiness -> social anxiety -> cognitive deficits around strangers. That hypothesis took me all of 1 minute to come up with, and it might be true, who knows? If resources were unlimited for testing it, we could just go ahead and do it. If it takes me 1 minute to come up with a probably-untested plausible alternative hypothesis to a field that has produced dozens and dozens of studies, it's a sign that the field is designing its experiments inefficiently, and we should discount the apparent strength of its findings. This doesn't mean that its current hypothesis is wrong, but that it has disguished alternative hypotheses by neglecting to test them and thus created a false impression of consensus. If it's not possible to test all the hypotheses we'd ideally like to explore, then too bad. Scientists don't get to say they've discovered "the tru
6jimmy
I don't think that's much of a problem. People get "hangry" too and that doesn't invalidate the usefulness of fasting nor does it imply that fasting has to be miserable like that. I think any prolonged discomfort that is unaddressed is going to make people cranky and bias them towards negative explanations of things. Address the discomfort, and the experience can change dramatically.  Perhaps the most useful thing I got out of experimenting with polyphasic sleep was a recognition of how well I could function when sleep deprived so long as I wasn't constantly battling an unaddressed urge to sleep. Sleep deprivation still made me dumber, but the crankiness and most of the dysfunction was actually a result of trying to ignore (and yet not completely tuning out)  my body screaming at me to sleep. Having a reference experience for being sleep deprived without craving sleep made it a lot easier to function when an hour or two short, and also more willing to go take a nap when I need one.
3Natália
I don't think the last point is not explained by mainstream sleep models.  Around 440,000 people in the United States are 3 standard deviations smarter than the US average.  That, together with the fact that some jobs (even some that are common among those 440,000 people) might have higher marginal returns to hours worked & past experience than to cognitive performance, makes me unsurprised that some people remain astonishingly productive for years on very little sleep. I'm not sure what would count as "mainstream sleep models" w.r.t sleep deprivation's antidepressive effects, but a lot of studies seem to confirm that acute SD increases BDNF, as Guzey pointed out.
2Elizabeth
I'm surprised to hear you say this and am now less sure what you're arguing against. 
3Natália
I think mainstream sleep models don't fail to predict that some people remain astonishingly productive for years on very little sleep. The meta-analyses I included showed moderate effect sizes for the impacts of sleep restriction on cognition, which aren’t enough to decrease a large baseline advantage in cognitive performance that much. 
2Elizabeth
okay. It sounds like the words "sleep deprivation" are doing a lot of work in the original post. You don't mean "sleeping much less than the average", but literally "depriving yourself of sleep you need"? Regardless of what mainstream science says, what gets filtered out to the public is very much "get N hours or you're going to die", if sleep scientists have the concept of standard deviation in their models I'll be impressed.
9DirectedEvolution
My interpretation is something like this: 1. Let's say that long-term sleep deprivation costs 15 IQ points (1 standard deviation) in terms of cognitive function. 2. Furthermore, let's imagine Bob is a researcher with a baseline IQ of 145, while his coworkers have IQs of 115. Bob is chronically sleep deprived, so his functioning IQ is 130. 3. Bob's functioning IQ is still above that of his coworkers, and he's also able to work more hours, and therefore will be astonishinly productive relative to his coworkers. 4. However, it may still be that his productivity would be higher still if he slept better. What he lost in time, he'd more than make up for in productivity per hour of work. 5. Therefore, the existence of rare, astonishingly productive, sleep-deprived workers is compatible with sleep deprivation being harmful to cognitive function and net negative to productivity even for these workers. Let's assume that at a research university, the mean IQ of faculty is 115, still with a standard deviation of 15 IQ points. In this case, Bob would merely be in the top 5% for IQ relative to his colleagues. At a major research university, with 5,000 faculty, there would be 250 faculty like Bob. We can imagine that their level of sleep deprivation would be normally distributed as well,  and so we could find quite a few examples of people like Bob just by looking around. Natália, is that a fair representation of the point you're making here?
4Elizabeth
I think that makes sense (and wasn't what Natalia was saying, but is compatible with it. Bob could be smater than average, have a lower optimal sleep length, and have fewer consequences per hour of foregone sleep ). I want to highlight that there are many tasks that don't benefit from the additional 15 IQ points, and if your total output depends more on those than on tasks that benefit from every last IQ point, then trading IQ points for more hours in the day can be the optimal move. But if Bob got smarter co-workers or became less resilient to sleep loss the trade-off might be different, and it's beneficial for him to check the trade-offs periodicially. I don't think that's what Alexey or Natalia were arguing, and it ignores health effects, but what I'm excited about in this discussion is rendering these kinds of trade-offs able to reasoned about
6DirectedEvolution
I agree about your point on making trade-offs possible to reason about. I’ve found that in discussing this away from LW, people will talk as if extra hours of sleep don’t come at the expense of waking hours. To them, at first glance, the decision is between “activity X, Y, and Z with 6 or 8 hours of sleep” rather than between “activity X and Y with 8 hours of sleep, or activity X, Y, and Z with 6 hours of sleep.” When I point this out, the follow-up is “but what if the lost sleep makes activity Z a negative experience?” When we eventually anchor on that being the question at hand, rather than the assumption by default, the conversation can become productive and interesting. I also agree with your bottlenecks-based framing. I think some tasks are bottlenecked by cognitive function (chess competitions), others usually aren’t but can be in rare/emergency circumstances that therefore require full-time high cognitive function (airline pilots), and many others aren’t bottlenecked by cognition (sailing races).
6gbear605
One counterpoint to that framing is that activities usually don't take a fixed amount of time. There are many things that I've done where I'll take a quarter of the time or less if I'm in a productive state rather than a non-productive state. Sometimes it might really be a trade off between "activity X, Y, and Z with 6 or 8 hours of sleep", or even "activity X, Y, and Z with 8 hours of sleep, or activity X and Y with 6 hours of sleep". I've also found that I'm less distracted with more sleep, so it could be "be productive for the most of the day with 8 hours of sleep, or browse social media for most of the day with 6 hours of sleep". This all has been most obvious when I've been working on intellectual activities, like writing or developing software, but I also tend to have slower reflexes when I've been sleeping less, so physical activities are also relevant. My personal sleep logs show that productive states are strongly correlated with the amount of hours of sleep I've gotten over the last month and the previous night. I don't have proof that the math really works out, but I strongly suspect it does. Over longer term periods where I've had more or fewer hours of sleep, I've gotten more positive outside feedback (grades, feedback in monthly reviews) in the the periods with more sleep. But of course I have no proof that this applies to everyone, it's correlation instead of causation, and it's certainly no double blind study. And of course the hardest problem is beating the akrasia to go to bed when I want to, but that's a separate issue.
3DirectedEvolution
Agree! I think we are coming to a model where N hours of SD results in an average of X% loss in productivity per hour, and a change from Y to Z daily productive hours. We want to know the sign of XZ - Y. Here, you may be arguing both that the performance loss is so great as to more than cancel out the increased number of working hours. You may also be arguing that we actually lose productive hours with sleep deprivation, despite being awake longer, because the loss of executive function makes us unable to focus on our work. Alternatively, it may be that the additional productive hours from SD can be put into low-demand but time-consuming life maintenance or pleasurable occupations that otherwise might get neglected. For example, we can imagine a busy student choosing between 8 hours of sleep, or 6 hours of sleep plus 2 hours of socializing. Or between 8 hours of sleep, or 7 hours of sleep plus a trip to the grocery store. Or between 8 hours of sleep and 7 hours of sleep plus 1 hour of exercise. Sacrificing sleep for efficiency-compounding tasks and meaningful occupations may cancel out or even reverse the cognitive impacts of SD itself. It may be that the worst effects of SD can be mitigated with a combination of practice, organizational supports, and stimulating activities and caffeine. It may also be that for some people, their baseline need for sleep is below the level recommended by sleep researchers, so that for them, reduced sleep comes at no cognitive cost. So the question is twofold. Does non-strategic mild-to-moderate SD lead to net positive or net negative results on some outcome, like productivity, pleasure, health, or meaningful experiences? Furthermore, to what extent can an SD-management strategy mitigate the negative impacts of mild SD? The stakes are rather significant.  Consider that 2 hours of daily SD from age 20-60 adds up to 3.3 years of additional wakefulness, distributed across the most productive and healthy part of the lifespan. If it r
7Natália
I tried to be explicit about the average number of hours slept in the studies included in these meta-analyses of experimental studies. Usually, it's about 4 to 5 hours, with the average seeming to be closer to 4. I also tried to be explicit about how many hours of sleep per night my claims refer to in my conclusion (except for the first claim, which I specified was about "shorter-than-average" sleep, and the last one, which is a harder-to-operationalize claim, but still didn't explicitly refer to "sleep deprivation"). Standard deviations are used very often; I would be surprised if they weren't used in 5% or more of the sleep studies I've read.
2Elizabeth
Sorry yes, of course they're including SD calculations in their papers. What I meant was that medical papers (or popular coverage thereof) almost never consider the implications of the fact that 0.3% of people will be >=3 standard devs out, and with a large enough population that's a lot of people. 

I've been meditating on the broader relationship of this post, and Guzey's, to the idea of instrumental rationality and the question of what rationality is good for. A idea is that instrumental achievement is bottlenecked by akrasia.

When we study sleep, we're really studying one aspect of fatigue, which is similar if not identical to akrasia. We can think of fatigue as being subdivided into physical and experiential components. Like other forms of fatigue, sleep pressure manifests both in physical brain structures and in our conscious, felt experience.

Sleep may be a specific solution to a specific physical or felt need, and our sense of sleep pressure may be well-tuned by evolution to indicate the amount of sleep required for optimal function. Sleep may also be a nonspecific solution to these needs, and sleep pressure may be only marginally attuned to the timing and duration of required sleep.

More broadly, we have agentic goals. To accomplish them, we generally must maintain our minds and bodies in order to work efficiently and maintain motivation, while also freeing up time to do that work.

It is possible to carefully calibrate many of our routines. For our diets, we can count calo... (read more)

(I'll reply in more substance by the end of the week -- have a big deadline coming up this Thursday)

From skimming the post, three parts stand out to me

1. the fact that acute sleep deprivation relieves depression in ~50% of people with depression seems completely unaddressed and Natália's section about bipolar people seems to imply that this would not be happening. I specifically noted in this the section Natália addresses by writing:

Lack of sleep is such a potent trigger for mania that acute sleep deprivation is literally used to treat depression. Aside from ketamine, not sleeping for a night is the only medicine we have to quickly – literally overnight – and reliably (in ~50% of patients) improve mood in depressed patients

2.

One of Guzey’s theses is that “[o]ccasional acute sleep deprivation is good for health and promotes more efficient sleep.” His argument supporting that thesis is pretty much that, because some types of acute stress (such as exercising and fasting) are good, and acute sleep deprivation causes acute stress, then acute sleep deprivation is also good. (Yes, that does seem to actually be the entirety of his argument in that section. You can read it yoursel

... (read more)

Natália doesn't set out to disprove all of your theses, but rather to put forth some counter-theses. She says:

I decided to write a post pointing out several of the mistakes I think he’s made, and reporting some of what the academic literature on sleep seems to show.

Read carefully, she neither claims that every point you've made is mistaken, nor to give a comprehensive review of the academic literature. So I don't think you can fault her for not addressing the point about the use of sleep deprivation as a depression cure. She's critiquing those theses of yours which she found weak, not issuing a comprehensive point-by-point criticism of your entire original post.

I think that you owe this level of care in interpreting her language, because you're insisting that she offer you that same level of care. You said:

people who sleep just 6 hours a night might have the lowest mortality

And then you complain when she rephrases this as:

six hours of sleep being optimal for mortality, as Guzey has hypothesized before (as well as evidence against eight hours being optimal for mortality, of course).

To my eye, these reflect approximately equal levels of imprecision, thought that's a purely subjective... (read more)

8Matthew Barnett
FWIW, I don't think the stubbed toe example is integral to the argument. The stubbed toe example was almost certainly just an analogy, not a knockdown argument, in regards to the more general point that the link between acute stress and health benefit seems to be weak. I think a better way of phrasing this crux is that it's unclear why this form of acute stress is beneficial, where this refers to sleep deprivation. It would be nice to get specific evidence regarding why sleep deprivation is the right type of acute stress to promote health, when the relationship does not hold in general. I don't think I'm objecting to your current summary of the debate necessarily, but I do think focusing on the stubbed toe example is mostly a distraction, and all parties are better off relying on different arguments.
5DirectedEvolution
I agree that stubbed toes needn't be a central example of "acute stress," and I would hope that most people are just using it as you suggest - a convenient analogy for a minor but pretty-clearly-bad form of injury, one that stands in contrast to  plausibly beneficial stressors like exercise.

1.

Natália's section about bipolar people seems to imply that [sleep deprivation's short-term antidepressant effects] would not be happening.

I disagree. I said,

A night of total sleep deprivation seems to be able to trigger full-blown mania in a substantial percentage of people with bipolar disorder (even those currently depressed) and even cause mania-like behavior in healthy subjects. Moreover, a shift towards mania or hypomania after a short night of sleep seems common in bipolar patients. 

Here, I think it was clear that what I said is consistent with sleep deprivation having antidepressant effects, and it could even be interpreted as implying that it does. So I think it's misleading to suggest that this section implied that the antidepressant effect does not exist.

2. Your section arguing that occasional sleep deprivation is good for health makes no mention of its antidepressant effects, which were addressed separately earlier on in your post. I thought you were making a separate argument in that section, which is why I countered with an appropriate analogy. I merely think that the argument "sleep deprivation causes acute stress, therefore it's good" is weak, and that was my ... (read more)

9DirectedEvolution
I don't quite follow. The closest you come in the quote is that it can "trigger full-blown mania," not that it is a depression treatment. Guzey says: I did not interpret this as saying sleep deprivation treats depression by causing mania. Instead, I think Guzey is suggesting that sleep deprivation treats depression through a neurological pathway that can also lead to mania in bipolar patients. I think it's fine to ignore this point if you're not interested in addressing it, but I don't think it's fair to characterize your reference to "trigger full-blown mania" as a clear acknowledgement of sleep deprivation's therapeutic benefits for some depressed patients. If that was indeed your conscious intent, then my feedback is that your writing was illegible in this area. As a caveat, it seems like you might have inferred from Guzey's overall post that he has a positive impression of mania. I think this is belied by phrases like "DO NOT TRY THIS IF YOU ARE BIPOLAR, YOU MIGHT GET A MANIC EPISODE," but not an impossible takeaway. I agree with your assessment, and Guzey really ought to read your article in depth before throwing shade on it. I think your commitment to constructive dialog is admirable here.

Switching to mania from depression usually (though not always) means that the depressive symptoms went away. But I agree that my phrasing was poor and I’ll edit it. 

Hi Natália,

I want to apologize for lack of proper engagement with the post and lack of replies to you as well as a high level of combativeness in the comments I did leave.

This stuff makes me anxious and I feel like I'm just unable to properly explain what makes me disagree so much with you. 

My best attempt is that my takes are some combination of:

  1. Academic literature is a highly adversarial playing field and although it's very tempting for smart outsiders to think that they can just go on Google Scholar and figure out what's going on, I don't think thi
... (read more)

I found this reply unpersuasive.

By numerical point:

  1. Speculation on OP's education is irrelevant. You reject lots of studies by PhDs that did study the field. If she misunderstood something, address the specific error.
  2. Deep skepticism of the sleep literature is fine, even if you rely on some sleep research yourself, but it's insufficient to respond to the objection of hypocrisy of relying on the sleep literature with "well, I'm really careful about which studies I use". You need to explain why the studies you use somehow avoid the methodological problems that cause you to reject other studies. If you don't, it seems like you are just cherry-picking supporting studies because they support you.
  3. It is SO strange to me that you rely so heavily on your personal experience, which is almost the least reliable scientific method available. Who's to say that your experience generalizes? You're a very unusual person. I'm always incredibly skeptical of people who have the position "everyone else's introspection is unreliable, but somehow I'm above all of that" or "you are all brainwashed by consensus, but I can figure out what's true and what's BS because I'm smart, educated, and careful".
  4. Your cla
... (read more)
9Rafael Harth
I don't know if this generalizes, but my experience with tone is that it's mostly unintentional. There've been many instances where I've written something that seemed perfectly appropriate to me at the time, only to be horrified at how sound when I read it a month later (and the result pattern-matches to guzey's comment). It also does not require a psychological trigger, it just happens by default when arguing with someone in text form (and it happens more easily when it's about something status-related like who made better arguments). Took a lot of deliberate effort to change the default to sounding respectful. I agree that it's bad enough to be worth mentioning, but I'd be quite surprised if it's the result of a strategic effort rather than of an unconscious signaling-related instinct.
4Dustin
I agree with you. I've found that I've lessened this experience of reading something I've wrote and being horrified at its tone by going back and reading my comments at various sites.  At least once a year I find myself going to my profile page at LW or some other site and just spending a couple of hours reading what I've wrote in the past.  I think this has helped me be more aware of what my tone is conveying.
8Natália
I have a tentative guess on why he's doing that, based on Scott Alexander’s post about trapped priors.  I’ll give an example of the basic problem outlined in the post myself, to spare you from having to read all of it before understanding my comment. Suppose that a physicist spends two hours trying to convince you that the Earth is flat. Would you see that as strong evidence that the Earth is flat? Personally, I’d see that as extremely weak evidence. Instead of updating much that the Earth is flat, the conversation would instead make me seriously consider the following more-plausible-to-me hypotheses:  * The physicist really enjoys pulling very elaborate pranks on people. * It’s April fools or something similar and I for some reason just haven’t realized it yet.  * The physicist currently has some sort of untreated psychosis.  * Any physics department the physicist has studied or worked on is extremely terrible and ought not to be trusted in the future.  * I am hallucinating, or otherwise have perceptions of the world that don’t track reality. Perhaps I’m in a dream, perhaps I have extremely-early-onset Alzheimer’s, perhaps I’m on a potent perception-altering drug.  Similarly, if I saw a lot of studies claiming to show something absurd like that prohibited-by-the-laws-of-physics "psychic" phenomena are real, I won’t need to read their methodology to conclude that there’s something wrong with them. And if a group of people claims to have such psychic powers, I won’t think twice before dismissing their personal experience as unreliable. And, at the same time, I'll accept, without batting an eye, studies and anecdotes claiming that such powers are not possible.  So dismissing arguments from experts, studies, and personal anecdotes as horribly flawed and no more than weak evidence — even before trying to assess their quality — is perfectly reasonable and Bayesian if they’re claiming something that you think is absurd. But clearly, if you have that attitude towa
6Benquo
If a physicist were to spend two hours trying to explain to me how they knew that the earth was flat, I'd expect to come away from that conversation with a better understanding of the physical world or the social construction of physics knowledge, which would better help me navigate my life, even if I ended up wronger on the bottom-line answer - because that's how epistemically persuasive explanations work, they have to show an ability to win bets either more often or with less computational cost than alternative hypotheses.
5Vaniver
I feel sort of weird about the 'trapped prior' point, because I think it's more reasonably pointed at academic fields than individual people? Like, it is not that surprising for an academic field to have 'core beliefs' that everyone who disagrees with is 'not in the field', given the forces that people in the field can exert on each other. One fun example of this is PhilGoetz's post Too Good To Be True, wherein he points to a claim that out of 60 studies studying vaccines and autism, none of them find any link, and then observes that by standard frequentist analysis, ~3 of them should have been significant at the p=0.05 level, and getting none of them significant at that level is pretty unlikely without suppression. And it's obvious why there would be suppression; no one wants to give ammunition to the enemy.   Separately, it feels like it doesn't really distinguish 'justified priors' from 'unjustified priors'. If you tell me that a psychology experiment found evidence of psychic effects, I will basically just not believe it. But hopefully if you transported my mind to a universe where psychics were real, I would believe the corresponding studies in those universes--because I had seen things like dowsing being used by oil companies in that universe. This is, from the perspective of a new study, 'my prior', but that prior is built out of all of the evidence that I've seen before. It's kind of fair to call my position on psychics a "trapped prior" but it feels more fair to call it "a mountain of evidence".
-1Yonatan Cale
Maybe better to assume good intentions? (And even if someone is biased or motivated by "impure" motives, we try to turn this into a high quality discussion if we can?)

Fair point, although I wasn't assuming any bad intentions, more like a hard-to-explain emotional intensity that seemed out of character for someone whose writing I am familiar with. But perhaps expressing my genuine surprise was not constructive—thanks. I removed this intro from my post.

2guzey
I think that if you ask anyone who knows me in-person they will tell you that I'm an unusually emotionally intensive person. My writing is also usually very emotionally intense but it tend to go through getting feedback from like 20 people who tell me to remove all of the excessive language and to tone it down before publication, so it ends up sounding normal. Comment do not go through this kind of process.

Thanks for engaging. I hope we can figure out what our cruxes are. 

On points 1 and 2: I still think you haven't addressed my counter-argument to points of that nature, which I've raised (1) in the post itself, (2) in my previous reply to you, and (3) in this comment. To reiterate some of it: you use sleep research to support some of your points, mostly in your review of Walker's book but also in Theses on Sleep, and it's not clear to me why research that shows that sleep deprivation is not as bad as people think is admissible evidence to you but research that shows that sleep deprivation is not harmless is not. I was, and remain, skeptical that you have a consistent and rigorous standard for what research you think is admissible and what research you think is not.

...

Just to reiterate: I believe that the points 4, 5.4.1, and 5.4.2 invalidate large chunks of sleep literature and are simply not possible under the default "sufficient sleep is good and necessary for proper functioning for normal people on the scale of a few days to a week".

Notice that my post does not argue against anything that is invalidated by 4, 5.4.1 or 5.4.2. 

As I elaborated before in my post, 5.4.1 pret... (read more)

-9guzey

This isn’t as important as my previous reply (in which I address your object-level arguments), but I wanted to perhaps note that most of your points 1.1 through 1.4 sound, to me, more like an attempt to generate an emotional reaction in the reader than a good-faith effort at pointing out mistakes you think I’ve made or investigating object-level disagreements (although I could be wrong). I don’t recall criticizing you for not having an MD or something, or publicly speculating that you have never thought about [important meta-level epistemological consideration]. 

I understand that the fact that I did not take biology or neuroscience classes in college is evidence that I would not have a good understanding of sleep research, but I think it is perhaps important to keep in mind that argument screens off authority here, and it sounds plausible that, a lot of the time, domain experts in the area would acquire knowledge in it the same way I do (by reading meta-analyses and systematic reviews, or textbooks based on those). They don't have some sort of magical essence that makes them more knowledgeable than everybody else could become. They do original research, but not in every sub-ar... (read more)

2ChristianKl
Domain experts usually do learn not only from public information. When it comes to the question of whether or not to believe a scientific paper domain experts usually learn the skills from talking with their collegues.  I had one bioinformatics professor who made a point of saying something in every lecture about how we shouldn't just believe the literature and how there's a lot of mistaken papers out there. One of your disagreements with guzey is whether "X is true because a meta-analysis says so" is a reasonable argument.  Talking to people is a good way to understand how untrustworthy they are. 

Thanks for this post, I've curated it! Reason why include:

  • In general when we curate a post (e.g. Theses on Sleep) I like curating thoughtful critique of the original post.
  • While I got a useful new mental model from Theses on Sleep about how to think about sleep, this post has done the valuable work of bringing up a lot of the counterarguments. I didn't put in the time to do it myself, so seeing the strength of them and critiques of various specific lines of reasoning in the prior post gives me a far better sense of how strong the arguments were in that post
... (read more)

Conditional on this, cognition does not return to baseline levels after several months of sleep restriction: 78%

Does this mean you're conditionally claiming that several months of sleep restriction results in permanent damage to cognitive faculties, irreversible even after long-term restoration of normal sleep?

No. (Thanks for pointing out that that’s not clear). My model is that moderate sleep restriction harms cognition the following a day, regardless of how many nights of it you’ve had before; I’m much more uncertain about whether it causes permanent damage to cognition.

3DirectedEvolution
Thank you for clarifying!

Anecdotally, since reading Guzey's post a month ago, I cut down my sleep from ~7.25 hours (5 nights 7.5 hours + 1 night 6 hours) to around 6.33-6.5 hours (1 night 7.5 + 2-3 nights 6). I found that doing just 6 hours 4+ days in a row led to noticeable tiredness, although I never tried just pushing through and seeing if I can get used to it.

Regardless, with the current sleep load, I feel pretty good, and I plan to continue it. However, I have noticed some rare working memory slip-ups, maybe one per day or every other day, that I don't think were as common before I dropped the sleep, although this isn't severe enough to make me want to stop.

3Elizabeth
Any updates? how did this work out in the long term?

There is something about Natália's and Guzey's interaction through your posts and in the comment section that doesn't feel right to me. I haven't been able to pin down exactly why I get this feeling but my best guess is that it seems like you are missing your actual disagreement.

To me it seems like you are both agreeing on the existing evidence is week. You bring up evidence that point in different directions sure but neither of you seem to bring up strong evidence.

However, you seem to have quite different beliefs on the matter. Natália says:

Our priors abo

... (read more)

Another odd discrepancy.

The largest single study in the Lowe meta-analysis is Banks (2010), which had 159/1688 of the participants (9.4%). In table 2, Lowe reports that Banks measured 1 day of sleep restriction. But Banks actually did 5 days of 4hr sleep restriction/4hr time in bed, followed by 1 "recovery" day of 0, 2, 4, 6, 8, or 10 hours of time in bed. It's unclear why Lowe coded it as 1 day of cumulative sleep restriction rather than 5-6. In addition, I don't understand how they would have incorporated effect sizes into their meta-analytic procedure.

G... (read more)

2Natália
Banks (2010) conducted neurobehavioral assessments on every day of sleep restriction (including the first one). See Figure 2 in their paper.  [In case you're wondering how Lowe et al extracted data from Figure 2, apparently there's software for that. From their article: ]
2DirectedEvolution
I'm not sure why that's relevant to my question? They still did 5-6 days (depending on whether you count the recovery day, on which many SR subjects got even less sleep than before) of sleep restriction in total. Unless Lowe is only counting the results from the first day of sleep restriction, which is possible but seems unlikely, describing this as 1 day of cumulative sleep restriction duration seems inaccurate. However, I will withhold judgment until I hear back from Lowe.
3Natália
Huh. I had assumed they might have been only including the first day, but you're right that that's unlikely. Probably a typo.  That wouldn't change their overall meta-analytic effect sizes, though, since they aggregate across different durations of sleep restriction when calculating it. 
3DirectedEvolution
Dr. Lowe did get back to me briefly to say she's looking into it. I'll post an update when I hear back.
3DirectedEvolution
If it was indeed a typo (there is at least one more in the same table - Saleh (2003) is duplicated when there should be one entry for 2003 and one for Saleh 2011). On the other hand, it's possible it was miscoded or misinterpreted. I'd be surprised if that were the case, but it's hard to see, for example, how they came up with a number for "mean hours of sleep deprivation" given the 6 different cases for the recovery day sleep in the experimental group, or whether they code an effect size for the group that was sleep deprived for 5 days and then got 10 hours of sleep on the recovery day.

Thanks for the taking the time to look into my essay.

Mendonça's claim 1: Guzey’s /r/BipolarReddit evidence is misleading

Here's what Mendonça writes in her first point that so conclusively demonstrates that the point I make is "misleading":

[Guzey's] evidence from bipolar disorder patients is not representative of what you see in the general population: both long and short sleep duration are associated with depression

The paper Mendonça cites looks at long-term long sleep and long-term short sleep, with their association with depression. My claim and my evide... (read more)

The paper Mendonça cites looks at long-term long sleep and long-term short sleep, with their association with depression. My claim and my evidence (from bipolar people) are concerned with short-term long sleep and short-term short sleep. 

Your specific claim about depression was “depression triggers/amplifies oversleeping while oversleeping triggers/amplifies depression.” Nowhere in the section did you specify your claim was about short-term long sleep.

Your evidence, too, barely concerns short-term long sleep: depressive episodes last about five or six months on average, which is often not what people have in mind when they think about “short-term” oversleeping, and it's common for them to last a year or more. 

It is further puzzling that in her refutation of my argument she:

  1. completely ignores the relationship between sleep and mania (i.e. she ignores one half of my argument and only discusses the part about depression).
  2. completely ignores the fact that in ~50% of ALL people with depression (not just bipolar), short-term short sleep relieves depression.

I don't disagree with you on the claim that sleep restriction or deprivation often causes mania, and can adequately treat de... (read more)

In the interest of simplicity, I'm going to look at individual pieces of your and Natalia's counterarguments. I won't do it all at once, but I'll try to be thorough over time. I'll be separating my analyses into separate comments.

To start with, you say:

Here's what Mendonça writes in her first point that so conclusively demonstrates that the point I make is "misleading":

[Guzey's] evidence from bipolar disorder patients is not representative of what you see in the general population: both long and short sleep duration are associated with depression

The paper Mendonça cites looks at long-term long sleep and long-term short sleep, with their association with depression. My claim and my evidence (from bipolar people) are concerned with short-term long sleep and short-term short sleep. That is, Mendonça's paper is simply not overlapping with with what I write about or what the evidence she calls "misleading" is concerned about. 

To break this into parts:

My claim and my evidence (from bipolar people) are concerned with short-term long sleep and short-term short sleep.

There are several types of bipolar, including bipolar I and bipolar II.

  • Bipolar I Disorder is defined by manic episodes t
... (read more)
5DirectedEvolution
Sleep compression and restriction is also used as a treatment for insomnia. From the sleep foundation (not a great source, but I think this is an OK as a description of the technique): I think this highlights that we need to distinguish between "spending more time in bed" and "getting more sleep." More on this later.
4Natália
The difference between BP II and BP I doesn't matter in respect to this. You need to have depressive episodes at least 2 weeks in length to be diagnosed with BP II.  From the DSM-5, which you can find on libgen:
6DirectedEvolution
Thanks for pulling that up. This makes the point you and I are making even stronger in this area.

I think the value was in the interesting idea rather than being particularly rigorous

The effects of napping found by the meta-study you linked are more complex than your quote suggests:

The literature reports that the benefits of daytime napping may last 2.5 h [68], with conflicting results during the sleep inertia period, i.e., after awakening [55,69]. These findings were in accordance with results from our study, in which the positive effects of the nap were mainly 30–120 min following awakening. For the 30 min after napping, results were mitigated and variable, depending on sensitivity analyses, reflecting putative effects of sleep inert

... (read more)
1Natália
The effect size I quoted includes studies in which cognitive testing was done in the sleep inertia period (and, in fact, they account for about 30% of the weight), and the mean nap duration among the studies in that meta-analysis was 55.4 ± 29.4 min (quite a bit longer than 20-30 minutes), so I don't think that was selective reporting.
3DirectedEvolution
That's a fair objection. A second selective reporting complaint that I would make about how the researchers framed their findings is that after outlier exclusion or after analyzing only RCTs, their global effect size drops from 0.18 to 0.07-0.08, with a 95% CI on both sides of 0. However, when describing this result, the researchers consistently, in both the abstract and in the sensitivity analysis section, refer to this as "similar results," without qualification and without reprinting the effect size number. To be fair, the only RCT model still finds a moderate effect size 61-120 minutes after napping, which tracks with the common sense idea that a nap can temporarily boost cognitive function once the sleep inertial period is over. It's odd, though, that their global model finds that napping provides the biggest benefits to alertness, while their only RCT analysis finds the worst results for napping on alertness, with the central estimate showing a negative impact. Possibly, this can all be explained if the few RCTs they found focused on testing less than 30 minutes after the nap, so that the apparent effect of napping on performance would be diminished. I haven't looked at them, so I don't want to be too strident in this critique.
3Natália
What you mean is that it showed a difference between the napping and control groups at t1 that favored the control group. This seems to have been due to baseline differences between the groups, because when you look at how the mean alertness of each group changed in the RCT analysis, they both improved, with that of the nap group having improved a bit more:
3DirectedEvolution
That's a good catch. So it sounds like they're comparing the "effects of a nap" by comparing the control group performance at t1 with the nap group performance at t1, rather than the difference in the magnitude of improvement between the groups from time t0 to t1? Example: * Controls score 1 for alertness at time t0, and a 3 at time t1. * Nappers score a -1 for alertness at time t0, and a 2 at time t1. * The nappers improved by 3, while the controls only improved by 2. * Yet the controls scored higher at t1, because they started at higher baseline alertness. If this is what's going on, my credence in napping being effective is increased. The gains in figure S4 vs. S7 are easy to discern, just by eye.
4Natália
Yeah, that is what Figure 4 (which you attached in your comment) shows, and what the effect sizes reported in the abstract refer to. There does seem to be a trend towards nap groups across the studies having a bit lower baseline cognitive performance: Figure S6 directly compares the change in performance of the groups, though the effect sizes are harder to interpret in it.
2DirectedEvolution
Thanks for digging out the supplemental figures! Interesting...

I'm surprised that tinkering with sleep is even an area of interest in LW circles. It's a classic Chesterton's fence not to mess with. Short-term sleep deprivation is fine and happens all the time, chronic sleep deprivation is bad, don't do it, focus on some other ways to squeeze more time out of the day, if that's what one is after. Leave the sleep research to the professionals, it's not a low-hanging fruit.

(Unless you intentionally sleep deprive to trigger bipolar hypomania and are reckless enough to risk it turning into a full mania.)

Chesterton's fence means that you want to investigate strange-seeming barriers, rules, and norms before tearing them down permanently, not that you should leave them in place unquestioned :) And the claims you make here are precisely the questions we are trying to address.

Sleep research provides an interesting exercise in rationality, because it touches on so many pertinent challenges. There's instrumental reasoning with imperfect information, figuring out how to conceptualize the issue, mechanistic reasoning, critical interpretation of scientific literature -- it's all there. Personally, I think it's a great rationalist exercise.

6Shmi
Indeed. And sleep is one of those, erected by evolution, pervasive across species, but with no clear reasons. Playing with it without clear understanding of what is going on is bound to backfire. Yes, as a pure exercise in rationality it is interesting. It is also called "science of sleep" and best done professionally. As a way to save time or to do self-experimenting, it's definitely not the way to go.

Your advice here is too imprecise to be debatable. What's the line between normal sleep variation, individual sleep differences, and "playing with it?" If we don't feel tired (as in mania), are we "playing with sleep" if we simply stay awake, as we feel inclined to do, or are we "playing with sleep" if we indulge in staying up late?

When you say that it's "bound to backfire," can you turn this into a claim precise enough to make into a bet?

When you say it's "best done professionally," do you mean that we should leave it to professional sleep researchers to run such studies, or that we should simply get the FDA-recommended amount of sleep and not ask why?

If you can make your claims precise enough to match the level of care, thought, and rigor on display here and in guzey's original post, I'd love to engage further!

8Shmi
Can't promise the "level of care, thought, and rigor on display here and in guzey's original post" in a single comment. Seems like asking a lot. But, to answer some of your questions: * In mania you are not playing with it, it is playing with you, and you have little control of what is going on. It is also known to be unhealthy in various ways. * By "playing with sleep" I mean not going with the flow by staying up or staying awake, but methodical deliberate attempts to change one's sleeping patterns. * "Bound to backfire": it is well established that chronic sleep deprivation leads to various side effects that are discussed in the OP (and also argued against by Alexey). One pitfall is that it is not just the side effects, but the inability to tell that something goes wrong as a result of sleep deprivation, thus destroying an essential feedback mechanism. * "Best done professionally" means exactly that, "professional sleep researchers". Someone who is qualified and is not sleep deprived while doing the research on willing subjects, or on those who are stuck with unusual or poor sleep. "we should simply get the FDA-recommended amount of sleep and not ask why" seems silly enough that I have trouble believing you actually considered that as something I might have suggested. Not sure if this is up to your standards, but I guess I tried.
6DirectedEvolution
This definitely helps, so thank you. * By "playing with sleep" I mean not going with the flow by staying up or staying awake, but methodical deliberate attempts to change one's sleeping patterns. It surprises me that you'd find relatively impulsive and ill-considered modifications of one's sleep routine to be preferable to a carefully thought-through approach. Can you say a little more as to why you hold this opinion? * "Bound to backfire": it is well established that chronic sleep deprivation leads to various side effects that are discussed in the OP (and also argued against by Alexey). One pitfall is that it is not just the side effects, but the inability to tell that something goes wrong as a result of sleep deprivation, thus destroying an essential feedback mechanism. I think we all pretty much agree here that sleep deprivation comes with downsides. It's really the magnitude of those downsides, potential for upsides, the impact of those downsides on things like health and functional performance that are in question, and how the downsides weigh against the benefits of ~12% additional waking hours that's in question. While it's fine to point out that downsides exist, your tone seems to imply that you believe that the existence of downsides settles the matter, while it seems to me that the whole point of broaching the discussion in the first place is that Guzey and I (and others) think it's worth discussing. * "Best done professionally" means exactly that, "professional sleep researchers". Someone who is qualified and is not sleep deprived while doing the research on willing subjects, or on those who are stuck with unusual or poor sleep. "we should simply get the FDA-recommended amount of sleep and not ask why" seems silly enough that I have trouble believing you actually considered that as something I might have suggested. I don't think it's silly to have interpreted you that way. After all, nobody here has suggested actually running any sort of formal sl
2Shmi
From a bunch of downvotes I assume that people don't want to see the types of comments I have made about tinkering with sleep being unwise, I think I said enough, anyway. 
5DirectedEvolution
Up to you! I’ve been upvoting you, though. My main concern in this thread is that I don’t want to see an opportunity for constructive debate stifled by people’s feelings getting hurt, or by frustrating levels of imprecision. So I find it annoying that people are downvoting you when you’re making an effort to provide an alternative perspective. If you’d like to PM with other thoughts, please do.

I don't think your 'our priors about sleep research should be high' argument holds, for the simple reason that everyone knows, now, that you're supposed to be tired and grumpy when you get less than "your eight hours". There is almost certainly a placebo effect in play, which means that you may well be allowing the sleep research itself to affect your priors about the plausibility of the sleep research! 

This doesn't undercut the rest of the post, but I do think "Occasionally stubbing your toe is good for health and promotes more efficient toe healing" is unfair. Lots of people believe that acute stress in the form of exercise, infection, fasting, or extreme temperatures is good for them, and sleep deprivation seems a lot more like those than like stubbing a toe. Of course those beliefs could be wrong, or sleep could not be a member of the set, but if the hypothesis were incorrect I think it would be a normal kind of incorrect, not something as ridiculous as "breaking a bone is good for you".

8Matthew Barnett
Why?

I agree with Elizabeth that it intuitively feels like SD fits more with things like exercise and infection than things like toe-stubbing. I did not have a ready-made answer before you asked the important question "why?," and I think that means you should discount this somewhat as rationalization. With that caveat, here goes!

First, let's look at some of the underlying physiological impacts of these potentially "good stressors."

  • Exercise: tear and repair of muscle fibers results in altered morphology
  • Infection: repeated exposure results in death of dysfunctionally reactive immune cells
  • Fasting: hormone changes in response to change in metabolism (I'm handwaving to the max!)
  • Extreme temperatures: one impact is that elevated body temperature causes the release of heat shock proteins, which directly impact immune system function, such as by increasing MHC class I loading (this is an important role of fever in fighting off infections). It also affects the material properties of proteins, which could impact things like flexibility but also enzyme reaction rates (though not necessarily in a straightforward way, since the body attempts to create homeostasis and there are all kinds of feedback l
... (read more)
4Elizabeth
Broadly, they train your body's homeostasis mechanisms to handle a wider range of conditions, which lets it cope better with aging.  For the immune system in particular, it's pretty well demonstrated that childhood disease load is setting expectations for future disease load. Muscles are strengthened via tiny tears that heal. Tears are stress, too large tears are damage, but small tears are the only way to grow the muscle.  Fasting triggers some hormonal changes that maybe include some housecleaning and repair mechanisms, although I know less about this.
6Elizabeth
A better answer: in the course of a week, you will naturally spend time eating and not eating, moving and not moving, and sleeping and not sleeping. Calling something "fasting" (as opposed to "not eating right this second"), "excess exercise" (as opposed to "moving"), or "sleep deprivation" (as opposed to "being awake") is begging the question. It assumes you know the correct balance and are describing a departure from that balance. Meanwhile, stubbing a toe is always a worse state than not having stubbed a toe. The correct amount of toe-stubbing is zero, we can assume any deviation is bad (but then argue about what is slow-motion assault and what is deeply useful trigger point massage therapy). [The immune system is not a member of the "all things in balance" class. Having zero pathogens and being adapted to an environment with zero pathogens is obviously superior, but the trade-offs are made weird by being an adversarial game with lock-in] My model is that the finding balance between eating/not-eating, moving/not-moving, and sleeping/not-sleeping, is not always trivial. Your body sends imperfect signals, the right answer varies based on the environment, which changes a lot, and you're making trade-offs you might not endorse if you had more info. Mild-to-moderate departures from your current balance can be informative in helping you find the correct balance and learn the trade-offs you're making, even if the experiment ends up being net negative in the short term You might argue that Alexey specifically used the words "sleep deprivation". I think that's a poor phrasing resulting from a bad model being embedded in the language, but if he comes back and says "no, I definitely meant deprivation-as-in-bad-for-you", it won't change my belief in this model.