On a related note, one of the neater papers I've seen recently:
"Air Pollution and Adult Cognition: Evidence from Brain Training", La Nauze & Severnini 2021: using 4.6m scores from Lumosity brain-game players to measure PM2.5 cognitive effects: large but heterogeneous by task/age/experience---this may explain the inconsistencies of all the previous air-pollution/cognition studies.
I think once on LW I found checking the reaction time with some app as a proposed fast and uncomplicated benchmark to investigate cognition correlations with time of the day, sleep deprivation etc.
http://www.quantified-mind.com/ contains many cognition tests and games which can be done quickly.
In addition to variance of opponents' skill, there is also a substantial luck factor. Any search algorithm that can't search the entire tree has uncertainty about how good moves are, which effectively translates into luck.
I think there's a good chance your observations are still valid (also because you have intuition about how well you play on top of results), but it is a possible factor.
You can get up to around 1600 ELO pretty much just by not making stupid one move blunders. I am ranked around 1550 and obvious blunders that I should have caught are consistently the reason I lose games.
I don't think this is in conflict with what I said. Blundering is itself a matter of luck. You can have both players play sufficiently reckless that they risk 1-move blunders, but then one person gets lucky that they've never made one. I assume you're familiar with the situation where you make a move that's only not a blunder due to where is something you didn't think about while making the move.
Also, I don't know how literal you meant your post, but I don't think it's true that you can get there by 'just' avoiding one-move blunders. If your positional play is sufficiently worse than your opponent's, you should lose even if your opponent blunders a minor piece at some point and you don't. I think it's more like, most people around 1500 are somewhat even in positional skill, and you can separate yourself from them by avoiding blunders.
(Out of curiosity (or perhaps because I'll challenge you), what time format are you 1550 in?)
I have a pretty big n dataset for Anki flashcards (and associated performance) and chess games performance that I could try measuring whether there's a predictive effect for long-term memory.
Here is what I did (n=524):
This has an R^2 of 0.016, and the coefficient is ~5.5e-05 (though it is pretty significant). So a performance rating of 1000 higher than predicted only yields a boost of ~5% additional accuracy on anki. Since the adjusted performance rating has a standard deviation of 208 points, that means if you're having a "top cognition" day that's 2 std's above average, that's only 2% higher anki accuracy. Not a lot.
Of note: using a "locally smoothed" performance rating (where I smoothed the perf rating, then subtracted that from the perf rating to get a residual) yielded no significant correlation between anki accuracy and perf rating. Arguably this is a stronger bit of evidence - the above (naïvely) assumes that the perf rating goes up linearly with time, but this version is able to deal with plateaus and different slopes in increasing/decreasing rating.
I'm open for code/analysis review if anyone wants to double check my work.
I have the same experience with my level of play on chess.com - there are bad days when it feels like my hand is playing and some good days when I feel more like my head is playing. Not sure what external variable it is related to. In any case, you can play me if you befriend me there first ("Losna1", Losna - one, not Losna - capital i). I am 1400 on 10" time control, not much fun if you are more than 200 points far from that...
I also have crazy variation in my online chess performance. My rating varies by more than 400 points on some sites.
Maybe worth pointing out that small differences in speed have a cumulative effect over the course of a game. But beyond speed I just seem to be blind to many possibilities on the off days.
I vaguely planned to use online chess ratings to do a gwern style study on the effects of ginseng one day.
I swing between 1100 and 900 in chess ratings really obviously. Some days I'll win and win and win until I'm around 1100; other days I'll fall and fall until I dip into the 800s. A 200 ELO point difference is equivalent to winning the game 76% of the time, so this means that "best day me" would beat "worst day me" in 76% of head-to-head chess matches.
I too play chess often and have had similar thoughts. But I wonder whether day-to-day variation in winning is mostly random, and any perceived correlation with other intellectual tasks is erroneous or self-fulfilling. You could do an experiment for a few weeks where you rate your ability on both each day - sometimes playing chess after, sometimes before other tasks - and measure the correlation. Of course it can’t be blinded so it may be self-fulfilling.
From another experiment I did on ways to improve sleep, I’ve found that self-perceived correlation between various ways to improve sleep (eg eating a banana before bed) and sleeping well turned out to be entirely illusory.
Also cf many years ago research showed that ‘form’ in sports (individual sportspeople going through extended periods of good or bad play) was almost entirely illusory and just random variation. Concurred by work I’ve been involved with on poker software - two identical computer programs playing each other can look like one is much better than the other for very long runs of hands.
I have noticed certain variations in my play depending on my emotional state. I am a casual player (1000 ELO on Lichess) but try to get in at least one game per day. It was a big deal for me to hit the 1000 mark, so I am very conscious of how much above it / below it I am at any time. When I am anxious or thinking about a past fight with my SO, or am stressed out about a current issue, I start losing games like crazy (And my ELO drops significantly). Once everything is fine and dandy, it starts climbing up again. I think chess is very much a game of extreme concentration and any intruding thoughts (which depending on the problem will penetrate with more / less ease) seep through and break that focus.
The AI on lichess sometimes makes clearly worse moves than the ones I made. There is also room for much more in depth analysis like drop, fork, defensiveness, etc. I switched from chess to Anki and Amphetype because, while not nearly as fun, they also taught me a skill. I will get back to chess when I find an affordable automatic board. Cognitive tracking is often discussed on reddit or the QS forums.
Exist a few papers on the subject of chess as a test of cognition.
"Using within-player comparisons, we find a statistically and economically significant decrease in performance when competing online compared to competing offline." Steffen Künn 2020
DB RCT Coffee causes "more reflective decision making processes. When not under time pressure," Franke 2017
I play a lot of chess and definitely notice a lot of day-to-day variation in how well I play. For me, the main factor seems to be if I had to wake up early or not. On days where I woke up naturally, I can pretty easily calculate out five or so moves, but on days where I had to wake up before the sun came up, it's much harder, and I can only clearly see two or three moves ahead. Caffeine seems not to affect this much.
I just want to say I came to the same conclusion as you (plaing 3-min games at chess.com several times a day, being in the 1250-1300 range) - quite a striking difference between good days and bad days, and only partly related to what I would expect (because of sleep deprivation etc.)
Also clearly better results in the evening than in the morning.
I have been thinking many times about statistically processing the results, but don't really have the time :)
It seems like it would be really interesting to study this quantitatively, recording data collected over time. If you (or anyone else) would be interested, I know a bunch of psychological research methods that could help.
How many games you play daily, and what time do you play them? If you have data from different times of the day, do you noticed any patterns? I once had a feeling that with some tasks at work I can "unblock" and really get the job done only after 5 or 6PM. But never put any serious effort into observing it for a longer period of time.
Lately I've been playing chess. I'm not very good at it. I started because our four-year-old got into it, and I preferred not to lose to her before she had earned it. I play online on chess.com, which has instantaneous matchups with strangers at approximately your level; I mostly play 5-minute timed games, where each player has a 5-minute clock for all of their moves and loses if they run out of time.
Doing this every day for a couple of months made some interesting patterns obvious. Firstly, I have good days and bad days, rather than just winning or losing mostly at random at my competence level.
Bad days are predicted by some of the things you'd expect, like poor sleep, high stress, or emotional distraction, but sometimes I have a bad-day-as-measured-by-chess when I wouldn't have predicted it, or (more rarely) a strikingly good day when I thought I was likely sleep deprived.
Secondly, playing chess on a good day versus on a bad day allows a lot of access to the feeling of being slightly smarter or slightly stupider than usual. In general, I think my ability to do high-level cognition varies a lot day to day and even within a day, but it's often really hard to tell if I'm blocked on a problem because I'm a little low on ability to think about it; usually, a problem will be different from other problems in a bunch of ways, and it's hard to tell how much of the difficulty is that I'm fatigued/distracted/whatever.
With chess, you're basically solving the same category of problem every time, with instantaneous feedback about whether your solution was any good, so I get unusually good felt-sense of where I would normally have seen something, but this time missed it, or alternatively where options and counteroptions are coming to me unusually quickly and have unusual depth and clarity.
Thirdly, having a good day at chess seems pretty strongly associated with having a good day at high-cognition-requiring tasks in general. In fact, it seems like a better predictor of this than asking myself 'do I expect to be good at cognition today?'.
On days when I do well at chess, I'm also likely to succeed at reading a complicated paper I previously bounced off because it was too hard, or at writing an article that has been blocked on loading all of it into working memory, or at having productive new thoughts if I dwell on an old problem for thirty minutes. I am not more likely to succeed at doing my taxes or other tasks that are motivation- rather than cognition- bottlenecked.
This could, obviously, be a self-fulfilling prophecy, or a placebo effect, but at least for me it makes playing chess a moderately valuable thing to do; in about ten minutes I can get a readout on how well my brain is working, and figure out from that which of my priorities for the week make sense to work on.
I am wildly uncertain how much this would generalize to other people, but it seems like a cheap experiment. Some suggestions, if you want to try it: