How to test your mental performance at the moment?

[-]CronoDAS16y70

Unless I'm actually nodding off, I can't tell the difference, at all. To paraphrase Paul Graham, one of the symptoms of bad judgment is believing that you have good judgment.

1loqi16y

Do you notice any subjective difference from caffeine? Sometimes just the act of trying to remember what some distinct mental state "is like" can give you a handle on your current state. Along the same lines, another fuzzy measure I've found is the ease with which I can recall what being really effective and flow-ish feels like. The further I am from that state, the more difficult it is to relate to or hold in mind the memory of it.

0CronoDAS16y

Caffeine makes me jittery. I generally avoid drinking it. (I've also never consumed any substantial amount of alcohol, either.)

[-]JustinShovelain16y70

Some things I use to test mental ability as well as train it are: BrainWorkshop (A free dualNback program), Cognitivefun.net (A site with assorted tests and profiles including everything from reaction time, to subitizing, to visual backward digit span), Posit Science's jewel diver demo (a multi-object tracking test), and Lumosity.com (brainshift, memory matrix, speed match, top chimp. All of these tests can be found for free on the internet).

Subjectively the regular use of these tests has increased my metacognitive and self monitoring ability. Anyone have ... (read more)

1gwern16y

Second Brain Workshop; the dual n-back task is one of the few things which has any research suggesting it generalizes/transfers, and so is really interesting compared to random mental trivia tests like most of the stuff on Luminosity.

[-]Eliezer Yudkowsky16y50

I second this question and a good solution might be highly important to my productivity.

[-]Steve_Rayhawk16y40

My brother suggests continuous performance tasks.

This is a review of four CPT packages from 2000, and this is an review of 3 CPT packages from 1995 from a user's perspective. These are some slides from 2003 about computerized CPTs and related test instruments.

Continuous performance tasks test sustained attention and need to be "purposefully boring", typically 10-25 minutes, which can be too long for daily use.

The Psychology Experiment Building Language comes with a battery of tests "designed to closely resemble many commonly-used psychologic... (read more)

4Steve_Rayhawk16y

Any psychological test of response time that uses only consumer PC hardware will have difficulty with timing resolution and calibration. Platforms like operating systems, libraries, and virtual machines, and competition by other processes for processor time, can add difficulty. Some operating systems have a scheduler resolution like 10ms on an unloaded system, and they put input events into a queue without recording when the events happened. Even if an operating system does not do this, a cross-platform media I/O platform like Flash (used by Cognitive Fun, with an 18ms frame delay) or SDL (used by PEBL) might. Computer displays refresh around every 18ms, and the top part of a display refreshes earlier than the bottom part. The display technology may add delays which depend on the starting and ending light levels. The synchronization of sound stimuli with other timings depends on the sound hardware, drivers, and libraries. Input device hardware and communication protocols can cause other delays; for example, USB mice and keyboards are by default polled by the OS only every 8ms (though this can be changed in software). Free libraries which are more careful about timing resolution are psychopy (which uses pygame for sound and pyglet for everything else) and pyepl. More information on response timing resolution is in the psychopy documentation or the sections on timing and CRT stimulus presentation in the documentation for the commercial neuroscience experiment software Presentation. If the experiment software only knows the time of an event up to the start and end of an interval during which the software didn't have CPU time or input poll results, then normal statistical methods would need that interval to be converted to a point estimate of the event time. This can cause bias. Intervals should be interpreted as a source of relative likelihoods, such as can be used in a maximum-likelihood estimator of the parameters of a distribution of possible values, or a Bayesian

2Steve_Rayhawk16y

(My brother works partly for the company that sells equipment, training, and interpretation services for the Test of Variables of Attention, which may be the continuous performance task implemented with the most attention to timing resolution. For frequent measurement of mental performance, the TOVA itself might not be worth the price (including $15 per test) outside of clinical ADHD assessment or personalized dose-response measurement, unless maybe you get a discount for non-commercial use, but he says "all the TOVA folks are very open-source and science/philosophy friendly and will help you out" with advice on using a computer to test mental performance (even if you're trying to build a DIY free competitor). He suggests dropping in on Steve Hughes at one of their training workshops. The next one is in Los Angeles on December 4.)

[-]Vladimir_Golovin16y40

The most reliable test I found, and by accident, was fighting bots on a certain Quake 3 map

For me, it's playing a TF2 Scout on a well-populated server running a Scout-friendly map. A kill/death ratio of 3:1 or greater means that I'm sharp, 2:1 is okay, and 1:1 or lower means that I'm sluggish.

However, in my case, using a game (especially a twitch-aim FPS) as a performance test defeats the purpose of testing because it primes me away from the task I've planned for the day and considerably shortens my attention span, so I never play anything before work.

... (read more)

[-]akshatrathi16y40

During a sleep experiment, I used to record my mental performance by a simple arithmetic game. Start with a 3 digit number, subtract 9, then 8, then 7...so on. Time yourself in the task. If the result is ±3 seconds to my average score, means I am quite active.

0righteousreason16y

That reminds me of "counting doubles" from Ender's Game: 2, 4, 8, 16 ... etc until you lose track.

1Vladimir_Nesov16y

The problem is that with a systematic enough approach, verifying that you've memorized the working data at each step until you do, it's possible to keep on going to 2^50 and beyond, losing all day on the activity :-)

[-]brilee16y30

(Joined just to comment!, been stalking for a while)

I find that sightreading music actually works well. If I'm wide awake/alert, I can sightread bach fugues with 4 voices, and I can mentally actively keep track of them simultaneously. On the low end of the scale, I find that I can only concentrate on one voice at a time and my sightreading performance drops significantly. I also play blitz go, but the problem is that you don't realize then your moves are bad, and it's only when you review your games the next day it's obvious that you've played bad moves.

A... (read more)

[-]pdf23ds16y30

Seth Roberts used a balance test to measure cognitive functioning to help determine whether various cognition-enhancing supplements were working. He would stand on a small platform and measure the time before he fell off, repeated like 5 times. One of his readers came up with another measure--arithmetic drills. A sheet full of single-digit addition or multiplication problems. The measure, of course, is time to completion.

I have a version of Tetris I think might be useful towards this end. In my version of the game, the speed increases when you clear lines ... (read more)

[-]Jordan16y30

I've paid close attention to my productivity and general cognitive agility for the last few years. I've found that it gets easier and easier to track, to the point that I can now identify how sharp I am on a given day without actually testing my mettle.

My sole motivation for tracking was to identify correlations between diet and cognition (which has been a pretty successful program). I imagine that once you start using predictions to inform your work schedule you have to be very careful you aren't tricking yourself to avoid work.

6jimmy16y

Would you mind sharing the results of your diet and cognition experiments?

7Jordan16y

Sure. I should mention before hand that I wouldn't expect my results to generalize to anyone else. I started tracking my diet when I began having health problems and distinct cognitive impairment. I imagine the majority of the benefit I've achieved from diet has been due to correcting things wrong rather than improving things right. That said, here's a brief overview: Foods that rapidly bring on brain fog, degree dependent on quantity: Raw carrots, raw bell peppers, peanut butter, walnuts, pure sugar (even natural sources, such as sugarcane, but tentatively not honey), bananas, raw apples, enriched grains (tentatively, I've only tried this experiment half a dozen times), tofu. Whether the food is cooked or not seems to matter, but I've only experimented with this for a handful of foods. Among the foods I do eat I prefer those that provide an even level of energy over time, to avoid crashes: lean protein, oats, olive oil, non-starchy vegetables; however, there are foods/drinks that provide positive cognitive benefit in the short run, such as: some alcoholic drinks, starchy vegetables (yams, sweet potatoes), coffee, mint tea. For the sake of simplicity in analysis I almost always restrict my meals to 4-5 ingredients, and almost never eat out.

2denisbider16y

My own observations about myself have been much more accidental, but I can add the following to your list: * Onions and mushrooms. Don't eat any of that in quantity if you want to stay alert. Large quantities of either make me want to sleep. * Lactose. I realized belatedly that I'm apparently lactose intolerant, like most people (depending on origin) over the age of 4. To avoid crashing, make your breakfast cereal with lactose-free milk.

0Jordan16y

I'm lactose intolerant as well. I refused to believe it most of my life. Ice cream was too delicious and cold cereal too convenient.

0jimmy16y

It still might generalize a bit if others have the same things wrong. I'd be interested to hear what sort of back story you have to make this a bigger problem for you. I'll try to pay attention when eating those sorts of foods and see if I notice anything different.

3Jordan16y

True, I'm sure some others could gain something useful, but a health forum would probably be a better place for all the details. I think there's a small LessWrong walk away message though, so I'll elaborate a bit here. I began having various symptoms about 5 years ago. The main things were heart issues (tachycardia, brachycardia, arrhythmias), digestive issues, low body temperature, and cognitive issues (confusion, reduced reflexes, poor multitasking). I was a typical junk food-eating,over caffeinated college student at the onset. After about a year of worsening symptoms I sought out medical help. Multiple doctors had nothing to say. The last doc I saw diagnosed me with hypothermia. I slapped my face and decided I could do better on my own, and thus started my food experiments. It took a few years and was more difficult than I would have thought (being skinny I never realized how dependent I was on certain foods) but I finally converged on a diet that has largely eliminated all my symptoms. Treating food like an experiment, rather than a source of pleasure or a chore, has been a great way to exercise rationality and will power. Nutrition is a complicated subject, confounded by conflicting information and individual differences. In addition there are numerous motivations for rationalizations (social pressure, convenience, cravings), and often little confidence about rational conclusions. Combine all that with a need to make level headed hypotheses and the ability to carry out the month long experiments and you have yourself a tricky exercise.

0[anonymous]16y

Would you mind trying Isomaltulose

[-]self-actualizing16y30

I have a 'calibration' set of puzzles that I do each morning. Put the ol' brain through its paces, if you will. Kakuro, Suduko, Cryptoquote, Jumble. I used to play daily Set. (www.setgame.com) It all boils down to pattern recognition, but I do feel better knowing that that bit of memory is still intact and relatively accessible. ;)

2saliency16y

I think the key is to have a third party test you. Ideally this third party has created a profile of you. The above is better then selecting test yourself. I think you could do better though. My solution would be to give an interactive test that gives you a harder or easier questions based off of how you answer the questions. I would incorporate this with a profile so that when you come back the next day it knows what level of questions to give you. Ideally you could test several types of IQ with few questions. The tests-questions-games like the above posts would be designed to be fun. At the end the test would tell you how you preformed compared to other times.

-1[anonymous]16y

wow negative karma.

3RobinZ16y

-1s happen - note it, try to figure out why, but it's not really a "wow".

[-]Vladimir_Nesov16y20

The problem is - other than observing deterioration of results, I have no idea if I'm in such a state or not. I cannot be sure if it's also true for others...

It doesn't need to get to results, the quality of the process can also be observed, so that lack of productivity is catched early:

Being sleepy
Not being able to follow a math book fast enough
Not being able to reliably set up a problem statement in imagination
Not being able to find a right word in conversation/writing

Performing some activity that isn't in itself useful seems like a waste of ti... (read more)

[-]denisbider16y20

I have found that my ability to solve complex problems is to some extent separate from my willpower to solve those problems.

One way I've measured my ability has been by solving Sudoku puzzles on my mobile phone. I try to solve them without "pencilmarks", so that I put down a number only when I'm sure where it belongs, in an attempt to hone my working memory and focus.

I've found that, when my brain is not working that well, I keep finding that I've just retraced the same logical steps from just a few moments ago, because I've forgotten the conclus... (read more)

[-]Phoenixism16y20

Reminds me of an experiment I read about in which subjects were given a pencil and a piece of paper and asked to strike the paper with the lead as much as they possibly could in a set amount of time. I believe this gave the experimenters a baseline upon which they could judge the participants' ongoing performance. If I remember correctly, they discovered that on days when the subjects' pencil strikes fell well below their average, they were more prone to accidents, injury, other physical mishaps.

Seems to me the exercise the test subjects participated i... (read more)

[-]James_Miller16y20

There is a game on Lumosity.com called Chalkboard Challenge. It gives you two expressions and you have to quickly judge which is larger. The better you do, the more time you are given to play. There are often tricks you can use so you don't have to exactly compute the value of each expression. I find the game tests my brain more than anything else I do. My results are strongly correlated with my brain "strength" at a given moment.

You have to pay for a membership to play.

[-]Mycroft6553616y20

I play a quick game of minesweeper on my phone. If I get a decent hard map solved in under 1 min I'm sharp. An easy map under 15 seconds. If i lose, I try and figure out if it was random or poor judgment. It's not as good as some other tests mentioned, but it's fast and mobile.

[-]Richard_Kennaway16y20

How do you test yourself?

By paying attention, not only to what I am doing, but to how well I am doing it.

0Zvi16y

This works, but it's often too late by the time that it does...

0Jonathan_Graehl16y

That sounds tiring :)

2Richard_Kennaway16y

Not as tiring as pressing on without noticing you aren't getting anything done.

1Jonathan_Graehl15y

I agree more with this comment more now than I did last year.

[-]Danneau16y10

Sudoku.

[-]fburnaby16y10

Sorry for the non-answer here, but I take a different approach: I work when I feel like it.

I'll try getting stared on the actual work, whether I want to or not (you have to overcome that initial mental inertia). Then once I'm about half an hour in to the work, if I find myself watching the clock or thinking about how I should go check my RSS feeds, I'll stop and switch to my list of more mundane tasks.

Just summarize the first half-hour of your work, and if it's crap, move on and come back to it later. If it's crap, you broke even, and if it's not you've saved yourself a half-hour wasted on Quake.

[-]Stuart_Armstrong16y10

Spelling. If I write something while tired, the spelling will always be mangled.

[-]Jonii16y10

This is offtopic, but anyway, since there seem to be many here that play online go, how about creating KGS room for lesswrongers? I'd be interested in playing with you guys. (This seemed partially related as so many mentioned "go" in some way in their responses. Dunno where else to throw a suggestion like this)

edit: Started a new thread here. If you're interested in playing with lesswrongers, reply there!

1Kaj_Sotala16y

If you don't get any responses here, I'd suggest trying again in an Open Thread.

0Blueberry16y

Or an IGS channel? I almost always play on IGS (http://www.pandanet.co.jp/English/). It has a much better interface and higher signal-to-noise ratio.

0akshatrathi16y

I'd be interested in such a room.

2Jonii16y

I'll create one, should be either under "new rooms" or "social", named "LessWrong". To help those interested, KGS is an acronym for Kiseido Go Server, url is http://gokgs.com, where you can either download a java app or use the one with your browser. Registering is not needed. So, let's hope something good comes out of this.

0akshatrathi16y

I've joined KGS. Look forward to playing GO!

[-]SarahNibs16y10

The most I have ever systematically done is during a weekend-long puzzle competition, with 12 people on my team, we decided that any time someone spent 30 minutes on a problem without making progress they should move to a different puzzle, and if they kept doing that, take their sleep. Dunno how well that translates to a domain where the problems aren't of known, order-of-magnitude-equal difficulty.

[-]steven046116y10

I've been using http://speedtest.10-fast-fingers.com/ but with no good evidence that it actually works.

[-]jimmy16y10

Are you saying that you cant tell the difference between working normally and working a significantly below normally, or that even when you're seriously impaired you feel fine (eg after being awake for 30 hours)?

I have been trying to pay attention to my mental ability since I started taking piracetam (which does produce a noticeable improvement), and I'm usually pretty good at determining whether I'm working better or worse than average. I think I have better resolution there due to practice.

However, sometimes I make much less progress than expected (ie ne... (read more)

3taw16y

I cannot, or at least until it gets to really extreme levels, and not reliably in any case. My sleepiness is not terribly correlated with my levels sleep deprivation - this could be taken to ridiculous levels back when I tried Provigil, as I didn't feel sleepy or tired or impaired in any way, after weeks of sleeping just a couple of hours a day, even though my performance suffered massively (as verified retrospectively by svn logs and such), and got back to high levels as soon as I slept it over. Normally it's not as extreme, but significant drop in performance happens many hours before I feel sleepy or tired, and can usually be fixed by a short 1-2h nap. It's even worse, as I don't know how sleepy I am - often I know I need some sleep, but my track record of predicting if it will be 1-2h nap or a full 8h sleep is hardly better than random. People with more regular 24h activity cycles probably don't have this problem as much.

[-]Morendil16y10

Hmm, I have noticed myself rationalizing "I'm going to play just one game of blitz Go" as a test of my mental performance. I'm pretty sure it's a rationalization, and therefore works the other way round.

I would tend to be suspicious of the above tests on that basis, though I can hardly assume your mind's workings are in any way similar to mine. ;)

It would seem simpler to assume that your causal model (tiredness etc. -> loss of performance) is in general correct, and to organize your work accordingly.

Some time management approaches (e.g. GTD) e... (read more)

[-]patrissimo16y00

Great question. Seth Roberts has studied this as part of self-experimentation. One of the major issues is finding tasks that don't have too large a learning effect, which somewhat confounds the data.

[-]aausch16y00

I've been relying on various incarnations of TextTwist, and 9x9 GO playing AIs. Nowadays I run little apps on my phone for just this purpose.

[-]jasonw16y00

As far as I can tell, nobody's suggested taking a properly scientific approach here.

Each time you want to measure, pick a small set of different tests to do and record the results. You should get enough data to do some decent stats quite quickly. You can discard any that correlate well with shorter tests.

You should include in the tests subjective self-assessment (maybe you're better at guessing than you think) and real-world task performance (even if it's just "tried to do something and gave up", "succeeded in complex task" etc)

(PS, I'm not a psychologist! No doubt there's all kinds of subtle tricks I'm missing here, but I think the principle is sound)

[-][anonymous]16y00

I try to avoid the temptation of IQ tests on the internet since they make me feel cocky for the rest of the day. Anyway:

www.iqtest.dk

I feel much more awake after going through it. The only question I had trouble with was the last one. I'm sure I have the right answer, but it feels like there's more to the question that I'm missing. So here's what I got out of it (spoilers): every shape is a shape-shifting entity moving one square to the right each turn.

If someone could fill me in on the rest, that'd be great. This has been killing me for about an hour.

*E: Moved this to the open thread.

5gwern16y

IQ tests as regular measurement tools are terrible ideas. You'll quickly burn through the doable ones available online, and you will have destroyed their reliability for yourself by test-retest and other learning effects. Almost any other test would be superior - play Grid Wars, for example.

0[anonymous]16y

I only went through this test once; though, I did reload and skip straight to question 39 a couple times. I tried it out since it was in the OP and I had some time to kill. Upvoted for recommending Grid Wars.

0gwern16y

Grid Wars is probably even better than Quake. If you're not alert, you'll know it in a few seconds rather than minutes, and there seems to be surprisingly little learning - my own high scores seem to be quite randomly distributed in time.

[-]Robin16y00

This reminds me of a story I was told about somebody who got so drunk that he forgot he was drunk and drove a car...

Though testing your mental capabilities is useful, there are some problems of trying to access your own mental state. First, if you believe your mental state varies throughout the day, then shouldn't your ability to access it also vary?

I'd say the tests provided by others are decent, but in many cases impractical or of limited use. Say you have to make an important boolean decision. You don't know how sharp your brain is, but you do know t... (read more)

[-]Yorick_Newsome16y00

I play a a few 3 minute blitz chess games at FICS. That way my results are quantitative, as I can see my rating going up or down. It's also possible to play a single 3 minute blitz game and estimate how well I seem to be calculating variations and seeing simple tactics. Not the most time-efficient method, I suppose.

The main indicator of my mental state is when there are many candidate moves; if I'm tired or mentally sluggish, I will spend up to 15 precious seconds finding a strong move. When I am in good shape and am in the groove, I normally find a strong continuation in about 5 seconds.

[-]jimmy16y00

Seth Roberts worked on the same problem when trying to determine if omega 3 helped his brain performance.

http://www.blog.sethroberts.net/category/nutrition/omega-3/omega-3-directory/

[-][anonymous]16y00

del

[-]Jonii16y00

http://www.iqout.com/index.php?lang=EN

This IQ-test could work. The questions are generated on-the-fly to match your level, and there is really little time to give the answer, so alertness is needed too.

0orangecat16y

That was rather interesting. I got a 137 but beyond the first few questions I wasn't sure of any of them. Usually the best I could do was identify a possible sub-pattern, narrow down the options based on that, and make an educated guess. I think I did better on the ones with varying numbers of dots and lines compared to the ones with just the shapes moving and morphing. Presumably if somebody took that test repeatedly (or possibly once if they're smarter than me), they'd figure out the class of algorithms being used and it would lose most of its value for determining immediate mental performance.

1Jonii16y

Actually, nope, I think that's exactly why it would make such an excellent benchmark for mental performance. The algorithms are surprisingly simple and after learning those, it's all about applying that limited library to the problem. Fast. If your performance is above average, you can do this a lot easier and faster and with less errors compared to a situation where you're not doing as well.

[-][anonymous]16y00

I find anything that's cognitively taxing makes a good test. By reflecting on the process of engaging with the subject, noticing what sorts of difficulties I have, how much more difficult than normal it seems to be, it's pretty obvious how I am doing relative to baseline. For example, opening a math book that I'm somewhat familiar with but never made it all the way through is a good test of my cognitive abilities. It's easy to look at a theorem or proof in a subject you're somewhat familiar with and know how difficult it would ordinarily be for you to unde... (read more)

[-]Bo10201016y00

I have a game on my phone from Lumosity.com called SpeedBrain. If I'm well rested or caffeinated I seem to perform better.

It's a nice little game, but I don't subscribe to any of their claims about how awesome their games are.

[-]lavalamp16y00

I have attempted to use freerice.com for this purpose (I was interested in quantifying just exactly how stupid I was, in real time, while I adapted to polyphasic sleeping. The answer was pretty damn stupid, but I didn't really need a test to tell me that). I can't say the results track particularly well with how I'm feeling subjectively, but I can't really say which of those is in error. At any rate, this probably doesn't measure very many components of intelligence. I didn't keep up with it because it is slow and it's hard to know when to stop and record your level.

Go problems would be more of a laziness test for me. The ones that are easy enough to not require work I probably know just as well in my sleep.

0Sebastian_Hagen16y

Could you give a short report on how PS worked out for you in the end? Saving on downtime seems like a major potential win to me, but probably isn't worth it if you have to trade any measurable amount of intelligence for it. I'd be interested in more empirical data regarding the advantages and disadvantages of PS over a monophasic sleep shedule.

1lavalamp16y

Intelligence returns to normal if not better (my subjective opinion) once completely adapted and no longer sleep deprived. That takes around a month. I'm still experimenting with different schedules; I may post something here on it eventually in the anti-akrasia category, as the posts here were one of the things that motivated me to try it in the first place.

1Vladimir_Nesov16y

I wouldn't believe my own subjective estimate on such things, only something objectively verifiable counts. It's too intangible, and a month is too long to accurately remember your own experience, to boot through a lens of what is possibly a different mental condition.

0lavalamp16y

I got very familiar with what it feels like to be at a particular level of of sleep deprivation in the course of adaptation, so I think my subjective opinion there is probably not so far off, but I agree I can't really expect anyone else to have much faith in it. I've attended one go tournament since I started, and I achieved a result better than my last few tournaments. OK, and I just beat my record at freerice.com. Other than that it will be difficult to come up with good data, as I've never taken an IQ test in my life. I should have taken an online one at least before I started, but I didn't think of it until it was too late, and I don't really trust the online ones anyway

[-]maahbaloch16y00

I think either of three methods listed by you do not really present the exact quantification of your mental performance. Your performance on these tests will most probably depend on your interest in that particular kind of test. Therefore any results that you may get from these tests will have that subjectivity in them brought to the entire process by your level of interest for that particular kind of test.

0taw16y

One big advantage of the three tests I described is that I usually get very good binary outcome - yes or no. It's not so with most other computer games, or other tests I tried. And they're all quite pleasant activities - even if I wouldn't necessarily want to spend an entire evening playing Quake 3 or doing go problems, I'm usually quite happy to spend 10 minutes on them.

LESSWRONG
LW

LESSWRONG
LW

24

How to test your mental performance at the moment?

24

24