We should feel good about the fact that some biases of different research designs will cancel each other out, while bad about our inability to weight each study optimally.

Experts vs. parents

by PhilGoetz 1 min read29th Sep 201023 comments


I'm reviewing the literature on the link between food dyes and hyperactivity.  Studies evaluate hyperactivity largely by observations made by teachers, psychologists, and/or parents.  Observations by trained professionals using defined scales are sometimes considered superior to observations by parents.  However, a meta-analysis of 15 studies (Schab+Trinh 2004, "Do artificial food colors promote hyperactivity in children with hyperactive syndromes?", Developmental and Behavioral Pediatrics 25(6): 423-434), found that:

While health professionals' ratings (ES = 0.107) and teachers' ratings (ES = 0.0810) are not statistically significant, parents' ratings are (ES = 0.441).

("Effect strength" = standard mean difference = average of (active - placebo) / standard deviation (pooled active and placebo).  Thanks to Unnamed for reminding me.)

This isn't saying that parents reported more hyperactivity than professionals.  It's saying that, across 15 double-blind placebo experiments, the behavior observed by parents had a strong correlation with whether the child received the test substance or the placebo, over four times as strong as that measured by professionals.  Conclusion:  Observation by parents is much more reliable than observation by trained professionals.

Schab & Trinh offered several reasons why this might be:  Administration of test substance might be timed so behavior changes occur primarily at home; parents observe insomnia while teachers observe attention; parents may detect behaviors that are not listed in the DSM for ADHD; and one more - parents may be "particularly attuned to the idiosyncrasies of their own children".  Gee, do you think?

Every parent is an expert on their child's behavior.  Just not an accredited expert.

Disclaimer: At least one study has found the opposite result (Schachter et al. 2001, "How efficacious and safe is short-acting methylphenidate for the treatment of attention-deficit disorder in children and adolescents?", Can. Med. Assoc. J. 2001, 165:1475-1488).  I haven't read it and don't know how strong the effect was.