If they're rushing then they can enter $5 as specifically instructed by the question. They can also check that if fits as a correct solution, which it does. The fact that so many people aren't doing that is interesting.
The report also details a study that tests instructions like "Be careful! Many people miss the following problem because they do not take the time to check their answer." Along with a big yellow warning icon. Does the warning work? Kindof! mTurk response times go from 22 seconds to 33 seconds. eLab responses go from 21 to 37 seconds (almost doubling). Did scores improve with greater attention paid? Not really! eLab correct answers went from 43% to 52%, a rise of only 9%. mTurk correct answers dropped from 34% to 25%.
I really don't think that they're wrong because they're rushing.
'A bat and a ball cost $110 in total.
The bat costs $100 more than the ball.
How much does the ball cost?
The answer is $5.
Please enter the number 5 in the blank below.
$_____'
That question is from one of 59 new studies (N = 72,310) which focus primarily on the “bat and ball problem," found in The formation and revision of intuitions - ScienceDirect by Andrew Meyer and Shane Frederic.
23% of respondents failed to give the correct answer.
I don't have anything insightful to say but wanted to share because it seems worth knowing.
Additionally, I'm not exactly sure how (and how much) I should update about this.
(Edit: I initially posted the wrong question)