LESSWRONG
LW

Personal Blog

9

Strong men are socialist - how to use a study's own data to disprove it

by Jacob Falkovich
31st May 2017
1 min read
5

9

This is a linkpost for https://putanumonit.com/2017/05/27/strong-men-are-socialist-reports-a-study-that-previously-reported-the-opposite/
Personal Blog

9

Strong men are socialist - how to use a study's own data to disprove it
5drethelin
2[anonymous]
0Pfft
0btrettel
0Pfft
New Comment
5 comments, sorted by
top scoring
Click to highlight new comments since: Today at 4:59 PM
[-]drethelin8y50

Sarcastic and rambling and not in a fun way to read. Either get to the point faster or make your asides less about mocking your target and more interesting.

Reply
[-][anonymous]8y20

Enjoyable read.

Reply
[-]Pfft8y00

This is amazingly great (I laughed out loud at the "Biceps-controlled socialism" graph), but I feel it only works because the original study authors made the rookie mistake of publishing their data set. The only time I have wanted to try something similar (for the brain mosaic paper), I hoped it would be possible to extract the data from the diagram, but no, the jpg in the pdf is sufficiently low-resolution that it doesn't work.

Reply
[-]btrettel8y00

I hoped it would be possible to extract the data from the diagram, but no, the jpg in the pdf is sufficiently low-resolution that it doesn't work.

I have been compiling a lot of data for part of my PhD and this is a lot more common than I am comfortable with. Personally, as a reviewer I've decided to outright reject papers that don't allow one to extract the data. My preference would be requiring publishing the data straight up, but I can see an editor viewing this as unreasonable, or an author not knowing how to publish data.

With this being said, it's worth asking authors for raw data. My prior on receiving raw data from an author is low, particularly if the study is older. You have nothing to lose, however, and you will sometimes gain respect for helpful researchers. One professor I emailed for about 25 year old data searched his office thoroughly, found the data I wanted on some old floppy disks, got a floppy drive working, and emailed me the files. That was a not insignificant amount of their time, and I'm very appreciative for it.

Also, frequently theses and dissertations have tabulated data. Beware of typos in the tables. Always perform some sort of sanity checks on the data. Checking the tables against the figures is one approach.

Reply
[-]Pfft8y00

So in the case of this particular paper, some other researchers did ask for the raw data, and they got it and carried out exactly the analysis I was interested in knowing about. So I guess it's a happy ending, except I didn't get to write a tumblr post back when there was a lot of buzz in the media about it. :)

Reply
Moderation Log
More from Jacob Falkovich
View more
Curated and popular this week
5Comments