Lesswrong 2016 Survey

It’s time for a new survey!

Take the survey now


The details of the last survey can be found here.  And the results can be found here.

 

I posted a few weeks back asking for suggestions for questions to include on the survey.  As much as we’d like to include more of them, we all know what happens when we have too many questions. The following graph is from the last survey.


http://i.imgur.com/KFTn2Bt.png

KFTn2Bt.png

(Source: JD’s analysis of 2014 survey data)


Two factors seem to predict if a question will get an answer:

  1. The position

  2. Whether people want to answer it. (Obviously)


People answer fewer questions as we approach the end. They also skip tricky questions. The least answered question on the last survey was - “what is your favourite lw post, provide a link”.  Which I assume was mostly skipped for the amount of effort required either in generating a favourite or in finding a link to it.  The second most skipped questions were the digit-ratio questions which require more work, (get out a ruler and measure) compared to the others. This is unsurprising.


This year’s survey is almost the same size as the last one (though just a wee bit smaller).  Preliminary estimates suggest you should put aside 25 minutes to take the survey, however you can pause at any time and come back to the survey when you have more time.  If you’re interested in helping process the survey data please speak up either in a comment or a PM.


We’re focusing this year particularly on getting a glimpse of the size and shape of the LessWrong diaspora.  With that in mind; if possible - please make sure that your friends (who might be less connected but still hang around in associated circles) get a chance to see that the survey exists; and if you’re up to it - encourage them to fill out a copy of the survey.


The survey is hosted and managed by the team at FortForecast, you’ll be hearing more from them soon. The survey can be accessed through http://lesswrong.com/2016survey.


Survey responses are anonymous in that you’re not asked for your name. At the end we plan to do an opt-in public dump of the data. Before publication the row order will be scrambled, datestamps, IP addresses and any other non-survey question information will be stripped, and certain questions which are marked private such as the (optional) sign up for our mailing list will not be included. It helps the most if you say yes but we can understand if you don’t.  


Thanks to Namespace (JD) and the FortForecast team, the Slack, the #lesswrong IRC on freenode, and everyone else who offered help in putting the survey together, special thanks to Scott Alexander whose 2014 survey was the foundation for this one.


When answering the survey, I ask you be helpful with the format of your answers if you want them to be useful. For example if a question asks for an number, please reply with “4” not “four”.  Going by the last survey we may very well get thousands of responses and cleaning them all by hand will cost a fortune on mechanical turk. (And that’s for the ones we can put on mechanical turk!) Thanks for your consideration.

 

The survey will be open until the 1st of may 2016

 


Addendum from JD at FortForecast: During user testing we’ve encountered reports of an error some users get when they try to take the survey which erroneously reports that our database is down. We think we’ve finally stamped it out but this particular bug has proven resilient. If you get this error and still want to take the survey here are the steps to mitigate it:

 

  1. Refresh the survey, it will still be broken. You should see a screen with question titles but no questions.

  2. Press the “Exit and clear survey” button, this will reset your survey responses and allow you to try again fresh.

  3. Rinse and repeat until you manage to successfully answer the first two questions and move on. It usually doesn’t take more than one or two tries. We haven’t received reports of the bug occurring past this stage.


If you encounter this please mail jd@fortforecast.com with details. Screenshots would be appreciated but if you don’t have the time just copy and paste the error message you get into the email.

 

Take the survey now


Meta - this took 2 hours to write and was reviewed by the slack.


My Table of contents can be found here.

273 comments, sorted by
magical algorithm
Highlighting new comments since Today at 4:02 PM
Select new highlight date

I am literally pregnant right now and wasn't sure how to answer the ones about how many children I have or if I plan more. (I went with "one" and "uncertain" but could have justified "zero" and "yes").

Congratulations!

My wife is also pregnant right now, and I strongly felt that I should include my unborn child in the count.

[Survey Taken Thread]

By ancient tradition, if you take the survey you may comment saying you have done so here, and people will upvote you and you will get karma.

Let's make these comments a reply to this post. That way we continue the tradition, but keep the discussion a bit cleaner.

I have taken the survey. I did not treat the metaphysical probabilities as though I had a measure over them, because I don't.

Similarly, I gave self-conscious nonsense numbers when asked for subjective probabilities for most things, because I really did not have an internal model with few-enough free parameters (and placement of causal arrows can be a free parameter!) to think of numerical probabilities.

So I may be right about a few of the calibration questions, but also inconsistently confident, since I basically put down low (under 33%) chances of being correct for all the nontrivial ones.

Also, I left everything about "Singularities" blank, because I don't consider the term well-defined enough, even granting "intelligence explosions", to actually talk about it coherently. I'd be a coin flip if you asked me.

So basically, sorry for being That Jerk who ruins the survey by favoring superbabies and restorative gerontology, disbelieving utterly in cryonics and the Singularity, and having completely randomized calibration results.

I took the survey 2 days ago. It was fun. I think I was well calibrated for those calibration questions, but sadly there was no "results" section.

Is it possible to self-consistently believe you're poorly calibrated? If you believe you're overconfident then you would start making less confident predictions right?

Being poorly calibrated can also mean you're inconsistent between being overconfident and underconfident.

You can be imperfectly synchronised across contexts & instances.

I have taken the survey. I like the new format.

Just finished. I'm sure my calibration was terrible though.

I have taken the survey. I left a lot of questions blank though, because I really have no opinion about many of them.

Took the survey, had the recurring survey confusion about some questions. For instance, I think some taxes should be higher and others should be lower. Saying I have no strong opinion is inaccurate but at least it seemed like the least inaccurate answer.

RE: The survey: I have taken it.

I assume the salary question was meant to be filled in as Bruto, not netto. However that could result in some big differences depending on the country's tax code...

Btw, I liked the professional format of the test itself. Looked very neat.

Took it!

It ended somewhat more quickly this time.

I completed the survey. I also like the new format - easy to read, good instructions etc.

Took survey. Didn't answer all the questions because I suspend judgment on a lot of issues and there was no "I have no idea" option. Some questions did have an "I don't have a strong opinion" option, but I felt a lot more of them should also have that option.

I have taken the survey.

The only option i think was missing was in the final questions about quantities donated to charities, an option such as "I intend to donate more before the end of the financial year" or similar. (and while likely not feasible, following up on those people in the next survey to see if they actually donated would be interesting)

For a few moments I was paralyzed with uncertainty about how humorous to try to make my "I took the survey" response, since many seemed to have made a similar attempt, thus this post took longer to finish than the survey itself, which I have taken.

For the interests of identity obfuscation, I have rolled a random number between 1 and 100, and have waited for some time afterwards.

On a 1-49: I have taken the survey, and this post was made after a uniformly random period of up to 24 hours.

On a 50-98: I will take the survey after a uniformly random period of up to 72 hours.

On a 99-100: I have not actually taken the survey. Sorry about that, but this really has to be a possible outcome.

I have taken the survey.

I think I spent about 1 hour and 20 minutes answering almost all of the questions. I'm probably just unusually slow. :P

Took the survey, and as others pointed out had some trouble with the questions about income (net? gross?) Also, is there any place where all the reading (fanfiction, books, blogs) hinted to in the survey are collected? I knew (and have read) some, but many I have never heard of, and would like to find out more.

Was taking it, and it crashed with a "This webpage is not available" error.

We had some power outage related downtime for three hours or so, should be back up now.

I'm a little unclear on how to proceed. I didn't establish a "save", so I can't really resume the survey. Does that mean I should start a new survey and pick up where I left off, or ... ?

If you'd be willing to go through the trouble of doing it, yes that's exactly what you should do. I didn't think of that, thanks.

Though from a data-consistency perspective people doing this would skew our response rate higher than it really is, I'd rather have the question data than an accurate response rate though so. shrug

On the session timeout front, we're trying something out to make the sessions longer, which should cut down on that particular problem significantly.

I've taken the survey.

Thanks Huluk for creating this subthread, very handy when reading others' comments about the survey itself.

Besides saying that I have taken the survey...

I would also like to mention that the predictions of probabilities of unobservable concepts was the hardest one for me. Of course, there are some in which i believe more than in some others, but still, any probability besides 0% or 100% seems really strange for me. For something like being in a simulation, if I would believe it but have some doubts, saying 99%, or if I would not believe but being open to it and saying 1%, these seem so arbitrary and odd for me. 1% is really huge in the scope of very probable or very improbable concepts which cannot be tested yet (and some may never ever be).

... before losing my sanity in trying to choose the percentages I would find plausible at least a few minutes later, I had to fill them based on my current gut feelings instead of Fermi estimation-like calculations.