[ Question ]

What would you need to be motivated to answer "hard" LW questions?

by Raemon 5mo28th Mar 201935 comments

48


Edit: Significantly rewritten. Original question was more specifically oriented around money-as-a-motivator.

One of the questions (ha) that we are asking ourselves on the LW team is "can the questions feature be bootstrapped into a scalable way of making intellectual progress on things that matter."

Motivations

Intrinsic vs Extrinsic

I'd cluster most knobs-to-turn here into "intrinsic motivation" and "extrinsic motivation."

Intrinsic motivation covers things like "the question is interesting, and specified in a way that is achievable, and fun to answer."

Extrinsic motivation can include things like "karma rewards, financial rewards, and other things that explicitly yield higher status for

(Things like "I feel a vague warm glow because I answered the question of someone I respect and they liked the answer" can blur the line between intrinsic and extrinsic motivation)

Improving Intrinsic Motivation

Right now I think there's room to improve the flow of answering questions:

  • New features such as the ability to spawn related questions that break down a confusing question into an easier.
  • Better practices/culture, such as as a clearer idea of how to specify questions such that they communicate what one needs to do to solve them (or, have a set of common practices among answerers such that this is easier to figure out).
  • A combination (wherein best practices are communicated via tooltips or some-such)

Bounties and Reliability

A lot of questions are just hard to answer – realistically, you need a lot of time, at least some of the time won't be intrinsically fun, and the warm glow of success won't add up to "a few days to a few months worth of work."

So we're thinking of adding some more official support for bounties. There have been some pretty successful bounty-driven content on LW (such as the AI Alignment Prize, the Weird Aliens Question, and Understanding Information Cascades), which have motivated more attention on questions.

Costly signaling of value

They showcase that the author of the question cares about the answer. Even if the money is still relatively minor, it reaffirms that if you work on the question, someone will actually derive value from it, which can be an actual important part of intrinsic motivation (as well as a somewhat legible-but-artificial status game you to more easily play, which I'd classify as extrinsic)

Serious times requires livable-money

In some cases you just actually need to put serious time into solving it to succeed, which means you either need to have already arranged your life such that you can spend serious time answering questions on LW, or you need "answering hard questions on LW" to actually provide you with enough financial support to do so.

This requires not just "enough" money, but enough reliability of money that "quit your day job" (or get a day job that pays-less-but-gives-more-flexiblity) is a an actual option.

What would it take?

So, with all that in mind...

What would it take for you (you, personally), to start treating "answer serious LW questions" as a thing you do semi-regularly, and/or put serious time into?

My assumptions (possibly incorrect) here are that you need a few things (in some combination)

  • A clear enough framework for answering questions, that relies on skills you already have (and/or a clear path towards gaining them)
  • A sense that the questions matter
  • Enough money (and reasonable expectation of earning it) for a given question that working on it is worth the hours spent directly on it, if you're doing work that's demanding enough that it doesn't funge against other hobby activities.
  • Enough reliability of such questions showing up in your browser that you can build a habit of doing so, such that you reallocate some chunk of your schedule (that formally went either to another paying job, or perhaps some intellectual hobby that trades off easily against question answering)

Some types of intellectual labor I'm imagining here (which may or may not all fit neatly into the "questions" framework).

  • Take an scientific paper that's written in confusing academic-speak PDF format, and translate it into "plain english blogpost."
    • bonus points/money if you can do extra interpretive work to highlight important facts in a way that lets me use my own judgment to interpret them
  • Do a literature review on a topic
  • If you already know a given field, provide a handy link to the paper that actually answers a given question.
  • Figure out the answer to something that involves research
    • (can include contributing to small steps like 'identify a list of articles to read' or 'summarize one of those articles' or 'help figure out what related sub-questions are involved')
  • Conduct a survey or psych experiment (possibly on mechanical turk)

"Serious" questions could range from "take an afternoon of your time" to "take weeks or months of research", and I'm curious what the actual going rate for those two ends of the spectrum are, for LW readers who are a plausible fit for this type of distributed work.

48

New Answer
Ask Related Question
New Comment

7 Answers

I feel like I should provide some data as someone who participated in a number of past bounties.

  1. For one small bounty <$100, it was a chance to show off my research (i.e., Googling and paper skimming) skills, plus it was a chance to learn something that I was somewhat interested in but didn't know a lot about.
  2. For one of the AI alignment related bounties (Paul's "Prize for probable problems" for IDA), it was a combination of the bounty giver signaling interest, plus it serving as coordination for a number of people to all talk about IDA at around the same time and me wanting to join that discussion while it was a hot topic.
  3. For another of the AI alignment related bounties (Paul's "AI Alignment Prize"), it was a chance to draw attention to some ideas that I already had or was going to write about anyway.
  4. For both of the AI alignment related bounties, when a friend or acquaintance asks me about my "work", I can now talk about these prize that I recently won, which sounds a lot cooler than "oh, I participate on this online discussion forum". :)

I don't think it's possible on LW. It's not a matter of money (ok, it is, in that I don't think anyone's likely to offer a compelling bounty that I expect to be able to win). It's not a matter of reliability of available offers (except that I don't expect ANY).

It's _is_ a question of reliability and trust, though. There are no organizations or people I trust enough to define a task well and make sure multiple aren't competing in some non-transparent way, so that I actually expect to get paid for the work posted on a discussion site. And I don't expect that I have enough track record for any bidder to prefer me for the kind of tasks you're talking about at the rates I expect. [edit to add] Nor do I have any tasks where I'd prefer a bounty or open-bid rather than finding a partner/employee and agreeing on specific terms.

It's also a question of what LW is for - posting and discussion of thought-provoking, well-researched, interestingly-modeled, and/or fun ideas is something that's very hard to measure in order to reward monetarily. Also, I'll be massively demotivated by thinking of this as a commercial site, even if I'm only in the free area.

My recommendation would be to use a different place to manage the tasks and the bid/ask process, and the acceptance of work and payment. Some tasks and their outputs might be appropriate to link here, but not the job management.

tl;dr: don't mix money into LW. Social and intellectual rewards are working pretty well, and putting commerce into it could well kill it.

If answering the question takes weeks or months of work, won't the question have fallen off the frontpage by the time the research is done?

What motivates me is making an impact and getting quality feedback on my thinking. These both scale with the number of readers. If no one will read my answer, I'm not feeling very motivated.

For the examples you give, the improvements you cite to intrinsic motivation + karma would be sufficient to motivate me for questions of the "take an afternoon of your time" type, which is approximately where my blogposts have been landing anyway. Further, several are already of the summarize papers/point to a list of sources type. On the long end of weeks or months, bounties in the hundreds would probably satisfy depending on the level of interest I have, which is the true variable in whether I engage.

It is hard to tell in the current format what kind of depth-of-answer the questioner is looking for, and what time frame would be appropriate for an answer. It is also hard to tell how well answered a question already is, which has a big impact on reading older questions or questions with many answers. Mostly I have been viewing questions at the same rate as blog posts, but it occurs to me that they don't age in the same way informative or exploratory posts do; the question is unresolved until it is.

Having some way to disentangle the content of this site from when it was posted would be handy.

Given that there is some probability of winning a question, let's just guess it's 20% on any particular question I might try to answer. This suggests to me a bounty of 5x whatever I would be willing to answer the question for in order to make me willing to do it. Assuming a question takes about a day of work (8 hours) to answer fully and successfully, and given our 5x multiplier, I'd be willing to try to answer a question I wasn't already excited to answer for other reasons if it paid about $1800.

Many others may have lower opportunity costs, though (and I undercounted a bit because I assume any question I would answer would deliver me at least some sense of value beyond the money; otherwise my number would probably jump up closer to $2500).

It would help if the poster directly approaches or tags me as a relevant expert.

If this blog's "hard questions" have utility, they should be novel, important, and answerable.

Important questions are highly likely to be known already among experts in the relevant field. If they're answerable, one of those experts is likely already working on it with more rigor than you're capable of extracting from a crowd of anonymous bloggers. I think, then, that any questions you ask have a high probability of being redundant, unimportant, or unanswerable (at least to a useful degree of rigor). Unfortunately, you're unlikely to know that in advance unless you vet the questions with experts in the relevant literature.

And at that point, you're starting to look like an unaccountable, opaque, disorganized, and underresourced anonymously peer-reviewed journal.

It might be interesting to explore the possibility that a wiki-written or amateur-sourced peer reviewed journal could have some utility, especially if it focused on a topic that is not so dependent on the expensive and often opaque process of gathering empirical data. I expect that anyone who can advance the field of mathematics is probably already a PhD mathematician. So philosophy, decision theory, something like that?

Developing a process to help an anonymous crowd of blog enthusiasts turn their labor into a respectable product would be useful and motivating. I would start by making your next "hard question" what specific topic such a PRJ could usefully focus on.