Voting Phase for 2019 Review

by Raemon2 min read13th Jan 202119 comments

53

LessWrong ReviewSite Meta
Frontpage

Click here to begin voting. Click here for a general overview of the 2019 review.

We are now in the final stretches of the LessWrong 2019 Review. We’ve spent 2 weeks nominating posts, and a month reviewing them. And today, we begin the final vote!

A recap on the major goals of the Review: 

  • First, it is an experiment in improving the LessWrong community's longterm feedback and reward cycle.
  • Second, it is an attempt to build common knowledge about the best ideas we've discovered on LessWrong.
  • Third, after the vote, the LessWrong Team will compile the top posts into a physical book. (We may make some judgment* calls about what to include, but will use the vote as a strong guideline)

Voting starts today (January 12th) and continues through Jan 26th. 

Who can vote?

All users registered before 2019 can vote, but the LW curation team will primarily be paying attention to the votes of users with 1000+ karma. 

(There will essentially be a “top users’ vote”, and a “people’s vote.” The results of both will be made public, but the longterm-users’ vote will be the main thing influencing the Best of 2019 Book)

Anyone can still write reviews

For all users and lurkers, regardless of karma, the next 2 weeks are your last opportunity to write reviews for any nominated posts in 2019, which can influence how people vote. As you can see below, all reviews are highlighted when a user is voting on a post. (To review a post, go to the post and click "Write A Review" at the top of the post.)

Posts need at least 1 review to appear in the vote. You can still review previously un-reviewed posts to put them on the ballot. (But, people who vote early might not see them)

How do I vote?

To vote, head over to lesswrong.com/reviewVoting

The vote has a simple first section, and a detailed-yet-optional second section based on quadratic voting

Sorting Posts Into Buckets

The first part of voting is sorting the nominated posts into buckets.

The five buckets are: No, Neutral, Good, Important, Crucial. Sort the posts however you think is best.

The key part is the relative weighting of different posts. For example, it won't make a difference to your final vote if you put every post in 'crucial' or every post in 'good'.

Fine-Tuning Your Votes

Once you're happy with your buckets, you can click 'Convert to Quadratic'. At this point the system converts your buckets roughly into their quadratic equivalents. 

The system will only assign integer numbers of votes, which means that it will likely only allocate around 80-90% of the total votes available to you. If you vote on a smaller number of posts (<10), the automatic system may not use your entire quadratic voting budget.

If you're happy with how things look, you can just leave at this point, and your votes will be saved (you can come back any time before the vote closes to update them). But if you want to allocate 100% of your available votes, you'll likely need to do fine-tuning. 

There are two key parts of quadratic voting you need to know:

First, you have a limited budget of votes.

Second, votes on a post have increasing marginal cost.

This means that your first vote costs 1 point, your second vote on that post costs 2 points, your third costs 3 points. Your nth vote costs n points.

You have 500 points to spend. You can see how many points you've spent at the top of the posts.

The system will automatically weight the buckets differently. For example, I just did this, and I got the following weightings:

  • Good: 2 votes.
  • Important: 4 votes.
  • Crucial: 9 votes.
  • Neutral: 0 votes.
  • No: -4 votes.

(Note that negative votes cost the same as positive votes. The first negative vote costs 1 point, the second negative vote costs 2 points, etc.)

You'll see your score at the top of the page. (When I arrived on the fine-tuning page, the system had spent about 416 points, which meant I had a significant number of votes left to buy.)

Once you're happy with the balance, just close the page; your votes will be saved.

You can return to this page anytime until voting is over, to reconfigure your weights.

Quick Note Buttons

This year, in addition to full reviews, we’re providing these “quick note” buttons.

For each post, click any of these buttons if it describes your opinion of the post. These will be aggregated to get a sense of people’s qualitative opinion on the post.

Happy Voting

That’s it! Go forth and vote, and let us know if you have any questions, bug-fixes, or suggestions for next year!


* The LW Team reserves the right to make judgment calls about what actually goes in the book. Last year, we left off a couple posts that were too long, and  included brief notes about a few lower-ranked posts that we felt were important. But, overall we'll be taking the vote as a strong indicator of what the LessWrong community thought was important.

53

18 comments, sorted by Highlighting new comments since Today at 6:27 AM
New Comment

I have voted! :)

I will maybe check it over once again before the voting period ends, but I'm taking a vacation and cutting myself off from most social/communal places on the internet including LW, so no promises.

Out of interest, due to the new renormalizing button, all my votes were lowered by 2 points. Whereas last year my votes span from about -1 to +8, this year they span from -3 to +6. I spent exactly 500/500 points. Doing a manual sum, my average vote was 0.39, meaning I used most of my voting power.

(In lots of my reviews, I said what I expected I'd vote on a post. Once you account for the -2 on everything, I was accurate in all of the predictions I made.)

I've also written down my guesses for what will be at the top of vote once it's all done.

Voting Suggestions

My current 3 guesses for most underrated posts are:

So I encourage you to check them out for voting on :)

I too have voted!

And then I walked away to come back later for points adjustments. I don't expect to make many more, but see no reason not to keep the option.

Yeah, I think the upfront "get mosts of the votes in" step is worth celebrating, even if you plan to fine-tune it later.

Woop, go you! :)

Note: If you haven't been voting because it felt overwhelming, and want to at least vote on posts that you remember well from 2019 – we've recently added a feature on the voting page where you can see which posts you previously voted on. You'll be able to quickly scan for posts that presumably you have at least some vague memories of.

Handy link for voting page.

Thanks, that's a helpful feature :)

Could be nice if it also differentiated between read and unread posts (some posts I'm not sure if I've read, but if it showed me i read it then it's more likely that it's a post i forgot about because it didn't affect me much, which can inform my vote)

Voted! I put most (but not all) my votes on AI Alignment posts that are incredibly valuable to me, and I believe to the field.

I might actually read some more stuff today and give my additional votes to a couple of relevant posts.

I've now completed my first pass of voting!

I think so far I've thought somewhat seriously about 1/3rd to 1/2 of the posts, and the rest I'm going somewhat off of vague memories (by default I don't vote very strongly on those posts).

I expect to look over some more posts over the next week and fine-tune the votes.

It seems that I can still modify my votes, despite voting being over?

Sorry, we just haven’t merged the update yet, but will shortly.

Something seems to be wrong with the voting system—I entered my votes around when the voting phase started, including adjusting my quadratic votes to ensure I was as close as possible to 500 while staying under, then came back today and found that now somehow I'm over 500, despite not having changed any of my votes in the interim.

Oh, sorry. I know why this happened. Last year we had a bug where downvotes of strength 1 didn't cost you anything (and downvotes of strength 2 only cost as much as strength 1, etc.), because of a simple off-by-one error in the formula we were using. 

When I pushed the vote on Monday night, I accidentally didn't include the fix I had for that bug, noticed my mistake on Wednesday and fixed it right then, hoping that nobody had entered all of their votes yet. Apparently I was wrong. 

I did check the data, and nobody else was as fast as you were, and nobody is currently above the 500 limit, so I don't think anyone else walked into the same trap. 

Sorry for the confusion! Should have double checked that nobody had indeed accidentally gone over the limit. 

Hrm. Sent a PM to chat more about it.

It would be nice if the interface showed your average vote so following the advice of keeping it somewhere between -1 and 1 would be easier.

It shows you as soon as your average goes above 1 or below -1, with a button to automatically renormalize (currently on my phone, otherwise would post a screenshot). 

I'd either mention that in the instructions, or make it visible all the time, cause i spent time on seeing if anyone mentioned how to check your average.

We tried making it visible all the time, but it ended up making the whole UI a bit too overwhelming. But agree that it should explain that the widget will show up somewhere in the instructions.