I had written this previously in The Review Phase comment threads, but to make it easier for people to see the individual reviews I gave prizes to:
This comment is incomplete, and I will likely edit some of the prizes slightly. But, I'd fallen behind on awarding prizes for reviews and I want to highlight Yes, the Lightcone team will give you money for reviewing stuff. So, I wanted to ship this rough version for now to give a sense of what sort of reviews I found valuable.
Thanks to the large number of people who've stepped up to review so far. (I'd still be excited for EffortReview that explore the details of some of the higher ranked posts)
Next round of prize announcements:
- $200 to Nostalgebraist for this review of Fun with +12 OOMs of Compute. I think I didn't really get the underlying point of the post until I read this.
- $200 to AllAmericanBreakfast for both his review and followup comment on Core Pathways of Aging.
- $200 to Akash for his review of ARC's first technical report: Eliciting Latent Knowledge, breaking out a lot of details about how ELK affected Akash and people in his cohort, and ways he thinks ELK could be improved.
- $150 to Alex Altair's review of Taboo Outside View.
- $150 to Yoav Ravid for their review of What do GDP Growth Curves Really Mean? I really liked that they did a mini-distillation of the older good comments about the post.
- $150 to la3orn for their self-review of EfficientZero: How It Works, in particular for grading their conclusions and predictions.
- $100 to A Ray's review of Coase's "Nature of the Firm" on Polyamory for explaining something subtly off about
- $100 to A Ray for his review of Politics is way too meta
- $100 to A Ray for his review of Cryonics Signup Guide #1: Overview
- $100 to Leon Lang for his fairly detailed review of Utility Maximization = Description Length Minimization.
- $100 to Valentine for his self review of Creating a Truly Formidable Art.
- $100 to Zach Davis for his self-review of Feature Selection, in particular for situating it more clearly in the context of his other work, and previous work it was building on.
- $100 to Dragon God for his review of EfficientZero: How It Works, which dug into some details and found a potential error.
- $100 to Vanessay Kosoy for their self-review of Infra-Bayesianism. I appreciated her laying out what developments had progressed in the intervening year.
- $100 to Neel Nanda for his self-review of Intentionally Making Close Friends
- $100 to Steven Byrnes review of Robin Hanson's Grabby Aliens model explained
- $100 to Turntrout for his self-review of Lessons I've Learned From Self-Teaching. I love that he had some data on how his attempts had faired.
- $100 to Adam Shimi for for his review of Biology-Inspired AGI Timelines: The Trick That Never Works. I found this particularly valuable for breaking out some of the mistakes Eliezer is pointing out in a fairly distilled fashion. (I personally had trouble actually taking away the points from the post itself)
- $100 to Vaniver for his review of ARC's first technical report: Eliciting Latent Knowledge.
- $100 to la3orn for their review of Coordination Schemes are Capital Investments.
- $100 to Rafael Harth for their review of [Book review] Gödel, Escher, Bach: an in-depth explainer
- $100 to MondSemmel for his review of Where did the 5 micron number come from? Nowhere good?
- $100 to Logan Strohl for their self-review of Catching the Spark. I like that Logan brought a lot more experience to bear on an earlier topic, including noticing a place where their original frame was maybe 50% wrong.
- $100 to Daniel Kokotajlo for his self review of What 2026 looks like.
- $75 to Alex Flint for his self-review of Agency in Conway's Game of Life. I appreciated the link to the Game of Life community where people were researching a question Alex had posted. (My impression is they were working on that independent of Alex's work)
- $75 to Adam Shimi for his review of What Multipolar Failure Looks Like, and Robust Agent-Agnostic Processes.
- $75 to Rafael Hart's review of Dear Self: We Need to Talk About Social Media. I particularly was interested in the articulation of how "vibe" comes across in posts, and why that matters. I did feel a bit surprised that Rafael found the particular vibe in Elizabeth's piece and not in many other LessWrong pieces (maybe he did, and just noticed it here?), but I found it a helpful frame for thinking about what value I get from posts.
- $100 to Leon Lang for their review of the Telephone Theorem.
Honorable Mentions:
These reviews didn't go into much detail, but added at least a couple new arguments or frames that I found useful, and each get $50
- Zach Stein-Perlman for their review of Fun with +12 OOMs of Compute. I appreciated him laying out the specific reasoning for how it improved his thinking.
- Adam Jermyn for their review of Your Cheerful Price.
- gears to ascension for their review of Your Cheerful Price.
- AllAmericanBreakfast for their review of Oliver Sipple. I particularly liked the evidence-evaluation investigation of "on one hand, it'd be weird to have two unrelated phenomena happening at the same time, so maybe the story just checks out. On the other hand, the two phenomena are only promotedto our attention because of
- Daniel Koko for his self-review of Birds, Brains, Planes and AI.
- Darmani for their self-review of Leaky Delegation: You Are Not A Commodity, with details about which of their attempts at employing their own advice hadn't worked out
- Eli Tyre for his self-review of How do we prepare for the final crunch time?
- John Wentworth on Highlights from the Autobiography of Andrew Carnegie
- Double's review of Against Neutrality About Creating Happy Lives.
- Alex Altair for his review of Lessons I've Learned From Self Teaching.
- Valentine for his review of Gravity Turn
- Daniel Kokotajlo for his review of Comments on Carlsmith's “Is power-seeking AI an existential risk?”
- Daniel Kokotaljo for his review of Fixing The Good Regulator Theorem
- A Ray for his review of Rationalism Before the Sequences. Reading the review got me thinking more about the value of good bibiographies.
- Coafos for his review of Social Behavior Curves, Equilibria and Radicalism
- Coafos for his review of Curing Insanity with Malaria.
- Srdjan Miletic for his review of https://www.lesswrong.com/posts/gziZACDg6EBpGZbJe/almost-everyone-should-be-less-afraid-of-lawsuits?commentId=i6B94ofbtKBdKridn
- Akash for his review of How To Get Into Independent Research On Alignment/Agency
- Unnamed for for their review of Taboo Outside View
- Srdjan Miletic for their review of Almost everyone should be less afraid of lawsuits
- iceman's review of Unnatural Categories are Optimized for Deception. I didn't feel like I learned new things from it exactly, but I do appreciate iceman laying out his reasoning, what he found valuable, and some specific reasoning that he may have done better because of it.
Oh, also meant to give a $200 prize to @Akash for his review of PR is Corrosive, Reputation is not (along with the full post he wrote, inspired by it. I'm thinking of this as roughly $100 for the review itself and another $100 for the post)
We've had a ton of reviews for the The LessWrong 2021 Review, and I wanted to take a moment to: a) share highlights from reviews that felt particularly valuable to me, b) announce the prizes so far, to give people a better idea of how prizes are awarded.
Tl;dr on prizes: we've awarded $4,425 in prizes so far. I've been aiming to pretty consistently give at least a $50-$100 prize for all reviews that put in some effort, and am happy to pay more for good reviews. The largest prize so far has been $200 but I'd be excited to give $500+ to in-depth reviews of posts that were ranked highly in the preliminary voting.
If you'd consider doing reviews but are uncertain about the payoff, PM me and most likely we can work out something out.
Reviews I liked
Nostalgebraist on Fun with +12 OOMs of Compute
After reading Nostalgebraist's review of Fun with +12 OOMs of Compute, I felt like I actually understood the main point for the first time:
(I think Zach Stein-Perlman's review makes a similar point more succinctly, although I found nostalgebraist's opening section better for driving this point home).
Nostalgebraist notes that the actual details of the post are pretty handwavy:
AdamShimi, Joe_Collman, Gyrodiot on Fun with 12 OOMs of Compute
Meanwhile, Adam Shimi had previously led a round of in-depth peer review in 2021, including delving into the same 12 OOMs post. Adam, Joe and Gyrodiot's Review of "Fun with +12 OOMs of Compute" was also nominated for the LessWrong 2021 Review as a top-level post, but I wanted to draw attention to it here as a
Some of their notes on the details:
I recommend reading their post in full. I'm likely to give this post a retroactive "review prize" that's more in the $400 - $1000 range but I haven't finished thinking about what amount makes sense.
Habryka on the 2021 MIRI Dialogues
Alex Ray on Coase's "Nature of the Firm" on Polyamory
Alex Ray left a few interesting reviews. I found his discussion on 1a3orn's essay, "Coase's "Nature of the Firm" on Polyamory", pretty valuable from the standpoint of "what makes an analogy work, or not?".
gears of ascension and Adam Jermyn on Your Cheerful Price
Eliezer's Your Cheerful Price has the highest number of reviews so far – it seemed like a lot of people had the post directly influence their lives. Several reviews mostly said 'yep, this was useful', But I'm including two reviews that went into some details:
gears of ascension:
Adam Jermyn:
Adam Shimi on What Multipolar Failure Looks Like, and Robust Agent-Agnostic Processes
AllAmericanBreakfastDirectedEvolution on Core Pathways of AgingAllAmericanBreakfastDirectedEvolutionhad both a high-level review and some nitty-gritty details examining John Wentworth's piece of the mechanics of aging.The high level overview:
And then digs into a claim that senolytics rapidly wear off:
John Wentworth replies, and I think the back-and-forth was worthwhile.
La3orn's Self Review of EfficientZero: How It Works
I particularly liked this for grading how his predictions bore out.
Akash on ARC's first technical report: Eliciting Latent Knowledge
Prizes
Prize Philosophy
It's a fair amount of work to review things, and my aim has been to pay people roughly commensurate with the time/value they're putting in. My policy has been to give $50 prizes for reviews which give at least some details on the gears of why a person liked or didn't like a post, $100 for reviews that made a pretty good faith effort to evaluate a post or provide significant new information, then more (using my judgment) if the review seems to be some combination of higher-value and and higher effort.
So far no one has written any particularly in-depth reviews during the Review Phase. In my ideal world, the top-scoring essays all receive a fairly comprehensive review that engages with both the gritty details of the post, and the high level "how does this fit into the surrounding landscape?". If you're interested in contributing high-effort review for the top-ranked posts so far, you can view the top-ranked posts here and see if any of them call out to you. I'd be happy to pay $300-$1000 for very comprehensive reviews.
In the past 1-2 years there have been some posts that were extensively reviewing other posts (i.e. Reviews of “Is power-seeking AI an existential risk?”, MIRI comments on Cotra's "Case for Aligning Narrowly Superhuman Models", and Review of "Fun with +12 OOMs of Compute", etc). I am leaning towards awarding those posts with retroactive prizes but am still thinking through it.
Prizes So Far
Here's the overall prize totals so far. You can see the complete list of review-prizes here, which includes some comments on what I found valuable.
Thanks to everyone has reviewed so far. Reminder that I'd like the top-scoring posts to get more in-depth reviews.