With apologies to those who didn't make this post, it seems you need to up your game

Yesterday, Alexander Wales published a post entitled "Can an LLM have taste? Inkhaven Week 1, ranked by Claude". I found this very entertaining.

He took Claude, used it to compare a bunch of inkhaven posts, ranked them, and provided us with this wonderful list of the top ten posts so far:

1. Three Stones are Enough: The Case Against Leaves, in Particular, Anna Mattinger
2. An open letter to 21 people I know who died, Layla Hughes
3. endometrial biopsy, kaylee
4. Softhead, macroraptor
5. Every Lighthaven Writing Residency, Layla Hughes
6. The largest manufacturer of feelings in human history, Natalie Cargill
7. The one that loved me most, MLL
8. I did it. I found the worst poem in the world., Natalie Cargill
9. “Love, Mum” - What AIs can’t see about abuse, Natalie Cargill
10. Lost Mesoamerican Technologies, Lost Futures

I thought this was great, so I decided to make my own version, but better. Rather than using pure ranking, I (or rather, Claude, who assisted me with the actual implementation of this hare-brained scheme) decided to use a Bradley-Terry model, which Claude informs me is rather like the Elo system used to rank chess players.

Using the Anthropic API, we gave Claude Opus 4.6 the following prompt (also written by Claude, but edited by me), including 8 posts for it to rank:

You are judging posts from Inkhaven, a writing residency where participants commit to publishing one blog post every day for 30 days. The residents are a mix of AI safety researchers, rationalists, fiction writers, and generally thoughtful people. The audience skews heavily rationalist — LessWrong regulars, EA-adjacent, people who take ideas seriously but also appreciate a good joke.

You will be shown 8 Inkhaven posts. Rank them by quality, from best to worst.

The question to ask yourself for each post: "Would a typical rationalist vote to read more of this sort of thing?" You're not rating a single post in isolation — you're judging whether the author, writing in this mode, should keep going. Insight, craft, honest thinking, and distinctive voice all count.

So does being funny — humour is a genuine virtue here, not a tiebreaker.

A few things to keep in mind:

- Do NOT be generous or encouraging. Predict the actual taste of the rationalist audience. Many of these posts will be mediocre and that's fine to say.

- Fiction, essays, rants, reviews, and technical posts are all on the same scale — judge each by whether it succeeds at what it's trying to do.

- Length is not quality. A tight 500 words can beat a bloated 3000.

- Weird and niche is fine, often good. Idiosyncrasy is often a feature, not a bug.

=== POST {i} ===

title + first 4000 chars of body.

Rank all 8 posts from best to worst. Think through your reasoning, then give your final answer as a comma-separated list of post numbers inside <answer> tags.

We did five iterations:

1. Get baseline estimates for each post
2. Get more accurate estimates for posts liable to be in the top 10
3. Get proper estimates for the posts we'd accidentally imported in the wrong format
4. Try to push my post from 2nd to 1st place (instead, it ended up in 10th).
5. Realise that we were missing a bunch of posts and add those in (it didn't change much)

$40 in burned API credits later, we got the following table:

Claude did a diligent bootstrapping check to ensure we had the right posts in the top 20, and found that post 19 was there 90% of the time, while Ben Sturgeon's Revisiting GSM-Symbolic hit a mere 22%. You're on thin ice, Ben.

Averaging the scores of the individual posts also enables us to give a ranking of the authors. The top 20 authors at inkhaven right now are... [drum roll]:

I should probably note that I filtered out anyone who hadn't published posts on at least 2/3rds of the days, so Vishal Prasad (+1.89σ, 2 posts), Robert Mushkatblat (+1.41σ, 4 posts), A.G.G Liu (+0.7σ, 1 post), Justin Kuiper (+0.32σ, 1 post) and Georgia Ray (+0.3σ, 1 post) didn't make the cut on number, despite having the quality.

Alexander Wales (-0.23σ, 4 posts), whose post inspired this one, is also, sadly, left out of the rankings. (Sorry).