2239

LESSWRONG
LW

2238
PracticalWorld Modeling
Frontpage

25

What do you Want out of Literature Reviews?

by Elizabeth
14th Jul 2025
Aceso Under Glass
4 min read
4

25

25

What do you Want out of Literature Reviews?
8DirectedEvolution
5Valdes
3Aprillion
2Elizabeth
New Comment
4 comments, sorted by
top scoring
Click to highlight new comments since: Today at 2:56 PM
[-]DirectedEvolution2mo82

Put your most important, concrete takeaways first, in a form anyone can understand. Write with authority. Put your methods, motives, and analysis details later.

Right now, I have to read too much text to figure out what you’re claiming and why it matters, even in the summary. For example, for your gargling paper, you could have led with:


“A few tiny studies show water gargling prevents illness, but I suspect publication bias is skewing the reported results. Notably, one paper showed long-term side effects from frequent gargling. In this review, I walk through the most important results and methods from each of these studies and discuss why I think there’s a risk of publication bias. For context, I’m an independent researcher with [relevant background qualifications] and this is a personal blog post, not traditional peer-reviewed research.”

Reply11
[-]Valdes2mo50

I think what I like the most is when there is a good summary story with the option to read more details on the parts that interest me, ideally with these complements forming their own coherent article. The structure this suggests is to have two posts:

  • One that gives a high level picture focused on the general story, what I need to understand, and what I am expected to understand. With regular (hyperlink) references to parts of the second.
  • One that gives a lot of details and can still stand on its own if I want a deep dive.

One great advantage of this is that the first also serves as a primer on the second, making it easier to read it.

I can't think of an example of blogposts doing this, though I think there are some sequences on LW that follow this logic. But some books do this, with detailed chapters following high level summaries. "The mind illuminated" and "Against method" come to mind.

Reply1
[-]Aprillion2mo30

If I may take the liberty for a somewhat broader take, what I value from human literature review over chatbot assistant slop (*cough* "research by LRMs/agents with internet search") is:

  • judgement (I mean having any spine whatsoever, even if "good judgement" were a better desiderata .. but that would be asking for sycophancy so I am not asking for any self-defining/defying qualities) and
  • faithful reasoning (frankly, I am not going to follow the steps to do my own research, but if I imagine I would be a better person and do my own research, it makes me feel better to see the steps / a recipe how I could form my own conclusions following a sound methodology ... what are the cruxes where "reasonable people can disagree" vs what are the tar pits where there's "no fucking way anyone shall possibly believe anything other than X") - I don't care whether I agree or disagree with an opinion, but I want to see firm hinges in arguments to help me understand multiple perspectives, anything is better than extreme vagueness that doesn't say anything of substance using too many words to not say it (called "AI slop" these days but that style of prose was invented long before AI and I am allergic to it ... I don't believe you are at any risk of producing that, so please keep that quality whatever else you might change)

..for example, if there are multiple reasons why a study is bad, it would be enough (for me) to explain in details only the worst reason without a long list of all bad things (if sample size was small, but there was also a bigger problem for which increasing sample size would not help anyway, it's fine to summarize all minor flaws in 1 sarcastic sentence and go into explanation just for the worst mistake they made, why their methodology could not possibly prove anything about the topic one way or the other ... or moving all extra info into appendix A-J or <details> element might be helpful to keep stuff short(er)((ish)))

Reply1
[-]Elizabeth2mo20

@Thomas Kwa I'm very interested in your take here, since you've caught an error in my work in the past. 

Reply
Moderation Log
More from Elizabeth
View more
Curated and popular this week
4Comments
PracticalWorld Modeling
Frontpage

Tl;dr how can I improve my literature-review based posts?

I write a fair number of blog posts that present the data from scientific papers. There’s a balancing act to this- too much detail and people bounce off, too little and I’m misleading people. I don’t even think I’m on the pareto frontier of this- probably I could get better at which details I share and how I share them, to improve readability and rigor at the same time. This post is a little bit my thoughts on the matter and a lot of requests for input from readers- what do you actually want to see? What are examples of doing this well? Any requests for me personally?

I ask for audience feedback explicitly at a few points, but please don’t limit yourself to those. I’m interested in all suggestions and examples .

Context

If you’re just tuning in, here’s a few examples of posts I mean:

  • Salt water gargling as an anti-viral
  • Long Covid Is Not Necessarily Your Biggest Problem
  • Iron deficiencies are very bad and you should treat them

These are all posts where the bulk of the text is describing individual papers, but I have some conclusion I would like the reader to consider.

My motivating example is my project on the risks of long term ketamine use. Right now I’m working on a technical post on how to translate doses consumed by humans into concentrations in the cerebro spinal fluid (draft ), which is reference material for a post people might actually read.

Principles

Epistemic Legibility

My goal is always to present information to people they can interpret for themselves, rather than rely on my summaries. My proudest moment as a researcher was when I was hired by a couple to investigate a particular risk during pregnancy, and due to different risk tolerances they came to opposite conclusions from the same model. To accomplish this, I need to give people the relevant details, in as digestible a format as possible. 

What helps you connect with scientific posts? Some ideas:

  1. My search process
  2. My selection criteria
  3. My conclusions
  4. Motivation
  5. Your ideas here

Then there’s the papers themselves. For the ketamine dosing post, there’s  <20, maybe <10 papers in the world that meet my criteria for inclusion, so it’s feasible to include details on each of them. But which details help people understand, and which aren’t worth the attention they cost? 

Some paper details I could include:

  1. Sample size. 
  2. Experimental set up
  3. Key graphs
  4. Description of results
    1. Averages, or with confidence intervals?
  5. My criticisms
  6. Your ideas here

Readability

All else equal, it’s better for a post to take less energy to read than more. Actually that’s not quite true- for posts that would be especially costly if I’m wrong or I expect to be misinterpreted, I will often bury the conclusion, like I did in this post on binge drinking. But we’ll ignore that for now and focus on the much more common case of wanting posts to be as accessible as possible. 

Detail and readability often trade off against each other, but what I’m looking for here is ways to improve readability while holding detail constant. Some ideas I have:

  1. Formatting, probably? Seems like it should help but I don’t know what specifically.
  2. Humor
    1. Unfortunately the easiest way to do this is to make fun of bad studies, which gets repetitive. 
  3. Explaining relevance to the main question
  4. Make the goal/main question clear
  5. Pictures? I’m unconvinced of this
  6. Your ideas here

Audience

Everyone says to have an audience in mind. There are two major audiences and two minor.

People who are Interested in the Opinions of Uncredentialed Internet Weirdos

This is a tautology, but refers to something much more specific than it looks at first. People who are interested in hearing uncredentialed randos describe and interpret academic papers have a lot more in common than just their willingness to do that. 

Some other traits they share: 

  1. Statistical literacy
  2. Desire for interpretations to be quantified
  3. Higher risk tolerance
  4. Your ideas here
  5. Interested in the specific topic- such as ketamine use, or long covid risk.
    1. It’s rare I want to convince people that they should be in a topic when they weren’t before. 
  6. Your ideas here

Fishing for Corrections

Some posts aren’t meant to be read widely. They’re meant to be a reference in other, more readable post, and to invite corrections from the three people who will read them. This is my intention for the ketamine dosage translation post– it’ll be lucky if it’s read by 10 people when it’s first published, but one of those might be quite useful. 

The primary benefit to me is catching mistakes before I write an entire 10,000 word post with information that could hurt people I’m wrong that depends on the false conclusion. It also feels virtuous to explain my reasoning in detail, even if nothing specifically good comes from it.  

Myself

Writing lets me think through things. I always budget at least as much time for the “writing” phase of a project as research, because there are gaps I don’t notice until I start writing them down. 

I’m interested in how this works for other people- have you found ways to improve your writing for yourself?

Potential clients

I make my living as a freelance researcher, with my blog being the major evidence I am good at this. I’d like clients who read my posts to be able to assess my skill level, even if they’re not interested in the topic and have no context. 

Conclusion, such as it is

I would like to get better at writing the kind of posts I write. In particular, I’d like to get better at conveying relevant information, in ways that take as little work from the reader as possible, but no less than that. I will be very grateful for feedback that helps me improve or that helps me create a framework by which I can improve. I expect that to mostly be critical, but compliments are helpful too- I’d hate to throw out the baby with the bathwater.