Are you writing an article that’s supposed to change people’s minds and/or actions? For example, an article about career advice, productivity, or getting people to apply for a job? Then this post is for you!
TL;DR: For important articles I recommend doing real user testing (even with just one person!). Try it and you’ll never go back.
I specifically recommend:
(1) Watch them read it, don’t just ask for Google Doc comments
Open a zoom, share screen, and ask them to share their thoughts live as they read.
(2) Show it to your target audience, don’t just ask others what your target audience would think
If your article is supposed to change developers’ career plans, ask a dev who might change their career plan. Don’t ask a biologist what a dev might think.
(3) Check if they changed their mind, not just if they like your writing
If they end with “I’ve got to do this now!”: That's a good sign.
If they say “sounds interesting, I’ll think about it”, that’s not as good.
I personally like to first ask the other person for their opinion (for example, “what’s your current career plan?”), then let them read the article, and then ask again.
The general intuition:
If you want to adopt my frame of mind, ask yourself: Would your idea of how to improve [the article’s persuasiveness] also improve [how engaging a startup’s app] is?
Here are some exercises, do you think these would work?
- Let GPT-3 look at the app (or article) and tell you how engaging the app (or how persuasive the article) is
- Try to guess in advance all the reasons why your target audience won’t agree with the article, or won’t be engaged with the app
Hint: Is the test [checking if a real user from your target audience changes their mind] or [using some proxy to guess whether a real user would change their mind] ?
How to start?
Easiest: post in Bountied Rationality, or in another group that has your target audience.
A draft post you can copy & paste: “Hey, I’ll pay $10 for someone in [describe your target audience here] to review a post of mine for the EA/lesswrong forum. I want to do it live, over a video call. The call is capped at 30 minutes (and expected to be less). Comment or message me if you’re interested. Tagging or referring relevant people is also appreciated.”
Thanks to Matt Brooks and Daniel Reeves for helping make this article better <3