Third in a series of short rationality prompts.
My opening rationality move is often "What's my goal?".
It is closely followed by: "Why is this hard? And, what can I do about that?".
If you're busting out deliberate "rationality" tools (instead of running on intuition or copying your neighbors), something about your situation is probably difficult.
It's often useful to explicitly enumerate "What's hard about this?", and list the difficulties accurately, and comprehensively[1], such that if you were to successfully deal with each hard thing, it'd be easy.
Then, you have new subgoals of "figure out how to deal with each of those hard-things." And you can brainstorm solutions[2].
Sometimes, those subgoals will also be hard. Then, the thing to do is ask "okay, what's hard about this subgoal, and how to do I deal with that?"
I'll do one example that's sort of "simple" (most of what I need to do is "try at all"), and one that's more complex (I'll need to do some fairly creative thinking to make progress).
I'm trying to fill out some paperwork. It requires some information I don't know how to get. (Later, it might turn out that the information requires navigating an opaque, badly designed website, and then I have to do some kind of annoying math operation on it)
Also, I didn't get enough sleep and am tired. My eyes are sliding off.
Default outcome: "ugh, this sucks", bouncing off, intermittently trying to work at it and then screw around on Facebook, neither successfully having fun nor completing the form.
What's hard about this? Well:
Trying to deal with both of those at once feels overwhelming. But, now it's more obvious I could split them up and deal with each separately.
#1. I'm tired
Let's start with this, since it'll be a force multiplier on the other one. How might I deal with it? Well, I do some Babbling / Metastrategic Brainstorming, and I come up with:
I try drinking some water. I'm still tired. I try doing a couple jumping jacks. I'm still tired.
Recursion: Turns out taking a nap is Hard.
Take a nap? Ugh, that'll take awhile and I'm kinda shit at napping and I just drank 2 red bulls. I don't expect napping to work.
Hmm. What if I tried just laying down in a dark room for awhile and half-assedly pretend to nap and let my mind wander?
Well, if I did that, I'd end up playing on my phone and not even doing half-assedly napping.
What if... I didn't bring my phone into the dark room?
Okay, you know, that might just work.
I try that. It works.
#2. I don't know where the information is
The silliest part here is how, by default, I'll periodically "start thinking about filling out the form", and try some autopilot-y move like "glance around on the tax form, hoping an answer will jump out at me." And then that fails, and I am mysteriously on facebook.
Writing out "I don't know how to get the information. How do I deal with that?", prompts the embarrassingly obvious answer of "I have to somehow figure out how to get the information" which implies "I need a plan for getting the information."
But, I notice my brain is still sliding off this problem. I try to think about it, but even with my fresh drink of water and/or nap there is something muddled and murky about it.
Recursion: Finding the information is also hard.
After thinking for a bit, I realize two things:
When I direct my attention to the second thing, what do I do about that? Well, some options:
Fortunately, in this case, I've previously learned the skill of buckling up and mentally prepare for multiple hard cognitive steps in a row. It seems like that's the move here. So, I do that.
Okay. That deals with problem B. Back to problem A. I don't know how to find the information. I need to make some kind of actual plan to find it. What are some things that might help?
This section is probably fractally hard, where each option is likely to run into another hard thing. But now that I've buckled up it feels less obnoxious to think multiple steps ahead.
Some ideas:
Which of these ends up helping depends on the exact paperwork I'm dealing with. But, it's kinda embarrassing how often I find myself "not even trying" to figure out how to find the information, and instead just sort of bopping around hoping it magically becomes easy. Once I've articulated the problem, it's usually easier to solve.
So I list out the confusing bits. I turn each confusion into an explicit step. I do the steps.
The hard problem has become kinda easy, if still kinda annoying.
And then I'm done.
The bureaucracy exercise was "easy" in some sense. You know how to fill out a form. You've done it a hundred times. Humans have done it millions of times. The only reason it's "hard" is that it kinda sucks. But if you sit down and carefully think through it, it's not too bad.
Sometimes, the reason something is hard is more like "you just really don't know how to do it", and you will need to find a new way of looking at the problem.
Recently, I vibecoding a fun side project. I wanted to make a website with lots of different versions of sheet music for a given song, and the ability to convert one type of music notation into another. One subthread of that was: can we scan static PDFs of sheet music, and convert them into a file format I could edit?
I asked a Claude Agent to write a conversion script.
It didn't work.
I said "hey Claude, it didn't work."
Claude said "oh, my bad. Let me try <some random ass thing>." Claude tried that random ass thing, and said "okay I fixed it!" It was not fixed.
I said "Please think harder, and make sure you fully understand the problem before declaring yourself done."
It did some minor ritual-of-cognition that looked vaguely like thinking harder, then did more of the same thing.
Geez Claude, you're debugging like me on a bad day, when I randomly flail instead of making a systematic checklist and trying to understand the problem.
If this were a serious project, I would say "ugh, I guess I have to debug for real", and Buckle Up again and start reading documentation and making lists of my confusions and building a model of the situation.
But, I didn't feel like it.
What's hard about this for Claude?
Well:
1. It's working with an external PDF conversion library. Maybe the library is jank. Maybe it needs to switch libraries or solve the problem a whole different way. The hard part here is that the action space is large, and Claude just doesn't have the good taste to navigate that action space.
2. Claude apparently just doesn't have the ability to judge whether it's solved the problem or not, just by looking at the code.
3. Also, Claude can't just look at my website, click the buttons, and see if the PDF gets successfully converted.
If I could solve any of those problems for Claude, it might stop claiming to be done when it clearly wasn't done, and instead keep trying.
Problem #1 does seem like a lost cause. Someday AI will have good judgment to navigate complex domains when it's first patternmatch guess doesn't work. Today is not that day. (And when that day comes, I'm scared).
Problems #2 and #3 both mean that Claude just has no ability to tell if it's done. Maybe I could convince it to stop pretending it was done, but that wouldn't get it done any faster.
I feel really suspicious of #2. Claude seems like it should be capable of reading through a codebase line-by-line, including stepping into the foreign library, and modeling whether each step should work. I've tried a whole bunch to get it to do this reliably, and haven't gotten it working yet. Maybe I'll find a good metacognition prompt someday, but, having put a dozen hours into it, it's not looking promising tonight.
#3?
Why can't Claude look at my website, click buttons, and see if the conversion is working?
There's actually some more recent tools that let it do directly that. I haven't bothered to install those tools because I'm lazy. I could install those tools, but then I'd have to do a few annoying steps in a row to make sure the tools work. This is a fun side project. I don't feel like it.
Hmm. Why does Claude need to navigate a webpage and click a button?
Because currently, the conversion script is built directly into the webpage, which can only be accessed via a browser.
Is there any good reason for that?
No. I could refactor this into a script that can be run individually as a test case, from the command line. The Claude Agent does have the ability to make and run scripts on the command line (if I give the permissions to do so).
"Hey Claude, can you write yourself a script that tests the conversion function, and continue re-running it while debugging until the script reliably works?"
"No problem." Claude does that. A few minutes of brute-force exploration later, it has solved the problem. It turns out, the debugging principle of "first, try to cleanly replicate the problem" is just as important for LLMs as for humans.
(This was only a week ago, but it seems generally useful for getting AIs to keep working until they're actually done. Sometimes it's a bit harder to convert a problem into something the AI can "see", but, I now have in my toolkit "think about how to convert a coding problem into something the AI can see")
When you have a task that feels frustratingly difficult, try asking yourself "what's hard about this?", and then followup with, "what do I do about that?"
Sometimes, this is much easier than these examples, and as soon as you ask "why is this hard?" you'll realize "because I need to think for 2 goddamn seconds in a row and I wasn't bothering to do that", and then you think about it for 2 goddamn seconds in a row and then the solution is obvious.
Sometimes, dealing with the Hard Part is also separately hard, and you need to recursively ask "Okay, why is that hard? What can I do about that?"
What is a difficult thing you're dealing with lately? What's hard about it? How can you deal with that?
(If you're comfortable writing up your answers in the comments, I'd appreciate it)
(Well, for lighter-weight problems, it may be enough just to list the single-hardest thing if it easily occurs to you)
See The Babble Challenge and Metastrategic Brainstorming for some background on training the muscle of "generate approaches to solving problems." (Though note one commenter's reply "sometimes the solution isn't actually to 'brainstorm', it's to go on a walk and let the my mind wander until it figures it out.")