I feel like sometimes I have a hard time keeping track of the experiences that form my intuitive beliefs. Sometimes I want to explain an abstract idea/situation and I would like to bring up some examples... and often I have a hard time of thinking of any? Even though I know the belief was formed by encountering multiple such situations in real life. It would be cool if my brain could list the "top 5 most relevant examples" that influenced a certain intuitive belief, but, in the language of this article, it seems to just throw away the training data after it trained on it.
Case in point: I cannot easily think of a past situation right now where I tried to explain some belief and failed to come up with examples...
Well, today GPT-5-Codex solved it on the 2nd try. (The first version it gave was already conceptually correct, but I guess had some subtle bug. After I told it to fix it and test the fix, it gave a working solution.)
I am just surprised how well the agentic loop is working. It cloned the specific Lean version's source code I was asking for, inspected it to understand the data structure, downloaded a release tarball to test it, all without losing track of its goals. All this would have been unimaginable ~a year ago.
So yeah, in 7 months (but maybe even 2 if you count the base GPT-5 attempt) we went from "not even close" to "solved" on this problem. Not sure how I should feel about this...
I wonder if there are people/groups who (implicitly) do the same with ChatGPT? If the chatbot says something it is considered truth, unless someone explicitly disproves it. (I think I have read stories hinting at this behavior online, and also met people IRL who seemed a bit to eager to take the LLM output at face value.)
How do you learn to replicate bugs, when they happen inconsistently
I don't have definitive advice here, I think this is a hard problem no matter your skill level. You can do things in advance to make your program more debuggable, like better logging, and assertions so you catch the bug closer to the root cause.
A more general pattern to look for is some tool that can capture a particular run of the system in a reproducible/replayable manner. For a single program running locally, a coredump is already quite good, you can look at the whole state of your program just before the crash. (E.g. the whole stack trace, and all variables. This can already tell you a lot.) I have also heard great things about rr, supposedly it allows you to capture a whole execution and single step forwards and backwards.
For distributed systems, like web applications, the problem is even harder. I think I have seen some projects aiming to do the whole "reproducible execution" thing for distributed systems, but I don't know of any that I could recommend. In theory the problem should not be hard, just capture all inputs to the system, and since computers are deterministic, just replay the inputs. But in practice, given the complexity of our software stacks, often determinism is more of a pipe dream.
How does one "read the docs?"
Something something "how to build up a model of the entire stack."
I think these are closely related. I imagine my "model of the entire stack" like a scaffolding with some knowledge holes that can be filled in quickly if needed. You should not have any unknown-unknowns. If I notice that I need more fidelity in some area of my model, that's exactly the docs I read up on.
When reading docs, you can have different intentions. Maybe you are learning about something for the first time, and just want to get an overall understanding. Or maybe you already have the overall understanding, and are just looking for some very specific detail. Often documentation is also written to target one of those use-cases, you should be aware that (well) documented systems often have multiple of these. This is one model I came across that tries to categorize documentation (though I am not sure I subscribe to these exact 4 categories):
Getting back to the "model of the entire stack" thing, I think it's very important for how (I at least) approach computer systems. I think this article by Drew DeVault in particular was an important mindset-shift back when I read it. Some quotes:
Some people will shut down when they’re faced with a problem that requires them to dig into territory that they’re unfamiliar with. [...] Getting around in an unfamiliar repository can be a little intimidating, but do it enough times and it’ll become second nature. [...] written in unfamiliar programming languages or utilize even more unfamiliar libraries, don’t despair. All programming languages have a lot in common and huge numbers of resources are available online. Learning just enough to understand (and fix!) a particular problem is very possible
I now believe that being able to quickly jump into unfamiliar codebases and unfamiliar languages is a very important skill to have developed. This is also important because documentation is often lacking or non-existent, and the code is the "documentation".
Also, I feel like the "model of the entire stack" thing is a phase shift for debugging once you get there. Suddenly, you can be very confident about finding out the root cause of any (reproducible) bug in bounded time.
If at any point you notice that your unfamiliarity with some part of the system is impeding you in solving some problem, that's a sign to study that area in more detail. (I think this is easier to notice when debugging, but can be equally important when building new features. Sometimes your unfamiliarity with a certain area leads you to build a more complex solution than necessary, since you are unable to search paths that route through that area. A map analogy here would be you having a dark spot on your map, and noticing whether it's likely that between two points, there could be a shorter path through the dark area.)
GPT-5 attempt: https://chatgpt.com/share/689526d5-8b28-8013-bcbf-00f76cd37596
It at least hallucinates less, but after recognizing the difficulty of the problem it just gives up, gives a (by its own admission) half-assed solution that cannot work, and goes on to explain why I should be asking for something different, and given my constraints a solution is not practical. (See sibling comment for a very much practical solution I wrote in ~2 hours.)
This 25m 80%-time horizon number seems like strong evidence against the superexponential model from ai-2027. On this graph the superexponential line shows 4h at the end of 2025. I feel like GPT-5 will be the biggest model release of the year, I don't see how we would see a model with an 8x time horizon of GPT-5 this year.
Ok, now this is actually really weird. You’re saying that the only reason you can think of to want to feel empathy toward other people… is to make yourself happier…?
I do most things in my life to try to make myself happier. (And I think this is the revealed preference of most people, even if they don't admit it, because being seen as "selfish" is sometimes socially not tolerated well.)
(But I don't have a fully fleshed out moral philosophy, and I would not feel confident in asserting that "I do everything only to make myself happier". E.g. I do notice myself being nicer to people in general than what would naively be predicted under the "100% selfish" model.)
People do sometimes write LW posts about things that aren’t, like… serious problems in their own personal lives.
I felt like the post was written in a tone describing a personal issue. And if this is not a personal issue for OP and they have some easy way of dealing with it, that seems like a weird thing to omit from the post.
I don’t really know why so many commenters are talking about things like a “psychological need to have people to feel empathy towards”. Where does OP talk about having any such thing…?
That was pure inference on my part. I do think having people to feel empathy towards makes me happier. I felt sad when I was grappling with similar questions years ago as OP, and I had to accept that I won't be able to feel empathy towards most people, I should just model them as part of the external world, and not try to imagine myself in their place.
If OP has some other reason for wanting to empathize with people, I would like to know, because I can't really imagine other important reasons.
1I don't quite understand why this is a large enough problem in your life even to trigger writing a LW post about?
I imagine that you would just curate your group of friends to consist of people you can empathize with, to fulfill your psychological need to have people to feel empathy towards, and just model all other humans as you would model any other part of the world.
Unless I am misunderstanding what you mean by "being able to empathize with someone", and you have a genuine shortage of people who would fulfill your requirements? (In my mind it's not insanely hard to find such people, I think above a certain level of communication and self-reflection ability this is a non-issue.)
Actually, I have been diving into some specific topics lately, and simultaneously formalizing the theorems in Lean to help with understanding. The amount of omissions and handwaving going on in "proofs" in textbooks is insane. (To the point where I am not smart enough to figure out how to fill in some omissions.)
And I know that textbooks often only present a summary of a proof, and cite a more detailed source. But sometimes there is no citation at all, and even in cases where a citation exists, it might not contain the missing details.
Hm... I don't feel like this is what's happening in most cases I encounter? Once I have a detailed pen-and-paper set-theoretic proof, it's mostly straightforward to translate that to Lean's type theory.