I think resulting AI will still be illegible and this is Thane's point: we can't write legible, conceptually outlined programs such that we can make inferences from source code like "this program can write Python programs in response to requests in natural language". I would expect such GOFAI-Opus 5.5 to be either a mess (weave of sorta locally legible rules which do not provide obvious explanations for observed behavior) or conceptually complicated (AIs would need to invent multiple breakthroughs to get probabilistic programming circa 1999 to the Opus-5.5 level and it would look like totally alien tech without developer-written textbook). Eliezer's market is about whether we find any structures from the second case.