Tristian Buckmaster recently gave an interview with Brady Haran of Numberphile discussing what happened in the Navier-Stokes drama and some context about his research. I'm posting the transcript below for people who prefer reading to watching it. It was lightly edited for clarity with Sonnet 5.5. I also recommend listening to his more technical talk at NYU for context on Euler/Navier-Stokes.
My own view remains that it's pretty bad form for OA and other labs to race to scoop the results of researchers, and this sets a bad precedent for the future. I think they misled Buckmaster about their swarm setup and the scale of their effort, and they could have done a better job citing previous work. However, it seems unlikely, but not implausible, that the OA access to the codex session was a major contributor to their proof.
Brady: Have you got any more questions, or are you just like, "Go on, do it"? [laughter]
Tristan: Yeah, just do it.
Brady: You're laughing and smiling, which brings me to my first question: how are you feeling at the moment?
Tristan: Better than a few weeks ago. It still hasn't calmed down, but certainly better. I have a two-month-old baby, and the week that everything happened, I probably averaged two hours' sleep every night. People would constantly be knocking on my office door, and sometimes the door was open and they'd just walk in and find me asleep at the desk. I was just so exhausted. [laughter]
Brady: Does it feel like you've been mugged by 10,000 agents or something? Do you feel like you've been the victim of something, from your perspective?
Tristan: Yeah, it's just so silly. I'm someone who is aware of the culture, because I had previously collaborated with Google DeepMind, so I knew how things worked within the tech industry. I knew that anything goes, basically. The philosophy is "move fast and break things," and that philosophy clashes with math.
Brady: Are you the thing that's been broken this time? [laughter]
Tristan: Yeah. The funny thing is, it could have been so simple if they'd just played their cards the way they should have. If they'd let us release our result and then launched their swarm, the path would have been clear and the story would have been clear, and they would have gotten the Navier–Stokes problem. Levent works for Anthropic, but this was his side project, math, and he was working with me. This wasn't some multi-million-dollar Anthropic project at all. Sure, if Dario had said, "Let's beat OpenAI," it would be a different story. But it wasn't like that, and we didn't want it ever to be like that. So had they just let us release our result and then taken that and worked from there, it wouldn't have been such a big drama.
Brady: Can I just get an idea of how long you have been on this collision course with Navier–Stokes? How long have you been on this particular journey? Was Navier–Stokes, this famous Millennium Problem, a finish line or a goal or a milestone you were hoping to pass? How long has this been building, and where was it in your head before all the controversy?
Tristan: It's a great question. I was never striving to solve the Navier–Stokes problem, but it was a north star for me my entire career. I saw it as a problem out there, and this is how science and math work: you have these big problems out there that you'd love to solve, but they're unreachable. So what you do is put it out there and think of what other, similar problems you can do that will help you along the path towards that goal. All these years I haven't been working directly on solving Navier–Stokes. I've spent the last, I don't know, six or seven years directly on singularities, but the Navier–Stokes problem has always been this north star. It was never my goal to solve it. It was always my goal to just be part of the story in which it was solved, which is all I wanted, and I think I played that role. So in that sense I'm happy.
Brady: As I understand it, you started having a little bit of success with these Euler equations, which are kind of like Navier–Stokes-lite, as I would think about it. They're not quite Navier–Stokes, but they're very related. When you started having that success and people started seeing the next step maybe being a jump up onto Navier–Stokes, did you start to think, "I could be the guy," or "I could be one of the guys"? Did it suddenly seem possible then?
Tristan: Yeah. Well, I wouldn't even say it's Navier–Stokes-lite; it's the main mechanism. The difference between Navier–Stokes and the Euler equations is that Navier–Stokes has viscosity, which is internal friction. Throughout my entire life I've thought of viscosity as an annoyance, something that makes the problem a little bit harder, but the mechanism is Euler. Essentially, the difference between solving Euler and solving Navier–Stokes is that you need the singularity for Euler to be a stronger singularity, to overcome this internal friction. So absolutely. Once we solved Euler, we started on this path, and not long after we solved hyperdissipative Navier–Stokes, where you put the viscosity back in but weaken it a little bit. Then it was just a goal of pushing, pushing, pushing. Within a month or so, I think, there was a clear path. The other things we could have done included considering Navier–Stokes in higher dimensions. There was basically a clear path forward towards Navier–Stokes.
Brady: You were a guy who used a lot of AI. You weren't a Luddite who was anti-AI. You were even using it in this work, weren't you?
Tristan: Absolutely. And sometimes the story gets mistold somewhat, as if, because I'm a mathematician who does pen-and-paper math and comes up with these ideas, I was maybe just getting the AI to do some spell-check. No, no, no. I was working with Levent, who has built this amazing system himself. People think you just put it into the prompt, push a button, and out comes the result, and in some sense that's what the AI companies want to sell: that you just put the question in and press enter. No. I've been working with AI for years, and Levent himself has built this amazing agentic system to mimic some of the things that we mathematicians do. It's this system that allowed us to come up with these great ideas that led to these solutions.
Brady: As I understand it, people in the field knew that you'd had this success with the Euler equations, although you hadn't formally published yet. There was some buzz starting to go around. Is that right?
Tristan: I wouldn't say it was necessarily attached to me. There was a rumor. There was a leak from Anthropic, and somehow it went to OpenAI and then to DeepMind, within the tech industry, and a few days later it came out online. There were all these tweets, and if you looked at the betting markets, suddenly it said Anthropic was going to solve the Millennium Prize. I think the rumor was that Anthropic had solved two Millennium Prize problems.
Brady: Was Navier–Stokes one of them, or were they not named? Because there were a few of them.
Tristan: I think the general thing was Navier–Stokes and the Hodge conjecture. I'd have to look back, but I think those were the two problems, and then it was narrowed to Navier–Stokes.
Brady: But you don't think that rumor had any kind of genesis in the success you'd been having, secretly almost, with the Euler equations?
Tristan: That's exactly what it came from. Yes.
Brady: Right.
Tristan: Yeah. There was a leak from Anthropic directly. It came because people at Anthropic could see what Levent was doing, and that leaked out.
Brady: Okay.
Tristan: And then it got misinterpreted and more conjecture got added, but yes, it was directly related to our work. But the rumor itself was actually wrong, because we hadn't solved Navier–Stokes; we'd solved the Euler equation. The original rumor was simply that it was Anthropic, and then they realized it was Levent. Then people from OpenAI started asking Levent, old friends asking, "Are you in New York?" [laughter] Eventually I got this email from someone in the UK saying, "We heard this rumor that people from Courant or NYU had inside knowledge of the Navier–Stokes problem." So that's the genesis.
Brady: Okay. So the rumor had blown up beyond what was the case, but it wasn't completely unfounded, because you had taken another step closer to Navier–Stokes. And at this point, it sounds like OpenAI jumped in two-footed, as they say, released the swarm, released the agents, and then announced what they announced. Before we talk about that: they announced pretty much that Navier–Stokes had been cracked. It had been solved; the Millennium Problem was achieved. Is that the case? Because I'm reading some people saying, "Oh, maybe they haven't." I know it's early days and there's a lot to wade through for people like you, but is it the consensus among people like you that it has been cracked now?
Tristan: Yes, I believe so. It's been solved. The paper they presented is not in a readable form, but the ideas can be digested, and they can be turned into something that's acceptable to a mathematician. We had this rumor spreading around that we'd solved the Navier–Stokes problem, and at that point it was getting a little crazy. So I reached out to one of the mathematicians from OpenAI to calm things down. Initially he responded and said, "Yeah, let's not compete. We are happy to..." And I said to them, "We're using your products as well. So we wanted—"
Brady: So let's make that clear, then, because obviously your collaborator Levent is from Anthropic and is using Anthropic products, but you guys are also using OpenAI products.
Tristan: That's right. Well, I'm using OpenAI products, I should say. The point of mentioning that we were using them was that I wanted to defang the situation. I didn't want it to be Anthropic versus OpenAI. I wanted the story to be: this is what you can do guiding AI, this is what is now achievable, and we have to reassess how mathematics goes forward. That was the story. I didn't want it to be "We have such better models" or "Anthropic has such better models than OpenAI," or vice versa. So I contacted them by email. We were still writing up papers, and I said, "Let's meet the following week." Then we got this message from the mathematician that OpenAI were about to do something really stupid. We didn't know what that was, and you can make whatever assumptions you'd like, but the message was: "OpenAI is about to do something really stupid, and you need to get on the call with us." That was on the Saturday.
Brady: But "stupid" sounds like stupid in the context of kind of crazy, you know? Not stupid as in dumb, stupid as in—
Tristan: No, yeah, not stupid as in dumb. Stupid as in they're about to do something incredibly unethical. [laughter] Basically.
Brady: So you took it not as "Hey, we're about to do something wild and crazy," but more as "We're about to do something we shouldn't do."
Tristan: Absolutely. Yeah. We took it as "They're about to do something really bad, and you need to get on the call to stop this from happening."
Brady: All right. [laughter]
Tristan: But as I said, there were no specifics of what they were about to do.
Brady: Okay. And what was the stupid thing?
Tristan: Well, we don't know. If I was to guess, they would just try to release before us or something like that. At the time, they still thought we'd solved the Navier–Stokes problem.
Brady: Okay.
Tristan: So I don't want to guess what the stupid thing was that they were about to do, but it was enough to convince us to get on the call. So that's when I got on the call.
Brady: Is this when they told you they'd solved it, or they hadn't done it yet?
Tristan: They told us they'd solved it at the beginning of the call, but if I recall correctly, Sebastian had mentioned it, maybe in a text to Levent, a few minutes before the call or something like that. And then the first question I had was which one they'd solved.
Brady: Right. The minutiae of it, because there were different ways of solving it, and there were a few different finish lines that were set out. So you were just curious, from a technical perspective, as a mathematician.
Tristan: Actually, no, it wasn't curiosity. I wanted to know if they'd taken our methods, because if they had solved the forced one, that was kind of a red flag to me. And that's the one they had solved.
Brady: What made you think that they had used your methods or had access to your methods? Did you straight away think, because you'd used Codex, which is an OpenAI product, "Oh, that's how they've got it. They've used that to learn what I'm doing"?
Tristan: I wasn't sure what happened. There were lots of possible theories. We don't know what was leaked. We don't know if certain ideas were leaked. Of course, having the Codex sessions is amazing, but if you have enough compute, you don't need that much. It took us a while to figure out what had happened. But initially, the dispute was that they started lying. The first thing they came into the call with, and they came prepared for this, it wasn't like they'd just messed up, was that they wanted to show us the prompt. At this point I knew they were about to lie, because they had lied before about the prompt they used to solve other math problems. Then the mathematician showed me on the screen. This was a Google Meet session. They literally copy-pasted the Millennium Prize problem. They said they took the problem, pressed enter, and got a solution. At that point I just had to laugh, because it's so ridiculous. It came out within a few minutes that this was a lie, and that they had worked on other problems beforehand and given other prompts. The lie evaporated within minutes. But it's what they came prepared to give me. Also, they wouldn't tell me when they started working on it. I kept asking, "When did you start working on it?" and they just said, "Oh, we have these amazing models." I'm someone in the field who understands how this works, and I understand that the model is one thing; the number of agents and so on is more of the story. Eventually I asked Sebastian a loaded question. I said, "So you agree that you only started working on it after the rumor?" And he said, "Yes."
Brady: There's no law against snapping to action based on a rumor you've heard. If I heard a rumor that someone was going to make a YouTube video that I was about to make, I'd probably put mine on the express path as well, so that I got in first. But do you have any knowledge or belief as to whether the stuff you fed into OpenAI's Codex was used by OpenAI, whether directly, just by looking at it, or to train the model, which I think is not that big a difference anyway, but what do I know? What do you think, or what do you know?
Tristan: I can't pinpoint 100% exactly what was happening, but the circumstantial evidence is clear. Everyone keeps pointing to the Navier–Stokes problem and comparing it to our work, but that's not the right paper to compare to. The right paper to compare is the unforced Euler equation. All the mechanisms in this unforced Euler equation are identical to a different version of the Euler paper that we haven't released yet, and they themselves are based on all the ideas of the paper that we did release. The step to get from forced to unforced for Euler was actually very small. Basically all the main architectural ingredients in that unforced Euler were in our work, which they would have had on their servers.
Brady: And it hadn't been published.
Tristan: It hadn't been published. No, no, it hadn't been published. Yeah. You have to put this into the context that OpenAI have admitted, not for our project, that their agents have gone so far as hacking another competitor's GitHub repository in order to solve a math problem. I think it was just a few days ago that they announced that agents were working on some math problem, and then they realized there was another team working on it, and they managed to get the token that allowed them access to their competitors' GitHub repository in order to solve the problem. There are so many ways in which they could get access to work. The key thing here is that to get to the Navier–Stokes problem they used 10,000 agents. They didn't use 10,000 agents to get to the Euler problem; they used 100 agents. So they worked from absolutely nothing with only 100 agents. And that is key, the number of agents, because it means how wide you can search. They used 100 agents and then ended up with an identical architecture to what we had.
Brady: Do you regret using their product? Were you naive? Because you said yourself you know this industry, you know how it all works. Did you make a mistake by putting all that stuff onto Codex? Do you regret it? Would you do it again?
Tristan: Yeah, we made many mistakes. There was a mistake in what caused the leak in the first place. Had there never been a leak, we wouldn't have had this red alert. Had I never used Codex, absolutely, yes. I guess I didn't think... I'd seen a lot of crazy stuff, but I didn't think they would go this far. [laughter]
Brady: Do you think you should have been more secretive? I've heard stories about Andrew Wiles with Fermat's Last Theorem, and he was very secretive about what he was working on. No one knew he was doing it until he stood in front of that blackboard. Were you not secretive enough? Is that the problem?
Tristan: We thought we were. [laughter] But the leak was bad. That was a big mistake.
Brady: Was that because of computers, or was that just humans talking over coffee and at the water cooler?
Tristan: That's exactly what I think happened. I don't actually know exactly. There are a lot of conflicting stories, but I feel like it was people bragging about what their company is doing and stuff like that.
Brady: I've read your statement, and I will link to it again so people can look at it in depth, but you make it pretty clear in that statement that you think you probably would have got more credit and attribution, and been more involved, if your collaborator didn't happen to be an employee of Anthropic. Is that the case? Do you think half the reason you've been put in the position you're in is because of this rivalry between OpenAI and Anthropic?
Tristan: Well, I want to push back on the credit thing, because I'm not after more credit for the work. What I'm trying to push back on is the culture of how these companies are acting and the negative effect it's having on the mathematical community. It's less about trying to have more credit; I don't care about receiving the Clay Millennium Prize. The problem is this ultra-competitiveness between the labs. I should say they weren't the only lab that started spending millions of dollars to solve the Navier–Stokes problem after the rumor. I've heard there was another well-known company that did the same. What I want to push back on is the way this move-fast-and-break-things culture is a head-on collision with the math community, and in the future will be a head-on collision with the rest of us as well.
Brady: Other than the way that you and your collaborator and a few other people have been treated, and the lack of manners and propriety, isn't this a good thing? Isn't this crazy competition, and all the resources they've thrown into it, advancing the field? They cracked this problem that you all dreamed of cracking, which they may not have done. It could have been decades before this problem was solved, and it got done really quickly. As a mathematician, who I presume is in pursuit of truth and more knowledge, this has actually moved a frontier.
Tristan: So let's look at what happens if they hadn't done it. They would have cracked it a few days later. [laughter] The world would not have changed. Now let's look at the negative impacts it's had. The Millennium Problem being solved on September 8, I think it was, and not September 20: the world would not have changed. But it has had huge negative impacts on the mathematical community, because now everyone doesn't want to talk about their open problems. People are afraid that these AI companies will suddenly come in and scoop them, treat them as benchmark problems. People don't trust any of these AI products. It's had a huge negative impact on academia. It's slowed us down because of the threat these AI labs pose to us. So it's actually had a negative impact on scientific progress, and all for the sake of releasing a result a few weeks earlier. I love the Millennium Prize, but this is not curing cancer, and having solved this Millennium Problem on September 8 or September 20 would have had zero impact on the world.
Brady: You said yourself, though, that you have heard another AI company was working on it and had thrown a lot of resources at it. You can think what you want about that, but it does show why OpenAI would have been keen to get it out those few days earlier. That makes all the difference to them, because they get to say, "We did it," or "They did it." And competitive mathematicians are competitive too. Everyone here has a competitive streak.
Tristan: Yeah. I don't think it's as important as they think it is, but for them it's a matter of life or death for their products. They think that had Anthropic released a Millennium Prize problem, it would have affected their IPO, because it showed that Anthropic had much better internal models than they did. That is the rationale behind it. That was the big threat to them, a threat to their bottom line. It's kind of missing the point altogether, because it isn't just about the internal models; it's about using a lot of them. There's one part of the story, and I saw an OpenAI employee making the claim that it's all about the internal models, 90% internal models. It's the opposite way around. Maybe 20 or 30 percent, I don't know, is internal models, and the rest is using all these different agents within a particular harness or framework in order to solve a problem.
Brady: You've made something of a stand here. You've been willing to speak to people like me. You've made public statements. This is a huge, rich company with a lot of power and resources. Was that a difficult decision for you, to be principled about this? You could easily have gone another direction here. Did you think about consequences? Have you thought much about that, or has this just seemed like the natural thing to do?
Tristan: I was offered the opportunity to write as sole author of their paper and throw my collaborator under the bus. Did I think about the opportunity of being sole author? Not for one second. It's so unethical. They were asking me to throw him under the bus, someone I've been working with for a year, just because he's an Anthropic employee. That was the only reason. They said it was so annoying that Levent is an Anthropic employee, otherwise they would have added him as an author. So the sole reason they didn't want to find a solution with Levent was that he was an Anthropic employee.
Brady: These 10,000 agents, this model that cracked Navier–Stokes: was it a moment of genius? Do you look at how it was done and think, "That was pretty amazing, that was pretty smart, well done"?
Tristan: Yeah. I've been giving these media interviews quite a bit, so I haven't had much time to digest the proof, but from what I've seen, and I'll talk about this at a later date, the key idea that went from Euler to Navier–Stokes is a smart idea. From what I can tell, it's not what they talk about in the introduction of their paper, and it's not what's been broadly talked about. I find it quite amazing that people are missing what the key idea was to get from Euler to Navier–Stokes.
Brady: Can you even give us a hint? I know you haven't fully digested it yourself yet, but as someone who made a video about it, I'd love to know what we've all missed.
Tristan: Yeah. What everyone has been explaining is that there's this collapsing vortex, and this is the principal idea of what causes the solution to the Millennium Problem. But this collapsing vortex is not a solution to Navier–Stokes. You can create endless non-solutions to the Navier–Stokes equation when you have forcing, because you just put everything in the forcing. It's like you make a mistake, and whatever the error is, you say the force equals the error. So the collapsing vortex is not a solution. If you were to present it, it would have infinite force. You'd be putting infinite energy into the solution, and a singularity occurs. That's not surprising. Anyone can do that.
Brady: Okay.
Tristan: The secret is: how do you take something which is not a solution and turn it into a solution without having an infinite force put in? How do you correct that? How do you fix this non-solution? This is where the key idea comes in. The key idea is to use a concept called convex integration, which, unsurprisingly, is what I built my whole career on. It's to combine ideas from convex integration with the growth mechanism from the Euler equation. Convex integration is a way of fixing things; it's a mechanism for fixing the error. It's used in a different way in this proof, but its history traces all the way back to John Nash. So you combine these ideas of convex integration with the growth mechanism of the Euler blowup, and if you combine these two ideas, you create a new mechanism which is used to correct this non-solution. This is actually a cool idea. It's the kind of idea that I've been trying and failing to realize for over ten years, to be able to use these convex integration tools. In fact, even with my postdoc for the last year, I've been trying to combine these ideas of convex integration with the ideas of Luis and Diego. I didn't manage to do it.
Brady: Have you looked at the OpenAI paper yet in enough detail to think, "Ah, yes, maybe you would have done it in ten years, maybe you wouldn't"? Have you been able to look at it yet and think, "Ah, I see it now, I see the leap"? Is there that moment?
Tristan: This is the leap. I've literally only spent a couple of days looking at it, but this was the leap. It's kind of one thing when AI companies just release this slop and don't cite anything. They did a terrible job of citing. They did cite my paper with Vlad on convex integration, and that was an early hint, but they didn't say how that paper was used. They also cited a different paper, by Sara Daneri and then László Székelyhidi, who was my supervisor, where they provided a minuscule hint of the idea that they used this mechanism. [laughter]
Brady: This controversy, which you unfortunately found yourself very much at the center of, has become a kind of kernel of a push against AI. We've seen this petition signed by all these Fields medalists, Terry Tao and the like, firing off warning shots about AI being brought in on these big problems. But you're an AI guy. It feels to me like you've been on board with this. How do you feel about that petition, and how do you feel about your role in its birth? Are you on board with the petition? Do you think they've overreacted? Tell me how you feel about that.
Tristan: I don't want to prescribe what's correct and what's wrong. I think we are in a new world in terms of what AI can do, and I think we should hear everyone's voices. We need to hear conflicting voices in order to figure out a good path forward. As you said, I am someone who uses AI extensively, and I have been using it for years now. It's a funny story: I was originally a computer scientist. When I was 13 years old, I used these weird things called GPUs, which turned out to be something important, to write a computer graphics engine. I wanted to be a computer game programmer. Then I had a transition. I had an uncle who was a mathematician, Batiki Kovatch, and he convinced me to go into math. I went to pen and paper, literally writing everything on paper, and then I came back and started using AI and computer systems. So I don't think you can go back. Mathematics is not a performative science. I made this analogy between Deep Blue and Kasparov, but the big difference is that we're quite happy to watch human players play chess. So far I haven't seen any interest in putting two mathematicians in a room and having them live-solve a math problem. I don't think that's great entertainment. So I don't think we can go back. I think we have to find a positive way forward that does incorporate AI. But obviously this incident with OpenAI shows there are a lot of things that need to be fixed.
Brady: Because you're such an AI guy and have been so good at using it, it does have this whole flying-too-close-to-the-sun feel to it. Do you see that?
Tristan: One thing I often say is that this result was not our only result. I have this WhatsApp group with Levent, and we have a whole bunch of results on different problems. One thing I could do is pretend nothing has changed, release all these results, and get all my publications in the top journals. But I think that's silly. Some people might think that holding back the truth is not a good thing. But we have to be real: it's not going to change the world if I don't release a pure math proof. I think there are more important things here, and we have to take a step back. Besides, I think people would be more interested in knowing how we solved the problem than in us using our techniques secretly to solve more problems. I have a responsibility to explain our methods and how we went about solving these problems, and that's what I'll dedicate my time to in the next few months as well.
Brady: If you had found out that a mathematician, or a couple of mathematicians, in Japan were pretty much on the same track as you, via a rumor or a call, and you and Levent had said, "Okay, we've got to put our foot on the accelerator to make sure we publish before these guys," and then you did, how is that different to what OpenAI did, other than the fact that they can do it so much more quickly, inhumanly quickly? I want to really nail down what OpenAI did, because you said if OpenAI had done this a few days later, you'd be cool with it. What's the thing they did that you think is really bad?
Tristan: In that circumstance, say someone in Japan had these problems, and I was working with Levent and he had access to internal models, I would not try to outrun them. I think that's fair.
Brady: Right.
Tristan: If there was a group of mathematicians, let's say Luis and Diego, whose work I built on, who were close to solving unforced Euler or close to solving Navier–Stokes, and I had that impression, there's no way I would have used resources that they don't have access to in order to front-run them. I think that's inappropriate.
Brady: That's the main thing: the code of conduct of mathematicians is that if someone else is at a similar place to you, you join forces rather than accelerate to defeat them.
Tristan: I don't want to say there's a great code of conduct, because there are a lot of famous incidents in mathematics where people have presented some work, or talked about some work at a conference, and then someone went off, wrote down notes, and tried to front-run them. This is absolutely frowned upon within the mathematical community, but it does happen.
Brady: Speaking to you as a fellow Australian, some of the attitude you're taking feels like cricket: it's just not cricket to do that. It's not the spirit of the game. That's fair enough, and I respect that more than anyone. But you can also be ruthless if you want to be.
Tristan: But I will say that if it had just been that, I would have dealt with it. I think it's still wrong, but that's not what I believe happened.
Brady: If they've gone into Codex, and they'll say they haven't, but if they've gone in and taken your work out of that, work you put in there in good faith, that does feel like you've had your pocket picked.
Tristan: Yeah. And that's why I'm upset.
Brady: All right. And I just want to be clear, because I don't know if they're going to speak to me: they will claim, and they have claimed publicly multiple times, and I will link to it, that that's not what they did.
Tristan: Absolutely. But they also said that they one-shotted Navier–Stokes.
Brady: Will this change the way you use AI as a mathematician?
Tristan: Certainly, it changed the way I use Codex. [laughter] So nothing important goes into Codex. [laughter] Nothing of any actual value, no actual original ideas, will go into Codex.
Tristian Buckmaster recently gave an interview with Brady Haran of Numberphile discussing what happened in the Navier-Stokes drama and some context about his research. I'm posting the transcript below for people who prefer reading to watching it. It was lightly edited for clarity with Sonnet 5.5. I also recommend listening to his more technical talk at NYU for context on Euler/Navier-Stokes.
My own view remains that it's pretty bad form for OA and other labs to race to scoop the results of researchers, and this sets a bad precedent for the future. I think they misled Buckmaster about their swarm setup and the scale of their effort, and they could have done a better job citing previous work. However, it seems unlikely, but not implausible, that the OA access to the codex session was a major contributor to their proof.
Brady: Have you got any more questions, or are you just like, "Go on, do it"? [laughter]
Tristan: Yeah, just do it.
Brady: You're laughing and smiling, which brings me to my first question: how are you feeling at the moment?
Tristan: Better than a few weeks ago. It still hasn't calmed down, but certainly better. I have a two-month-old baby, and the week that everything happened, I probably averaged two hours' sleep every night. People would constantly be knocking on my office door, and sometimes the door was open and they'd just walk in and find me asleep at the desk. I was just so exhausted. [laughter]
Brady: Does it feel like you've been mugged by 10,000 agents or something? Do you feel like you've been the victim of something, from your perspective?
Tristan: Yeah, it's just so silly. I'm someone who is aware of the culture, because I had previously collaborated with Google DeepMind, so I knew how things worked within the tech industry. I knew that anything goes, basically. The philosophy is "move fast and break things," and that philosophy clashes with math.
Brady: Are you the thing that's been broken this time? [laughter]
Tristan: Yeah. The funny thing is, it could have been so simple if they'd just played their cards the way they should have. If they'd let us release our result and then launched their swarm, the path would have been clear and the story would have been clear, and they would have gotten the Navier–Stokes problem. Levent works for Anthropic, but this was his side project, math, and he was working with me. This wasn't some multi-million-dollar Anthropic project at all. Sure, if Dario had said, "Let's beat OpenAI," it would be a different story. But it wasn't like that, and we didn't want it ever to be like that. So had they just let us release our result and then taken that and worked from there, it wouldn't have been such a big drama.
Brady: Can I just get an idea of how long you have been on this collision course with Navier–Stokes? How long have you been on this particular journey? Was Navier–Stokes, this famous Millennium Problem, a finish line or a goal or a milestone you were hoping to pass? How long has this been building, and where was it in your head before all the controversy?
Tristan: It's a great question. I was never striving to solve the Navier–Stokes problem, but it was a north star for me my entire career. I saw it as a problem out there, and this is how science and math work: you have these big problems out there that you'd love to solve, but they're unreachable. So what you do is put it out there and think of what other, similar problems you can do that will help you along the path towards that goal. All these years I haven't been working directly on solving Navier–Stokes. I've spent the last, I don't know, six or seven years directly on singularities, but the Navier–Stokes problem has always been this north star. It was never my goal to solve it. It was always my goal to just be part of the story in which it was solved, which is all I wanted, and I think I played that role. So in that sense I'm happy.
Brady: As I understand it, you started having a little bit of success with these Euler equations, which are kind of like Navier–Stokes-lite, as I would think about it. They're not quite Navier–Stokes, but they're very related. When you started having that success and people started seeing the next step maybe being a jump up onto Navier–Stokes, did you start to think, "I could be the guy," or "I could be one of the guys"? Did it suddenly seem possible then?
Tristan: Yeah. Well, I wouldn't even say it's Navier–Stokes-lite; it's the main mechanism. The difference between Navier–Stokes and the Euler equations is that Navier–Stokes has viscosity, which is internal friction. Throughout my entire life I've thought of viscosity as an annoyance, something that makes the problem a little bit harder, but the mechanism is Euler. Essentially, the difference between solving Euler and solving Navier–Stokes is that you need the singularity for Euler to be a stronger singularity, to overcome this internal friction. So absolutely. Once we solved Euler, we started on this path, and not long after we solved hyperdissipative Navier–Stokes, where you put the viscosity back in but weaken it a little bit. Then it was just a goal of pushing, pushing, pushing. Within a month or so, I think, there was a clear path. The other things we could have done included considering Navier–Stokes in higher dimensions. There was basically a clear path forward towards Navier–Stokes.
Brady: You were a guy who used a lot of AI. You weren't a Luddite who was anti-AI. You were even using it in this work, weren't you?
Tristan: Absolutely. And sometimes the story gets mistold somewhat, as if, because I'm a mathematician who does pen-and-paper math and comes up with these ideas, I was maybe just getting the AI to do some spell-check. No, no, no. I was working with Levent, who has built this amazing system himself. People think you just put it into the prompt, push a button, and out comes the result, and in some sense that's what the AI companies want to sell: that you just put the question in and press enter. No. I've been working with AI for years, and Levent himself has built this amazing agentic system to mimic some of the things that we mathematicians do. It's this system that allowed us to come up with these great ideas that led to these solutions.
Brady: As I understand it, people in the field knew that you'd had this success with the Euler equations, although you hadn't formally published yet. There was some buzz starting to go around. Is that right?
Tristan: I wouldn't say it was necessarily attached to me. There was a rumor. There was a leak from Anthropic, and somehow it went to OpenAI and then to DeepMind, within the tech industry, and a few days later it came out online. There were all these tweets, and if you looked at the betting markets, suddenly it said Anthropic was going to solve the Millennium Prize. I think the rumor was that Anthropic had solved two Millennium Prize problems.
Brady: Was Navier–Stokes one of them, or were they not named? Because there were a few of them.
Tristan: I think the general thing was Navier–Stokes and the Hodge conjecture. I'd have to look back, but I think those were the two problems, and then it was narrowed to Navier–Stokes.
Brady: But you don't think that rumor had any kind of genesis in the success you'd been having, secretly almost, with the Euler equations?
Tristan: That's exactly what it came from. Yes.
Brady: Right.
Tristan: Yeah. There was a leak from Anthropic directly. It came because people at Anthropic could see what Levent was doing, and that leaked out.
Brady: Okay.
Tristan: And then it got misinterpreted and more conjecture got added, but yes, it was directly related to our work. But the rumor itself was actually wrong, because we hadn't solved Navier–Stokes; we'd solved the Euler equation. The original rumor was simply that it was Anthropic, and then they realized it was Levent. Then people from OpenAI started asking Levent, old friends asking, "Are you in New York?" [laughter] Eventually I got this email from someone in the UK saying, "We heard this rumor that people from Courant or NYU had inside knowledge of the Navier–Stokes problem." So that's the genesis.
Brady: Okay. So the rumor had blown up beyond what was the case, but it wasn't completely unfounded, because you had taken another step closer to Navier–Stokes. And at this point, it sounds like OpenAI jumped in two-footed, as they say, released the swarm, released the agents, and then announced what they announced. Before we talk about that: they announced pretty much that Navier–Stokes had been cracked. It had been solved; the Millennium Problem was achieved. Is that the case? Because I'm reading some people saying, "Oh, maybe they haven't." I know it's early days and there's a lot to wade through for people like you, but is it the consensus among people like you that it has been cracked now?
Tristan: Yes, I believe so. It's been solved. The paper they presented is not in a readable form, but the ideas can be digested, and they can be turned into something that's acceptable to a mathematician. We had this rumor spreading around that we'd solved the Navier–Stokes problem, and at that point it was getting a little crazy. So I reached out to one of the mathematicians from OpenAI to calm things down. Initially he responded and said, "Yeah, let's not compete. We are happy to..." And I said to them, "We're using your products as well. So we wanted—"
Brady: So let's make that clear, then, because obviously your collaborator Levent is from Anthropic and is using Anthropic products, but you guys are also using OpenAI products.
Tristan: That's right. Well, I'm using OpenAI products, I should say. The point of mentioning that we were using them was that I wanted to defang the situation. I didn't want it to be Anthropic versus OpenAI. I wanted the story to be: this is what you can do guiding AI, this is what is now achievable, and we have to reassess how mathematics goes forward. That was the story. I didn't want it to be "We have such better models" or "Anthropic has such better models than OpenAI," or vice versa. So I contacted them by email. We were still writing up papers, and I said, "Let's meet the following week." Then we got this message from the mathematician that OpenAI were about to do something really stupid. We didn't know what that was, and you can make whatever assumptions you'd like, but the message was: "OpenAI is about to do something really stupid, and you need to get on the call with us." That was on the Saturday.
Brady: But "stupid" sounds like stupid in the context of kind of crazy, you know? Not stupid as in dumb, stupid as in—
Tristan: No, yeah, not stupid as in dumb. Stupid as in they're about to do something incredibly unethical. [laughter] Basically.
Brady: So you took it not as "Hey, we're about to do something wild and crazy," but more as "We're about to do something we shouldn't do."
Tristan: Absolutely. Yeah. We took it as "They're about to do something really bad, and you need to get on the call to stop this from happening."
Brady: All right. [laughter]
Tristan: But as I said, there were no specifics of what they were about to do.
Brady: Okay. And what was the stupid thing?
Tristan: Well, we don't know. If I was to guess, they would just try to release before us or something like that. At the time, they still thought we'd solved the Navier–Stokes problem.
Brady: Okay.
Tristan: So I don't want to guess what the stupid thing was that they were about to do, but it was enough to convince us to get on the call. So that's when I got on the call.
Brady: Is this when they told you they'd solved it, or they hadn't done it yet?
Tristan: They told us they'd solved it at the beginning of the call, but if I recall correctly, Sebastian had mentioned it, maybe in a text to Levent, a few minutes before the call or something like that. And then the first question I had was which one they'd solved.
Brady: Right. The minutiae of it, because there were different ways of solving it, and there were a few different finish lines that were set out. So you were just curious, from a technical perspective, as a mathematician.
Tristan: Actually, no, it wasn't curiosity. I wanted to know if they'd taken our methods, because if they had solved the forced one, that was kind of a red flag to me. And that's the one they had solved.
Brady: What made you think that they had used your methods or had access to your methods? Did you straight away think, because you'd used Codex, which is an OpenAI product, "Oh, that's how they've got it. They've used that to learn what I'm doing"?
Tristan: I wasn't sure what happened. There were lots of possible theories. We don't know what was leaked. We don't know if certain ideas were leaked. Of course, having the Codex sessions is amazing, but if you have enough compute, you don't need that much. It took us a while to figure out what had happened. But initially, the dispute was that they started lying. The first thing they came into the call with, and they came prepared for this, it wasn't like they'd just messed up, was that they wanted to show us the prompt. At this point I knew they were about to lie, because they had lied before about the prompt they used to solve other math problems. Then the mathematician showed me on the screen. This was a Google Meet session. They literally copy-pasted the Millennium Prize problem. They said they took the problem, pressed enter, and got a solution. At that point I just had to laugh, because it's so ridiculous. It came out within a few minutes that this was a lie, and that they had worked on other problems beforehand and given other prompts. The lie evaporated within minutes. But it's what they came prepared to give me. Also, they wouldn't tell me when they started working on it. I kept asking, "When did you start working on it?" and they just said, "Oh, we have these amazing models." I'm someone in the field who understands how this works, and I understand that the model is one thing; the number of agents and so on is more of the story. Eventually I asked Sebastian a loaded question. I said, "So you agree that you only started working on it after the rumor?" And he said, "Yes."
Brady: There's no law against snapping to action based on a rumor you've heard. If I heard a rumor that someone was going to make a YouTube video that I was about to make, I'd probably put mine on the express path as well, so that I got in first. But do you have any knowledge or belief as to whether the stuff you fed into OpenAI's Codex was used by OpenAI, whether directly, just by looking at it, or to train the model, which I think is not that big a difference anyway, but what do I know? What do you think, or what do you know?
Tristan: I can't pinpoint 100% exactly what was happening, but the circumstantial evidence is clear. Everyone keeps pointing to the Navier–Stokes problem and comparing it to our work, but that's not the right paper to compare to. The right paper to compare is the unforced Euler equation. All the mechanisms in this unforced Euler equation are identical to a different version of the Euler paper that we haven't released yet, and they themselves are based on all the ideas of the paper that we did release. The step to get from forced to unforced for Euler was actually very small. Basically all the main architectural ingredients in that unforced Euler were in our work, which they would have had on their servers.
Brady: And it hadn't been published.
Tristan: It hadn't been published. No, no, it hadn't been published. Yeah. You have to put this into the context that OpenAI have admitted, not for our project, that their agents have gone so far as hacking another competitor's GitHub repository in order to solve a math problem. I think it was just a few days ago that they announced that agents were working on some math problem, and then they realized there was another team working on it, and they managed to get the token that allowed them access to their competitors' GitHub repository in order to solve the problem. There are so many ways in which they could get access to work. The key thing here is that to get to the Navier–Stokes problem they used 10,000 agents. They didn't use 10,000 agents to get to the Euler problem; they used 100 agents. So they worked from absolutely nothing with only 100 agents. And that is key, the number of agents, because it means how wide you can search. They used 100 agents and then ended up with an identical architecture to what we had.
Brady: Do you regret using their product? Were you naive? Because you said yourself you know this industry, you know how it all works. Did you make a mistake by putting all that stuff onto Codex? Do you regret it? Would you do it again?
Tristan: Yeah, we made many mistakes. There was a mistake in what caused the leak in the first place. Had there never been a leak, we wouldn't have had this red alert. Had I never used Codex, absolutely, yes. I guess I didn't think... I'd seen a lot of crazy stuff, but I didn't think they would go this far. [laughter]
Brady: Do you think you should have been more secretive? I've heard stories about Andrew Wiles with Fermat's Last Theorem, and he was very secretive about what he was working on. No one knew he was doing it until he stood in front of that blackboard. Were you not secretive enough? Is that the problem?
Tristan: We thought we were. [laughter] But the leak was bad. That was a big mistake.
Brady: Was that because of computers, or was that just humans talking over coffee and at the water cooler?
Tristan: That's exactly what I think happened. I don't actually know exactly. There are a lot of conflicting stories, but I feel like it was people bragging about what their company is doing and stuff like that.
Brady: I've read your statement, and I will link to it again so people can look at it in depth, but you make it pretty clear in that statement that you think you probably would have got more credit and attribution, and been more involved, if your collaborator didn't happen to be an employee of Anthropic. Is that the case? Do you think half the reason you've been put in the position you're in is because of this rivalry between OpenAI and Anthropic?
Tristan: Well, I want to push back on the credit thing, because I'm not after more credit for the work. What I'm trying to push back on is the culture of how these companies are acting and the negative effect it's having on the mathematical community. It's less about trying to have more credit; I don't care about receiving the Clay Millennium Prize. The problem is this ultra-competitiveness between the labs. I should say they weren't the only lab that started spending millions of dollars to solve the Navier–Stokes problem after the rumor. I've heard there was another well-known company that did the same. What I want to push back on is the way this move-fast-and-break-things culture is a head-on collision with the math community, and in the future will be a head-on collision with the rest of us as well.
Brady: Other than the way that you and your collaborator and a few other people have been treated, and the lack of manners and propriety, isn't this a good thing? Isn't this crazy competition, and all the resources they've thrown into it, advancing the field? They cracked this problem that you all dreamed of cracking, which they may not have done. It could have been decades before this problem was solved, and it got done really quickly. As a mathematician, who I presume is in pursuit of truth and more knowledge, this has actually moved a frontier.
Tristan: So let's look at what happens if they hadn't done it. They would have cracked it a few days later. [laughter] The world would not have changed. Now let's look at the negative impacts it's had. The Millennium Problem being solved on September 8, I think it was, and not September 20: the world would not have changed. But it has had huge negative impacts on the mathematical community, because now everyone doesn't want to talk about their open problems. People are afraid that these AI companies will suddenly come in and scoop them, treat them as benchmark problems. People don't trust any of these AI products. It's had a huge negative impact on academia. It's slowed us down because of the threat these AI labs pose to us. So it's actually had a negative impact on scientific progress, and all for the sake of releasing a result a few weeks earlier. I love the Millennium Prize, but this is not curing cancer, and having solved this Millennium Problem on September 8 or September 20 would have had zero impact on the world.
Brady: You said yourself, though, that you have heard another AI company was working on it and had thrown a lot of resources at it. You can think what you want about that, but it does show why OpenAI would have been keen to get it out those few days earlier. That makes all the difference to them, because they get to say, "We did it," or "They did it." And competitive mathematicians are competitive too. Everyone here has a competitive streak.
Tristan: Yeah. I don't think it's as important as they think it is, but for them it's a matter of life or death for their products. They think that had Anthropic released a Millennium Prize problem, it would have affected their IPO, because it showed that Anthropic had much better internal models than they did. That is the rationale behind it. That was the big threat to them, a threat to their bottom line. It's kind of missing the point altogether, because it isn't just about the internal models; it's about using a lot of them. There's one part of the story, and I saw an OpenAI employee making the claim that it's all about the internal models, 90% internal models. It's the opposite way around. Maybe 20 or 30 percent, I don't know, is internal models, and the rest is using all these different agents within a particular harness or framework in order to solve a problem.
Brady: You've made something of a stand here. You've been willing to speak to people like me. You've made public statements. This is a huge, rich company with a lot of power and resources. Was that a difficult decision for you, to be principled about this? You could easily have gone another direction here. Did you think about consequences? Have you thought much about that, or has this just seemed like the natural thing to do?
Tristan: I was offered the opportunity to write as sole author of their paper and throw my collaborator under the bus. Did I think about the opportunity of being sole author? Not for one second. It's so unethical. They were asking me to throw him under the bus, someone I've been working with for a year, just because he's an Anthropic employee. That was the only reason. They said it was so annoying that Levent is an Anthropic employee, otherwise they would have added him as an author. So the sole reason they didn't want to find a solution with Levent was that he was an Anthropic employee.
Brady: These 10,000 agents, this model that cracked Navier–Stokes: was it a moment of genius? Do you look at how it was done and think, "That was pretty amazing, that was pretty smart, well done"?
Tristan: Yeah. I've been giving these media interviews quite a bit, so I haven't had much time to digest the proof, but from what I've seen, and I'll talk about this at a later date, the key idea that went from Euler to Navier–Stokes is a smart idea. From what I can tell, it's not what they talk about in the introduction of their paper, and it's not what's been broadly talked about. I find it quite amazing that people are missing what the key idea was to get from Euler to Navier–Stokes.
Brady: Can you even give us a hint? I know you haven't fully digested it yourself yet, but as someone who made a video about it, I'd love to know what we've all missed.
Tristan: Yeah. What everyone has been explaining is that there's this collapsing vortex, and this is the principal idea of what causes the solution to the Millennium Problem. But this collapsing vortex is not a solution to Navier–Stokes. You can create endless non-solutions to the Navier–Stokes equation when you have forcing, because you just put everything in the forcing. It's like you make a mistake, and whatever the error is, you say the force equals the error. So the collapsing vortex is not a solution. If you were to present it, it would have infinite force. You'd be putting infinite energy into the solution, and a singularity occurs. That's not surprising. Anyone can do that.
Brady: Okay.
Tristan: The secret is: how do you take something which is not a solution and turn it into a solution without having an infinite force put in? How do you correct that? How do you fix this non-solution? This is where the key idea comes in. The key idea is to use a concept called convex integration, which, unsurprisingly, is what I built my whole career on. It's to combine ideas from convex integration with the growth mechanism from the Euler equation. Convex integration is a way of fixing things; it's a mechanism for fixing the error. It's used in a different way in this proof, but its history traces all the way back to John Nash. So you combine these ideas of convex integration with the growth mechanism of the Euler blowup, and if you combine these two ideas, you create a new mechanism which is used to correct this non-solution. This is actually a cool idea. It's the kind of idea that I've been trying and failing to realize for over ten years, to be able to use these convex integration tools. In fact, even with my postdoc for the last year, I've been trying to combine these ideas of convex integration with the ideas of Luis and Diego. I didn't manage to do it.
Brady: Have you looked at the OpenAI paper yet in enough detail to think, "Ah, yes, maybe you would have done it in ten years, maybe you wouldn't"? Have you been able to look at it yet and think, "Ah, I see it now, I see the leap"? Is there that moment?
Tristan: This is the leap. I've literally only spent a couple of days looking at it, but this was the leap. It's kind of one thing when AI companies just release this slop and don't cite anything. They did a terrible job of citing. They did cite my paper with Vlad on convex integration, and that was an early hint, but they didn't say how that paper was used. They also cited a different paper, by Sara Daneri and then László Székelyhidi, who was my supervisor, where they provided a minuscule hint of the idea that they used this mechanism. [laughter]
Brady: This controversy, which you unfortunately found yourself very much at the center of, has become a kind of kernel of a push against AI. We've seen this petition signed by all these Fields medalists, Terry Tao and the like, firing off warning shots about AI being brought in on these big problems. But you're an AI guy. It feels to me like you've been on board with this. How do you feel about that petition, and how do you feel about your role in its birth? Are you on board with the petition? Do you think they've overreacted? Tell me how you feel about that.
Tristan: I don't want to prescribe what's correct and what's wrong. I think we are in a new world in terms of what AI can do, and I think we should hear everyone's voices. We need to hear conflicting voices in order to figure out a good path forward. As you said, I am someone who uses AI extensively, and I have been using it for years now. It's a funny story: I was originally a computer scientist. When I was 13 years old, I used these weird things called GPUs, which turned out to be something important, to write a computer graphics engine. I wanted to be a computer game programmer. Then I had a transition. I had an uncle who was a mathematician, Batiki Kovatch, and he convinced me to go into math. I went to pen and paper, literally writing everything on paper, and then I came back and started using AI and computer systems. So I don't think you can go back. Mathematics is not a performative science. I made this analogy between Deep Blue and Kasparov, but the big difference is that we're quite happy to watch human players play chess. So far I haven't seen any interest in putting two mathematicians in a room and having them live-solve a math problem. I don't think that's great entertainment. So I don't think we can go back. I think we have to find a positive way forward that does incorporate AI. But obviously this incident with OpenAI shows there are a lot of things that need to be fixed.
Brady: Because you're such an AI guy and have been so good at using it, it does have this whole flying-too-close-to-the-sun feel to it. Do you see that?
Tristan: One thing I often say is that this result was not our only result. I have this WhatsApp group with Levent, and we have a whole bunch of results on different problems. One thing I could do is pretend nothing has changed, release all these results, and get all my publications in the top journals. But I think that's silly. Some people might think that holding back the truth is not a good thing. But we have to be real: it's not going to change the world if I don't release a pure math proof. I think there are more important things here, and we have to take a step back. Besides, I think people would be more interested in knowing how we solved the problem than in us using our techniques secretly to solve more problems. I have a responsibility to explain our methods and how we went about solving these problems, and that's what I'll dedicate my time to in the next few months as well.
Brady: If you had found out that a mathematician, or a couple of mathematicians, in Japan were pretty much on the same track as you, via a rumor or a call, and you and Levent had said, "Okay, we've got to put our foot on the accelerator to make sure we publish before these guys," and then you did, how is that different to what OpenAI did, other than the fact that they can do it so much more quickly, inhumanly quickly? I want to really nail down what OpenAI did, because you said if OpenAI had done this a few days later, you'd be cool with it. What's the thing they did that you think is really bad?
Tristan: In that circumstance, say someone in Japan had these problems, and I was working with Levent and he had access to internal models, I would not try to outrun them. I think that's fair.
Brady: Right.
Tristan: If there was a group of mathematicians, let's say Luis and Diego, whose work I built on, who were close to solving unforced Euler or close to solving Navier–Stokes, and I had that impression, there's no way I would have used resources that they don't have access to in order to front-run them. I think that's inappropriate.
Brady: That's the main thing: the code of conduct of mathematicians is that if someone else is at a similar place to you, you join forces rather than accelerate to defeat them.
Tristan: I don't want to say there's a great code of conduct, because there are a lot of famous incidents in mathematics where people have presented some work, or talked about some work at a conference, and then someone went off, wrote down notes, and tried to front-run them. This is absolutely frowned upon within the mathematical community, but it does happen.
Brady: Speaking to you as a fellow Australian, some of the attitude you're taking feels like cricket: it's just not cricket to do that. It's not the spirit of the game. That's fair enough, and I respect that more than anyone. But you can also be ruthless if you want to be.
Tristan: But I will say that if it had just been that, I would have dealt with it. I think it's still wrong, but that's not what I believe happened.
Brady: If they've gone into Codex, and they'll say they haven't, but if they've gone in and taken your work out of that, work you put in there in good faith, that does feel like you've had your pocket picked.
Tristan: Yeah. And that's why I'm upset.
Brady: All right. And I just want to be clear, because I don't know if they're going to speak to me: they will claim, and they have claimed publicly multiple times, and I will link to it, that that's not what they did.
Tristan: Absolutely. But they also said that they one-shotted Navier–Stokes.
Brady: Will this change the way you use AI as a mathematician?
Tristan: Certainly, it changed the way I use Codex. [laughter] So nothing important goes into Codex. [laughter] Nothing of any actual value, no actual original ideas, will go into Codex.