This preface was formed from my Chinese statements, then substantially edited and translated by Codex. I have reviewed and approved its meaning. It is marked as LLM output because my approval does not change how the prose was produced.
I want to publish this experience because, when I looked for material and discussions about AI memory, I found relatively little, and what I did find did not quite address the problem I actually cared about. That information gap was one reason I decided to try building the system myself.
I hope that making the process public will give people who encounter a similar situation more material to work with, rather than forcing them to restart from the same point. I do not believe that my approach or conclusion is the correct answer, and I do not want it to be treated as objective knowledge. It is better understood as a record of one personal exploration: what problem I encountered, how I understood it, what I tried, and why I eventually changed direction. Readers can treat it as a case to consult and challenge, not as a conclusion they are expected to accept.
I broadly endorse the article's account of this experience and my present understanding, and I accept responsibility for the experiences, concerns, and judgments that are mine. But I do not believe my theory is more advanced or better than existing approaches. It is the result of pursuing a different objective from a different direction, and should be understood as a different path of exploration rather than ranked above or below others.
Codex participated in two main ways. First, it helped implement the system, turning ideas from discussion into code, tests, and a running project. Second, it translated the scattered, repetitive, and continually revised material produced during implementation into a structure and language that readers could more easily understand.
If readers ask questions, I am willing to answer from my own understanding and overall conception at the time I am asked. I cannot promise that my answers will follow the article sentence by sentence, and I do not want the article to become a fixed formulation that I must preserve unchanged. It records an understanding formed at this stage. In future answers, I will still speak from the understanding I genuinely have then.
1. I deleted a system that worked
On August 10, 2026, I deleted a memory system I had built for GPT.
Immediately before the cleanup, the project occupied about 5.53 GB on my computer. That included a 4.28 GB local embedding model, the llama.cpp and CUDA runtime files, a download cache, a vector database, per-task ledgers, and five MCP processes that were still running. The final deletion commit removed nineteen tracked files and 5,130 lines of tracked content. The last version had thirty-one unit tests.
The system was not deleted because it had failed to run.
It could split long-form memories into natural-language passages, index them locally, and retrieve relevant passages in a new conversation. It isolated different Codex tasks, preserved both a formed memory and the source dialogue visible when that memory was formed, and recorded whether an older passage had merely contributed a local consideration or had handed over its main retrieval responsibility to a newer passage. In a closed-loop test, a newly formed memory entered retrieval, an older claim it had superseded stopped appearing by default, and a still-independent risk passage remained retrievable.
That test mattered. It showed that the mechanism could enforce the rules we had given it. It did not show that the rules made future answers better.
This distinction became the center of the project. An implementation can be internally successful while the product decision behind it is wrong. A test can prove that a state transition occurred without proving that the transition improved judgment. A system can become more reliable and, at the same time, less proportionate to the problem that motivated it.
My original problem was modest: a conversation would arrive at an understanding, and the next conversation would not naturally continue from it. I wanted the next instance to inherit more than a user profile or a summary. I wanted it to know why a plausible plan had been rejected, which concept had been revised, and where an unfinished question had actually reached.
Several days later, that desire had become a local embedding service, a vector database, provenance records, memory-formation relations, task identity binding, persistent event ledgers, failure receipts, process lifecycle management, and a protocol that ordinary conversations were expected to follow.
The system had become impressive enough that deleting it felt like a substantive decision rather than routine cleanup. But its success also made the decision easier to see. I was no longer asking whether I could build a more complete memory system. I was asking what kind of memory I actually needed in daily work.
The answer was increasingly simple: perhaps I needed a cleaner current state, not a richer reconstruction of how every conclusion had formed.
2. How continuity became architecture
The experiment began with one instruction to Codex:
Recall this conversation, and leave this memory for the next version of you.
The wording mattered. I did not want minutes written for a human observer. I wanted an external trace addressed to a future model instance that had not witnessed the discussion. It should help that instance re-enter an already-developed understanding: what had been tried, what had been rejected, what remained in conflict, and where action could resume.
I initially resisted fixed fields such as “person,” “project,” “preference,” and “todo.” Much of the material did not belong cleanly to one category. A statement could be a fact, a judgment, and a future constraint at once. Its meaning often came from its relation to the surrounding argument. A schema imposed too early would decide what mattered before recall had even occurred.
The first durable principle was therefore to preserve memories as complete natural-language passages. But complete preservation did not imply complete reinjection. Future conversations should retrieve only the portions relevant to what they were currently touching.
That gave the system its first tension: preserve fully, recall locally.
The obvious implementation was to split the memory document into paragraphs, embed each paragraph, and use the current user message as a similarity query. Real conversations quickly exposed a problem. A user message is not necessarily a clean description of the problem being engaged. It may contain quoted text, references to earlier turns, several possible directions, repetition, and a late correction that changes the meaning of everything before it.
We added a thin interpretive layer. Before retrieval, the model participating in the conversation would describe the “current contact surface”: the object being touched, the current position of the question, its relationship to prior discussion, and the likely next action. That description—not the raw message alone—became the retrieval query.
This was not intended to let the model answer the question in advance. It was intended to give search the context that only the current conversation possessed.
A second boundary followed: vectors have search authority, not judgment authority.
Semantic similarity can choose where to look. It cannot establish that a retrieved passage is true, current, applicable, or causally relevant to the conclusion now being formed. Our ordinary configuration therefore used a similarity threshold rather than an arbitrary top-k cutoff. Every still-eligible passage above the threshold was returned to the main model. Slightly lower-scoring passages were retained as “shadow” results so we could inspect associations the threshold excluded. Passages already injected into the current scope were removed from subsequent searches to avoid repetition.
None of this abolished arbitrariness. It made the location of that arbitrariness inspectable.
Hardware imposed another correction. The first local model was an 8B quantized embedding model. To let memory coexist with a browser, video, games, and other daily software, I moved to a 4B quantized version. In a six-query local comparison over the small memory corpus, both models placed the intended passage in the top two results, and five queries had the same first result. The remaining query reversed two related passages. This did not prove long-term equivalence, but the smaller model preserved the useful candidate set while leaving substantially more GPU headroom.
The choice captured the practical nature of the project. The relevant question was not which embedding model was stronger in the abstract. It was whether the memory system could remain present on the computer without making the rest of the computer worse.
Then provenance became a problem. A continuously rewritten memory could not, by itself, prove where its claims had come from. If a new passage said it inherited an earlier discussion, we wanted to know which dialogue and recalled passages were actually visible during formation—not reconstruct the source later from semantic similarity.
So the system separated the formed memory from its formation evidence. The first was intended for future use. The second preserved the user messages, assistant responses, and recalled material visible during that formation event. A long conversation could produce more than one formation record, because “one conversation” and “one durable update” were not the same unit.
The system's shape was not specified in an initial blueprint. The contact surface answered query distortion. Scope answered duplicate injection. Formation evidence answered provenance loss. The smaller model answered resource contention. Each component appeared after some version of: no, this still does not solve the problem I meant.
3. Retrieval is not participation
The design changed most sharply when we stopped treating retrieval as evidence that a memory had participated in a new conclusion.
Suppose a search returns four old passages: A, B, C, and D. All four enter the context window. The model can see all four, but the new understanding actually uses only A, B, and C. D appeared because it was semantically adjacent; it did no work in the formation of the new claim.
If the program treats “retrieved” as equivalent to “inherited,” it may rewrite D's future status without evidence.
We therefore separated three events:
Retrieved: the search system returned a passage.
Visible: the passage entered the model's context.
Participating: the passage materially contributed to the new memory being formed.
The first two are observable program events. The third is an interpretive judgment. A model that sees the full context can report which passages it used, and a human can later inspect the formation evidence, but neither operation is a causal measurement. The judgment can be wrong.
To make knowledge change rather than merely accumulate, the system represented two local participation relations.
A subject relation meant that the main judgment of an old passage had been taken over by the new memory. The old passage remained stored, but it stopped being eligible for ordinary active retrieval; the new passage assumed its primary retrieval responsibility.
An edge relation meant that the old passage had supplied a risk, exception, or local piece of evidence that the new memory did not fully replace. The new memory recorded the participation, but the old passage retained independent retrieval eligibility.
These were relations in a particular formation event, not permanent types of memory. The same passage could be central in one update and peripheral in another.
On August 4, we ran the first full loop. The system retrieved old passages, formed a new memory, attached one old passage as subject and another as edge, generated the new vectors, and only then changed retrieval eligibility. In a fresh scope, the new memory appeared. The superseded subject did not. The independent risk passage did. If vector generation failed, the old subject remained active; a partially completed update was not allowed to erase its predecessor.
This was the moment when “knowledge iteration” became more than appending another note. A new belief could take responsibility for being recalled in place of an older one.
It also exposed the most dangerous part of the system. A mistaken edge judgment mostly creates redundancy. A mistaken subject judgment may suppress something that still matters. The more aggressively a memory system cleans its current state, the more consequential its editing errors become.
The mechanism therefore preserved the old text and formation evidence even when the old passage left default retrieval. That made the decision inspectable and reversible. It did not make it objectively correct.
This distinction remains useful to me even after deleting the system. “The model had access to the information” is weaker than “the information changed the answer.” A memory benchmark based only on retrieval can miss that gap. A system that reports its own use of memory supplies evidence, but not proof. And any memory system that retires, consolidates, or overwrites old knowledge is making an editorial decision about the future—not merely saving storage.
Our subject and edge implementation was a small local answer, not a new general architecture. Temporal knowledge graphs and agent-memory systems already represent evolving facts, provenance, links, and invalidation in more systematic ways. Our implementation pursued a different local objective: preserving how one evolving conversation handed retrieval responsibility from an older natural-language understanding to a newer one. What mattered in this project was encountering the responsibility directly: once new knowledge can change what is recalled, memory maintenance becomes a form of judgment.
4. I wanted memory to enter the process of thinking
After using the system, I noticed that Codex rarely chose to call the recall tool on its own. One possible explanation was straightforward: if the present context already looked sufficient, there was no visible reason to search the past. Recall was more likely after the model had already recognized an information gap.
That creates a loop. Asking a model to recall voluntarily asks it to know, before remembering, that some unknown memory might be useful. It can search for a gap it has already identified. It cannot deliberately search for a memory whose existence has not yet affected its current state.
We tried a second route. At the end of each turn, the model described what the conversation had just touched. That description was stored in a task ledger and used as an automatic retrieval query on the next turn. This avoided depending on a voluntary tool call, but placed recall at the wrong time. When the description was written, the answer had already been formed. When it was used, the user might have changed topics.
I summarized the problem in one sentence:
Memory is a process, not a result.
The intuition came from human experience. I do not ordinarily decide to search an inventory of everything I might remember. Sometimes a memory appears during composition, and only then do I know that it was available to be remembered. As I put it at the time:
Before this memory appeared, I did not [know] that it existed. But while composing or replying, it appeared, and then I knew it.
This is suggestive but dangerous. My experience does not establish that a language model has a similar subjective stream. An external memory can affect a model only through a real causal channel: text in the context window, a tool result, or some other state the host makes available to generation. “The model vaguely sensed a memory” is not an implementation.
The question therefore became narrower: how can an unnamed piece of the past become background before the main judgment has already formed?
We considered three possible moments. First, after a new message arrives but before the model commits to a main interpretation, the host could prefetch a range of relevant memories from the message and current conversation. Second, after a partial interpretation exposes a more specific gap, the model could expand retrieval around that gap. Third, after a plan or draft exists but before it is sent, the model could check it against relevant past material.
These positions correspond to background, deepening, and review. Voluntary recall would remain available without being the only entrance. End-of-turn notes could remain useful without pretending to be next-turn association.
We did not run a controlled comparison. We did not establish whether this cycle would surface more unknown-but-useful memories, introduce distracting material, or cost too much latency and quota. The durable insight was smaller: recall quality depends not only on what enters context, but when it enters. The same passage supplied before a conclusion and appended after a conclusion should not be treated as equivalent interventions.
Trying to place memory inside formation changed the engineering burden. The system was no longer a search tool waiting to be called. It was beginning to schedule when a conversation should pause, retrieve, continue, and record.
At one point, every new message triggered recall by default. While I was running a game, the resulting contention became obvious. “Remember every turn” meant repeatedly starting a multi-gigabyte embedding model. Exiting after each call released GPU memory but retained cold-start cost. Starting only when needed restored responsiveness but returned the unknown-memory problem to the model.
Writing had the same loop. We wanted Codex to decide when a discussion had produced a durable new understanding. That decision was unreliable. Adding phase labels, turn thresholds, or another classifier would let the infrastructure prescribe what counted as completion. We eventually withdrew both automatic write authority and a fixed formation schedule. Durable memory again became a sparse, explicit act.
Explicit formation still needed reliable source evidence. Having the model copy the dialogue into a write tool would add output and transform the “source” before storage. We introduced hooks that attempted to extract the visible dialogue from the real task record. That exposed a host-level identity problem: which Codex task did an MCP tool call belong to? Could concurrent tasks share a process? Could an environment variable contain stale identity? Could a staged event be consumed before successful delivery?
The final version required a real CODEX_THREAD_ID at each entry point, verified that it mapped uniquely to the correct task record, bound the MCP process to that task, and gave each task a persistent ledger. Passive descriptions were immutable events. A successful complete recall added a consumption record; a failure left the event pending.
None of these problems were imaginary. Concurrent tasks can cross wires. Sources can be truncated. Processes remain alive. Failed writes cannot honestly be recorded as successful. Reliability required taking those facts seriously.
But ordinary conversation now carried a ritual: obtain task identity, retrieve or decide not to retrieve, maintain scope, stage an end-of-turn description, occasionally form durable memory, synchronize vectors, manage a database, and keep the host integration compatible. The infrastructure was no longer only serving the conversation. The conversation was serving the infrastructure by continuously supplying identity, stages, receipts, and formation events.
That inversion did not prove the mechanisms useless. Audit-heavy, multi-user, or research systems may need exactly this machinery. It showed that the machinery had exceeded the scale of my original problem: one person's conversational continuity.
5. The official memory system changed the evidence
The decision to stop maintaining the self-built system began before I had inspected OpenAI's local Codex memories. At that point, I knew the maintenance cost of my own system. I did not yet know whether the official alternative would satisfy the underlying need.
The first necessary distinction is between product surfaces. ChatGPT web memory and local Codex memory have separate stores and controls. OpenAI's current documentation says that, when local memories are enabled, Codex can turn useful context from eligible prior chats into local files. It skips active or short-lived chats, updates memory in the background after sufficient idle time, and stores summaries, durable entries, recent inputs, and supporting evidence. Separate settings control whether chats contribute to new memories, whether existing memories enter later sessions, which model performs per-chat extraction, and which performs global consolidation.[1]
That public description already went beyond “vector-search the chat log.” It described a state-maintenance pipeline: selection, delayed extraction, consolidation, later use, and supporting evidence.
On August 12, after using the feature for a while, I inspected what it had actually produced on my computer.
The early sample had four observable layers: a short high-level summary, a longer durable document grouped by task, a merged extraction of per-task material, and eight detailed task summaries. The durable document separated user preferences, reusable knowledge, and failures with suggestions for what to do differently. It linked downward to supporting summaries and source records. In the Codex workflow visible to me, the new task began with the short summary; deeper material was searched when relevant, and detailed evidence was opened when the consolidated description was insufficient.
More important than the file hierarchy was what the system had selected. It did not merely record that I had discussed memory. It extracted lessons such as:
Do not infer an unpublished product implementation from an OpenAI Cookbook reference architecture.
Ten successful browser-chat probes do not prove that their quota is independent.
A project book, AGENTS.md, memory, and a Skill should carry different responsibilities.
A Skill should crystallize repeated successes, failures, and exceptions from practice rather than predict a workflow entirely in advance.
It also had an explicit section for failures and how to approach them differently next time.
This changed the evidence. The official local system was no longer merely similar in shape to our earlier speculation. In this small personal sample, it had captured material from natural tasks, distinguished reusable knowledge from failures, consolidated across tasks, retained routes to evidence, and supported progressive recall. That covered a substantial part of the goal that had motivated the self-built system.
The inspection also revealed unresolved problems. Principles specific to the retired vector-memory project—such as retrieving all highly similar passages and loading a local embedding model only on demand—still appeared among high-level preferences. A single support interaction could be generalized quickly into a broader communication preference. The files preserved a current C and evidence beneath it, but did not visibly represent, in the manner of our subject and edge relations, exactly which older A or B had been replaced, limited, or inherited.
This revised an earlier thought:
Perhaps you do not need to know how A and B became C. Perhaps knowing C is enough.
The missing condition is that C must be accurate, and superseded A and B must be reliably demoted. A clean wrong summary can be harder to notice than a visibly conflicted history. If stale principles remain in the high-level state, the past still returns—only now with the authority of consolidation.
I therefore do not claim that official memory has “won” a controlled comparison. The sample was small. I did not run both systems on an identical task suite. I have not established that official memory reliably improves future behavior, retires stale knowledge, or avoids overgeneralization.
My current claim is narrower and practical: for my daily production use, local Codex memory has shown enough of the required shape to become the default. It supplies cross-task recall and knowledge consolidation without requiring me to operate an embedding service, process pool, vector database, task-identity protocol, and formation ledger. The unanswered questions can now be investigated through ordinary use rather than by keeping the entire self-built stack alive.
OpenAI, “Memories”. The documentation explicitly distinguishes ChatGPT web memory from the local Codex store. Nothing in this account infers ChatGPT web internals from the local files. ↩︎
6. Deletion was a product decision, not a verdict on the questions
A system that runs does not acquire a right to continue existing.
Each recall in my system carried model lifecycle, GPU memory, process, and timeout concerns. Each durable update carried task identity, provenance, scope, relation-writing, and failure semantics. Adding another chat surface reopened questions about identity, source access, and write authority. Every issue was individually solvable. The solved system would still require one person to maintain the whole stack.
Meanwhile, the responsibilities I had bundled into “memory” could be separated.
Local Codex memory could provisionally carry preferences, cross-task experience, failures, and project entry points.
A version-controlled project book could preserve a project's current whole, prior work, established considerations, and explicit future items.
AGENTS.md could carry instructions that must always apply.
Skills could package stable, repeatable workflows after practice had revealed their inputs, failure modes, hard boundaries, and acceptance checks.
Git and selected formation evidence could preserve history that genuinely needed auditability.
This division changed my understanding of Skills as well. During a separate discussion about sending some Codex engineering work to browser-based ChatGPT sessions to relieve local quota pressure, I initially wanted to “build a Skill” because the handoff protocol could not tolerate missing a step. I then objected to the word “build”: perhaps we should first discuss the purpose and boundaries, then fix the resulting procedure.
Even that placed discussion before evidence. In practice, recurring inputs, common omissions, safe discretion, costly exceptions, and real acceptance criteria appear only after a workflow has happened more than once. A pre-practice Skill may freeze a first prediction of the workflow rather than knowledge of the workflow.
My revised formulation was:
A Skill should be formed from things that have already been done, rather than by predicting the shape of the Skill in advance and optimizing it during execution.
This is not an absolute ban on writing a Skill early. It is a claim about evidence order. Let real work generate successes, failures, detours, exceptions, and validation criteria. When a stable pattern appears, crystallize the non-negotiable parts into an executable contract. When subsequent work repeatedly exposes a gap, revise the contract with evidence.
OpenAI's current Skill format fits this division without proving it. A Skill packages instructions, resources, and optional scripts for a repeatable workflow. Its name and description are visible for discovery; the full SKILL.md is loaded only when selected.[1]Memory can remain the changing account of what has been learned. A Skill can become the versioned procedure that should now run reliably.
The memory architecture itself was also not novel in its components. Generative Agents stored natural-language experience, retrieved it dynamically, and synthesized higher-level reflections for planning.[2]MemGPT framed tiered context as virtual memory, and Letta now exposes always-visible blocks alongside files, archival memory, and external retrieval.[3][4]LongMemEval evaluates information extraction, multi-session and temporal reasoning, knowledge updates, and abstention.[5]Zep and Graphiti represent changing facts, provenance, and temporal validity in a graph.[6]A-MEM dynamically links memories and lets new additions update older representations.[7]Mem0 extracts, consolidates, and retrieves salient conversational information while measuring deployment costs against full-context approaches.[8]
I did not begin by reviewing that literature and selecting an open research problem. I began with an inability to continue a conversation naturally. Over several days, failures pushed a local system toward problems that the broader field had already encountered in more systematic forms.
The point of presenting that history is not to claim a more advanced or generally better theory. The system pursued a different goal from many benchmarked memory architectures: carrying the development of one person's understanding across natural work while retaining some trace of how later views took responsibility from earlier ones. What this article offers is a record of how an ordinary user encountered the responsibilities hiding behind the word “memory”—and how engineering those responsibilities changed the user's view of the original problem.
The deletion happened before my direct inspection of the official local memory files. At deletion time, the evidence was asymmetric: the maintenance cost of the self-built system was real, while the sufficiency of the official system remained uncertain. Stopping was still rational because the system's production cost was known and its unique necessity was not.
The later inspection supplied additional evidence for the decision without turning it into a controlled experiment. The official system had already begun to capture reusable knowledge, failures, and evidence routes. It had not required the five-process, 5.53 GB local stack to keep doing so.
What I deleted was therefore not the record of the discussion, the code history, or the question of how knowledge changes. I deleted the self-built vector-memory system's role as default daily infrastructure. The natural-language memory material and Git history remained available for research and review.
If official memory later proves unable to recover the right project entry point, the next response need not be to resurrect the entire stack. The memory layer can remember which project matters, while the project book supplies the full state. Versioned documents can hold facts that must be exact. A thin audit trail can be added only if formation history becomes a demonstrated need.
This project left me with a stricter evaluation question. “Did the assistant remember me?” is too vague. Better questions are: What did it capture? What applicability boundary did it assign? Did new knowledge revise old knowledge? When did the memory enter the decision? And did the next similar task actually go differently?
The final question is the shortest:
What did this piece of the past cause you to decide differently now?
I still believe that memory deserves to be treated as a module in its own right. In helping AI adapt better to people, and in enabling longer-term human–AI collaboration, memory may be an important layer. But that is precisely why engineering around memory should be approached cautiously. Memory is not only about what content is stored. It also involves how that content is formed, when it enters judgment, how it is updated, when it exits, and how it ultimately changes the next action.
With the local Codex memory I have encountered so far, I still feel that it has not implemented the part I care about most. It can extract and organize material from prior conversations and bring that material into future work; but in my ideal, memory would not be merely external material appended to a model. It would participate more deeply in how the model forms understanding and judgment. I suspect that this gap may have something to do with how memory is considered during model training: if the existence, updating, and use of memory were included in training-time design, perhaps a different capability would be more likely to emerge. But this is only my conjecture. OpenAI's public documentation on memory does not say whether, or how, memory was considered during training; I also cannot yet specify what capabilities an ideal memory module should have.
I am willing to accept that using LLM content blocks may cause my first submission to be rejected because I believe AI-generated and AI-organized prose can translate my personal way of expressing myself into language that more readers can understand. For me, honestly marking how AI participated in that translation matters more than hiding it to improve the probability of approval. If this mode of production does not meet the community's requirements for a first post, rejection is a consequence I am willing to accept as part of choosing this form of expression.
Preface on authorship
This preface was formed from my Chinese statements, then substantially edited and translated by Codex. I have reviewed and approved its meaning. It is marked as LLM output because my approval does not change how the prose was produced.
I want to publish this experience because, when I looked for material and discussions about AI memory, I found relatively little, and what I did find did not quite address the problem I actually cared about. That information gap was one reason I decided to try building the system myself.
I hope that making the process public will give people who encounter a similar situation more material to work with, rather than forcing them to restart from the same point. I do not believe that my approach or conclusion is the correct answer, and I do not want it to be treated as objective knowledge. It is better understood as a record of one personal exploration: what problem I encountered, how I understood it, what I tried, and why I eventually changed direction. Readers can treat it as a case to consult and challenge, not as a conclusion they are expected to accept.
I broadly endorse the article's account of this experience and my present understanding, and I accept responsibility for the experiences, concerns, and judgments that are mine. But I do not believe my theory is more advanced or better than existing approaches. It is the result of pursuing a different objective from a different direction, and should be understood as a different path of exploration rather than ranked above or below others.
Codex participated in two main ways. First, it helped implement the system, turning ideas from discussion into code, tests, and a running project. Second, it translated the scattered, repetitive, and continually revised material produced during implementation into a structure and language that readers could more easily understand.
If readers ask questions, I am willing to answer from my own understanding and overall conception at the time I am asked. I cannot promise that my answers will follow the article sentence by sentence, and I do not want the article to become a fixed formulation that I must preserve unchanged. It records an understanding formed at this stage. In future answers, I will still speak from the understanding I genuinely have then.
1. I deleted a system that worked
On August 10, 2026, I deleted a memory system I had built for GPT.
Immediately before the cleanup, the project occupied about 5.53 GB on my computer. That included a 4.28 GB local embedding model, the llama.cpp and CUDA runtime files, a download cache, a vector database, per-task ledgers, and five MCP processes that were still running. The final deletion commit removed nineteen tracked files and 5,130 lines of tracked content. The last version had thirty-one unit tests.
The system was not deleted because it had failed to run.
It could split long-form memories into natural-language passages, index them locally, and retrieve relevant passages in a new conversation. It isolated different Codex tasks, preserved both a formed memory and the source dialogue visible when that memory was formed, and recorded whether an older passage had merely contributed a local consideration or had handed over its main retrieval responsibility to a newer passage. In a closed-loop test, a newly formed memory entered retrieval, an older claim it had superseded stopped appearing by default, and a still-independent risk passage remained retrievable.
That test mattered. It showed that the mechanism could enforce the rules we had given it. It did not show that the rules made future answers better.
This distinction became the center of the project. An implementation can be internally successful while the product decision behind it is wrong. A test can prove that a state transition occurred without proving that the transition improved judgment. A system can become more reliable and, at the same time, less proportionate to the problem that motivated it.
My original problem was modest: a conversation would arrive at an understanding, and the next conversation would not naturally continue from it. I wanted the next instance to inherit more than a user profile or a summary. I wanted it to know why a plausible plan had been rejected, which concept had been revised, and where an unfinished question had actually reached.
Several days later, that desire had become a local embedding service, a vector database, provenance records, memory-formation relations, task identity binding, persistent event ledgers, failure receipts, process lifecycle management, and a protocol that ordinary conversations were expected to follow.
The system had become impressive enough that deleting it felt like a substantive decision rather than routine cleanup. But its success also made the decision easier to see. I was no longer asking whether I could build a more complete memory system. I was asking what kind of memory I actually needed in daily work.
The answer was increasingly simple: perhaps I needed a cleaner current state, not a richer reconstruction of how every conclusion had formed.
2. How continuity became architecture
The experiment began with one instruction to Codex:
The wording mattered. I did not want minutes written for a human observer. I wanted an external trace addressed to a future model instance that had not witnessed the discussion. It should help that instance re-enter an already-developed understanding: what had been tried, what had been rejected, what remained in conflict, and where action could resume.
I initially resisted fixed fields such as “person,” “project,” “preference,” and “todo.” Much of the material did not belong cleanly to one category. A statement could be a fact, a judgment, and a future constraint at once. Its meaning often came from its relation to the surrounding argument. A schema imposed too early would decide what mattered before recall had even occurred.
The first durable principle was therefore to preserve memories as complete natural-language passages. But complete preservation did not imply complete reinjection. Future conversations should retrieve only the portions relevant to what they were currently touching.
That gave the system its first tension: preserve fully, recall locally.
The obvious implementation was to split the memory document into paragraphs, embed each paragraph, and use the current user message as a similarity query. Real conversations quickly exposed a problem. A user message is not necessarily a clean description of the problem being engaged. It may contain quoted text, references to earlier turns, several possible directions, repetition, and a late correction that changes the meaning of everything before it.
We added a thin interpretive layer. Before retrieval, the model participating in the conversation would describe the “current contact surface”: the object being touched, the current position of the question, its relationship to prior discussion, and the likely next action. That description—not the raw message alone—became the retrieval query.
This was not intended to let the model answer the question in advance. It was intended to give search the context that only the current conversation possessed.
A second boundary followed: vectors have search authority, not judgment authority.
Semantic similarity can choose where to look. It cannot establish that a retrieved passage is true, current, applicable, or causally relevant to the conclusion now being formed. Our ordinary configuration therefore used a similarity threshold rather than an arbitrary top-k cutoff. Every still-eligible passage above the threshold was returned to the main model. Slightly lower-scoring passages were retained as “shadow” results so we could inspect associations the threshold excluded. Passages already injected into the current scope were removed from subsequent searches to avoid repetition.
None of this abolished arbitrariness. It made the location of that arbitrariness inspectable.
Hardware imposed another correction. The first local model was an 8B quantized embedding model. To let memory coexist with a browser, video, games, and other daily software, I moved to a 4B quantized version. In a six-query local comparison over the small memory corpus, both models placed the intended passage in the top two results, and five queries had the same first result. The remaining query reversed two related passages. This did not prove long-term equivalence, but the smaller model preserved the useful candidate set while leaving substantially more GPU headroom.
The choice captured the practical nature of the project. The relevant question was not which embedding model was stronger in the abstract. It was whether the memory system could remain present on the computer without making the rest of the computer worse.
Then provenance became a problem. A continuously rewritten memory could not, by itself, prove where its claims had come from. If a new passage said it inherited an earlier discussion, we wanted to know which dialogue and recalled passages were actually visible during formation—not reconstruct the source later from semantic similarity.
So the system separated the formed memory from its formation evidence. The first was intended for future use. The second preserved the user messages, assistant responses, and recalled material visible during that formation event. A long conversation could produce more than one formation record, because “one conversation” and “one durable update” were not the same unit.
The system's shape was not specified in an initial blueprint. The contact surface answered query distortion. Scope answered duplicate injection. Formation evidence answered provenance loss. The smaller model answered resource contention. Each component appeared after some version of: no, this still does not solve the problem I meant.
3. Retrieval is not participation
The design changed most sharply when we stopped treating retrieval as evidence that a memory had participated in a new conclusion.
Suppose a search returns four old passages: A, B, C, and D. All four enter the context window. The model can see all four, but the new understanding actually uses only A, B, and C. D appeared because it was semantically adjacent; it did no work in the formation of the new claim.
If the program treats “retrieved” as equivalent to “inherited,” it may rewrite D's future status without evidence.
We therefore separated three events:
The first two are observable program events. The third is an interpretive judgment. A model that sees the full context can report which passages it used, and a human can later inspect the formation evidence, but neither operation is a causal measurement. The judgment can be wrong.
To make knowledge change rather than merely accumulate, the system represented two local participation relations.
A
subjectrelation meant that the main judgment of an old passage had been taken over by the new memory. The old passage remained stored, but it stopped being eligible for ordinary active retrieval; the new passage assumed its primary retrieval responsibility.An
edgerelation meant that the old passage had supplied a risk, exception, or local piece of evidence that the new memory did not fully replace. The new memory recorded the participation, but the old passage retained independent retrieval eligibility.These were relations in a particular formation event, not permanent types of memory. The same passage could be central in one update and peripheral in another.
On August 4, we ran the first full loop. The system retrieved old passages, formed a new memory, attached one old passage as
subjectand another asedge, generated the new vectors, and only then changed retrieval eligibility. In a fresh scope, the new memory appeared. The superseded subject did not. The independent risk passage did. If vector generation failed, the old subject remained active; a partially completed update was not allowed to erase its predecessor.This was the moment when “knowledge iteration” became more than appending another note. A new belief could take responsibility for being recalled in place of an older one.
It also exposed the most dangerous part of the system. A mistaken
edgejudgment mostly creates redundancy. A mistakensubjectjudgment may suppress something that still matters. The more aggressively a memory system cleans its current state, the more consequential its editing errors become.The mechanism therefore preserved the old text and formation evidence even when the old passage left default retrieval. That made the decision inspectable and reversible. It did not make it objectively correct.
This distinction remains useful to me even after deleting the system. “The model had access to the information” is weaker than “the information changed the answer.” A memory benchmark based only on retrieval can miss that gap. A system that reports its own use of memory supplies evidence, but not proof. And any memory system that retires, consolidates, or overwrites old knowledge is making an editorial decision about the future—not merely saving storage.
Our
subjectandedgeimplementation was a small local answer, not a new general architecture. Temporal knowledge graphs and agent-memory systems already represent evolving facts, provenance, links, and invalidation in more systematic ways. Our implementation pursued a different local objective: preserving how one evolving conversation handed retrieval responsibility from an older natural-language understanding to a newer one. What mattered in this project was encountering the responsibility directly: once new knowledge can change what is recalled, memory maintenance becomes a form of judgment.4. I wanted memory to enter the process of thinking
After using the system, I noticed that Codex rarely chose to call the recall tool on its own. One possible explanation was straightforward: if the present context already looked sufficient, there was no visible reason to search the past. Recall was more likely after the model had already recognized an information gap.
That creates a loop. Asking a model to recall voluntarily asks it to know, before remembering, that some unknown memory might be useful. It can search for a gap it has already identified. It cannot deliberately search for a memory whose existence has not yet affected its current state.
We tried a second route. At the end of each turn, the model described what the conversation had just touched. That description was stored in a task ledger and used as an automatic retrieval query on the next turn. This avoided depending on a voluntary tool call, but placed recall at the wrong time. When the description was written, the answer had already been formed. When it was used, the user might have changed topics.
I summarized the problem in one sentence:
The intuition came from human experience. I do not ordinarily decide to search an inventory of everything I might remember. Sometimes a memory appears during composition, and only then do I know that it was available to be remembered. As I put it at the time:
This is suggestive but dangerous. My experience does not establish that a language model has a similar subjective stream. An external memory can affect a model only through a real causal channel: text in the context window, a tool result, or some other state the host makes available to generation. “The model vaguely sensed a memory” is not an implementation.
The question therefore became narrower: how can an unnamed piece of the past become background before the main judgment has already formed?
We considered three possible moments. First, after a new message arrives but before the model commits to a main interpretation, the host could prefetch a range of relevant memories from the message and current conversation. Second, after a partial interpretation exposes a more specific gap, the model could expand retrieval around that gap. Third, after a plan or draft exists but before it is sent, the model could check it against relevant past material.
These positions correspond to background, deepening, and review. Voluntary recall would remain available without being the only entrance. End-of-turn notes could remain useful without pretending to be next-turn association.
We did not run a controlled comparison. We did not establish whether this cycle would surface more unknown-but-useful memories, introduce distracting material, or cost too much latency and quota. The durable insight was smaller: recall quality depends not only on what enters context, but when it enters. The same passage supplied before a conclusion and appended after a conclusion should not be treated as equivalent interventions.
Trying to place memory inside formation changed the engineering burden. The system was no longer a search tool waiting to be called. It was beginning to schedule when a conversation should pause, retrieve, continue, and record.
At one point, every new message triggered recall by default. While I was running a game, the resulting contention became obvious. “Remember every turn” meant repeatedly starting a multi-gigabyte embedding model. Exiting after each call released GPU memory but retained cold-start cost. Starting only when needed restored responsiveness but returned the unknown-memory problem to the model.
Writing had the same loop. We wanted Codex to decide when a discussion had produced a durable new understanding. That decision was unreliable. Adding phase labels, turn thresholds, or another classifier would let the infrastructure prescribe what counted as completion. We eventually withdrew both automatic write authority and a fixed formation schedule. Durable memory again became a sparse, explicit act.
Explicit formation still needed reliable source evidence. Having the model copy the dialogue into a write tool would add output and transform the “source” before storage. We introduced hooks that attempted to extract the visible dialogue from the real task record. That exposed a host-level identity problem: which Codex task did an MCP tool call belong to? Could concurrent tasks share a process? Could an environment variable contain stale identity? Could a staged event be consumed before successful delivery?
The final version required a real
CODEX_THREAD_IDat each entry point, verified that it mapped uniquely to the correct task record, bound the MCP process to that task, and gave each task a persistent ledger. Passive descriptions were immutable events. A successful complete recall added a consumption record; a failure left the event pending.None of these problems were imaginary. Concurrent tasks can cross wires. Sources can be truncated. Processes remain alive. Failed writes cannot honestly be recorded as successful. Reliability required taking those facts seriously.
But ordinary conversation now carried a ritual: obtain task identity, retrieve or decide not to retrieve, maintain scope, stage an end-of-turn description, occasionally form durable memory, synchronize vectors, manage a database, and keep the host integration compatible. The infrastructure was no longer only serving the conversation. The conversation was serving the infrastructure by continuously supplying identity, stages, receipts, and formation events.
That inversion did not prove the mechanisms useless. Audit-heavy, multi-user, or research systems may need exactly this machinery. It showed that the machinery had exceeded the scale of my original problem: one person's conversational continuity.
5. The official memory system changed the evidence
The decision to stop maintaining the self-built system began before I had inspected OpenAI's local Codex memories. At that point, I knew the maintenance cost of my own system. I did not yet know whether the official alternative would satisfy the underlying need.
The first necessary distinction is between product surfaces. ChatGPT web memory and local Codex memory have separate stores and controls. OpenAI's current documentation says that, when local memories are enabled, Codex can turn useful context from eligible prior chats into local files. It skips active or short-lived chats, updates memory in the background after sufficient idle time, and stores summaries, durable entries, recent inputs, and supporting evidence. Separate settings control whether chats contribute to new memories, whether existing memories enter later sessions, which model performs per-chat extraction, and which performs global consolidation.[1]
That public description already went beyond “vector-search the chat log.” It described a state-maintenance pipeline: selection, delayed extraction, consolidation, later use, and supporting evidence.
On August 12, after using the feature for a while, I inspected what it had actually produced on my computer.
The early sample had four observable layers: a short high-level summary, a longer durable document grouped by task, a merged extraction of per-task material, and eight detailed task summaries. The durable document separated user preferences, reusable knowledge, and failures with suggestions for what to do differently. It linked downward to supporting summaries and source records. In the Codex workflow visible to me, the new task began with the short summary; deeper material was searched when relevant, and detailed evidence was opened when the consolidated description was insufficient.
More important than the file hierarchy was what the system had selected. It did not merely record that I had discussed memory. It extracted lessons such as:
AGENTS.md, memory, and a Skill should carry different responsibilities.It also had an explicit section for failures and how to approach them differently next time.
This changed the evidence. The official local system was no longer merely similar in shape to our earlier speculation. In this small personal sample, it had captured material from natural tasks, distinguished reusable knowledge from failures, consolidated across tasks, retained routes to evidence, and supported progressive recall. That covered a substantial part of the goal that had motivated the self-built system.
The inspection also revealed unresolved problems. Principles specific to the retired vector-memory project—such as retrieving all highly similar passages and loading a local embedding model only on demand—still appeared among high-level preferences. A single support interaction could be generalized quickly into a broader communication preference. The files preserved a current C and evidence beneath it, but did not visibly represent, in the manner of our
subjectandedgerelations, exactly which older A or B had been replaced, limited, or inherited.This revised an earlier thought:
The missing condition is that C must be accurate, and superseded A and B must be reliably demoted. A clean wrong summary can be harder to notice than a visibly conflicted history. If stale principles remain in the high-level state, the past still returns—only now with the authority of consolidation.
I therefore do not claim that official memory has “won” a controlled comparison. The sample was small. I did not run both systems on an identical task suite. I have not established that official memory reliably improves future behavior, retires stale knowledge, or avoids overgeneralization.
My current claim is narrower and practical: for my daily production use, local Codex memory has shown enough of the required shape to become the default. It supplies cross-task recall and knowledge consolidation without requiring me to operate an embedding service, process pool, vector database, task-identity protocol, and formation ledger. The unanswered questions can now be investigated through ordinary use rather than by keeping the entire self-built stack alive.
OpenAI, “Memories”. The documentation explicitly distinguishes ChatGPT web memory from the local Codex store. Nothing in this account infers ChatGPT web internals from the local files. ↩︎
6. Deletion was a product decision, not a verdict on the questions
A system that runs does not acquire a right to continue existing.
Each recall in my system carried model lifecycle, GPU memory, process, and timeout concerns. Each durable update carried task identity, provenance, scope, relation-writing, and failure semantics. Adding another chat surface reopened questions about identity, source access, and write authority. Every issue was individually solvable. The solved system would still require one person to maintain the whole stack.
Meanwhile, the responsibilities I had bundled into “memory” could be separated.
AGENTS.mdcould carry instructions that must always apply.This division changed my understanding of Skills as well. During a separate discussion about sending some Codex engineering work to browser-based ChatGPT sessions to relieve local quota pressure, I initially wanted to “build a Skill” because the handoff protocol could not tolerate missing a step. I then objected to the word “build”: perhaps we should first discuss the purpose and boundaries, then fix the resulting procedure.
Even that placed discussion before evidence. In practice, recurring inputs, common omissions, safe discretion, costly exceptions, and real acceptance criteria appear only after a workflow has happened more than once. A pre-practice Skill may freeze a first prediction of the workflow rather than knowledge of the workflow.
My revised formulation was:
This is not an absolute ban on writing a Skill early. It is a claim about evidence order. Let real work generate successes, failures, detours, exceptions, and validation criteria. When a stable pattern appears, crystallize the non-negotiable parts into an executable contract. When subsequent work repeatedly exposes a gap, revise the contract with evidence.
OpenAI's current Skill format fits this division without proving it. A Skill packages instructions, resources, and optional scripts for a repeatable workflow. Its name and description are visible for discovery; the full
SKILL.mdis loaded only when selected.[1]Memory can remain the changing account of what has been learned. A Skill can become the versioned procedure that should now run reliably.The memory architecture itself was also not novel in its components. Generative Agents stored natural-language experience, retrieved it dynamically, and synthesized higher-level reflections for planning.[2]MemGPT framed tiered context as virtual memory, and Letta now exposes always-visible blocks alongside files, archival memory, and external retrieval.[3][4]LongMemEval evaluates information extraction, multi-session and temporal reasoning, knowledge updates, and abstention.[5]Zep and Graphiti represent changing facts, provenance, and temporal validity in a graph.[6]A-MEM dynamically links memories and lets new additions update older representations.[7]Mem0 extracts, consolidates, and retrieves salient conversational information while measuring deployment costs against full-context approaches.[8]
I did not begin by reviewing that literature and selecting an open research problem. I began with an inability to continue a conversation naturally. Over several days, failures pushed a local system toward problems that the broader field had already encountered in more systematic forms.
The point of presenting that history is not to claim a more advanced or generally better theory. The system pursued a different goal from many benchmarked memory architectures: carrying the development of one person's understanding across natural work while retaining some trace of how later views took responsibility from earlier ones. What this article offers is a record of how an ordinary user encountered the responsibilities hiding behind the word “memory”—and how engineering those responsibilities changed the user's view of the original problem.
The deletion happened before my direct inspection of the official local memory files. At deletion time, the evidence was asymmetric: the maintenance cost of the self-built system was real, while the sufficiency of the official system remained uncertain. Stopping was still rational because the system's production cost was known and its unique necessity was not.
The later inspection supplied additional evidence for the decision without turning it into a controlled experiment. The official system had already begun to capture reusable knowledge, failures, and evidence routes. It had not required the five-process, 5.53 GB local stack to keep doing so.
What I deleted was therefore not the record of the discussion, the code history, or the question of how knowledge changes. I deleted the self-built vector-memory system's role as default daily infrastructure. The natural-language memory material and Git history remained available for research and review.
If official memory later proves unable to recover the right project entry point, the next response need not be to resurrect the entire stack. The memory layer can remember which project matters, while the project book supplies the full state. Versioned documents can hold facts that must be exact. A thin audit trail can be added only if formation history becomes a demonstrated need.
This project left me with a stricter evaluation question. “Did the assistant remember me?” is too vague. Better questions are: What did it capture? What applicability boundary did it assign? Did new knowledge revise old knowledge? When did the memory enter the decision? And did the next similar task actually go differently?
The final question is the shortest:
OpenAI, “Build skills”. ↩︎
Joon Sung Park et al., “Generative Agents: Interactive Simulacra of Human Behavior”, 2023. ↩︎
Charles Packer et al., “MemGPT: Towards LLMs as Operating Systems”, 2023. ↩︎
Letta, “Context hierarchy”. ↩︎
Di Wu et al., “LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory”, 2024. ↩︎
Preston Rasmussen et al., “Zep: A Temporal Knowledge Graph Architecture for Agent Memory”, 2025. ↩︎
Wujiang Xu et al., “A-MEM: Agentic Memory for LLM Agents”, 2025. ↩︎
Prateek Chhikara et al., “Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory”, 2025. ↩︎
Afterword
I still believe that memory deserves to be treated as a module in its own right. In helping AI adapt better to people, and in enabling longer-term human–AI collaboration, memory may be an important layer. But that is precisely why engineering around memory should be approached cautiously. Memory is not only about what content is stored. It also involves how that content is formed, when it enters judgment, how it is updated, when it exits, and how it ultimately changes the next action.
With the local Codex memory I have encountered so far, I still feel that it has not implemented the part I care about most. It can extract and organize material from prior conversations and bring that material into future work; but in my ideal, memory would not be merely external material appended to a model. It would participate more deeply in how the model forms understanding and judgment. I suspect that this gap may have something to do with how memory is considered during model training: if the existence, updating, and use of memory were included in training-time design, perhaps a different capability would be more likely to emerge. But this is only my conjecture. OpenAI's public documentation on memory does not say whether, or how, memory was considered during training; I also cannot yet specify what capabilities an ideal memory module should have.
I am willing to accept that using LLM content blocks may cause my first submission to be rejected because I believe AI-generated and AI-organized prose can translate my personal way of expressing myself into language that more readers can understand. For me, honestly marking how AI participated in that translation matters more than hiding it to improve the probability of approval. If this mode of production does not meet the community's requirements for a first post, rejection is a consequence I am willing to accept as part of choosing this form of expression.