This is an automated rejection. No LLM generated, assisted/co-written, or edited work.
Read full explanation
Retrieval is the read path. Persistent agents also need rules for what becomes belief, what supersedes what, and what is allowed to be forgotten.
Epistemic status
I have spent roughly a year building production memory systems: a knowledge graph over the organisation's own history, and a structural map over its codebase. Everything below is sorted into three piles, and I would like the sorting to be visible rather than implied.
Observed in production. The asymmetry between reading and writing. What write decisions concretely look like when you have to make them. The co-occurrence deprecation in §3. The dependency-map finding in §4.
Working hypothesis. That uncued salience is a distinct failure mode rather than a special case of retrieval failure or under-specification. I believe this, I have not established it, and §5 names the experiment that would settle it.
Speculation. The compression story in §6 about why it happens. It is a behavioural description with a mechanism attached, and the mechanism is the weak part.
One clarification before anyone checks. One of the ingested corpora is LongMemEval. I used it as an ingestion target, not as a benchmark: I have no QA accuracy number for it, and it would be dishonest to let the name imply one. That gap is deliberate in the sense that I know it is there, and it is a real weakness — see open problem 8.
Anything here that reads like a general law is a hypothesis with a single supporting instance. I have tried to phrase things so that you can tell which is which.
There are two claims in this post that I would drop under stated conditions. They are in §1 and §5, marked.
1. There is a read path and a write path
A recent post here argued that prosaic memory — long context plus explicitly written-down documentation — may substitute for continual learning in the near term, and noted in passing that getting a model to write good memories is itself difficult: it has to know what it knows from context versus weights, and predict what a future instance will do with the note.1
I want to take that passing difficulty as the main event. In an organisational setting it is not a subproblem of continual learning. It is most of the work.
Here is the smallest illustration I have. Four statements about one refund policy, all true when written, all still sitting in the corpus:
2024-03, policy document: refunds within 30 days.
2025-01, revised policy document: refunds within 14 days.
2025-06, message from the head of support in a team channel: "we've been doing 30 for enterprise accounts anyway, nobody objected."
2026-02, a customer email promising 30 days, sent by someone who had read the first document.
Retrieval works perfectly here. Ask what is our refund policy and a competent system returns all four, correctly ranked, with sources. It has done its job.
Now answer the question. What is the policy? Which statement supersedes which? Does a practice adopted informally by the support lead override a document? Does the 2026 email create an obligation, and to whom? Is the third statement evidence about policy, or evidence about drift between policy and practice — and are those the same record?
None of that is a retrieval question. Retrieval hands you the material; something else has to decide what the organisation now believes.
Read path: given this task, what should the system know? Write path: given what just happened, what should the system remember, as what kind of object, superseding what, on whose authority, for how long?
I am not claiming the read path is solved. It clearly is not — "Du et al." hold retrieval fixed at perfect, mask everything except the relevant tokens so the model can only attend to them, and still see performance fall by 13.9% to 85% across model–task pairs at inputs well inside the advertised windows.2 (The top of that range is a synthetic arithmetic probe; the math, QA and code results are less dramatic and still clearly negative.) The read path has real unsolved problems.
My claim is comparative, and it is the one I would like argued with:
Claim. In organisational settings, write policy — not retrieval quality — is currently the binding constraint on whether accumulated context improves decisions.
What would show me wrong. Hold write policy fixed at something naive (append everything, no typing, no supersession, no provenance) and improve retrieval substantially: better embeddings, better reranking, better query decomposition, larger windows. If most of the available gain in downstream decision quality comes from the retrieval side, the asymmetry I am describing is not real and I am overfitting to the systems I happened to build. I do not have that experiment. I would like someone to run it.
That experiment is hard for a boring reason — "downstream decision quality" is expensive to measure — but it is not conceptually hard, and I would update on it.
2. A write is an epistemic operation
In a database, a write is a transaction: it either lands or it does not. In an organisational memory, a write changes what the system is prepared to assert. Those are different kinds of operation and they need different machinery.
The first thing that stopped being optional for us was distinguishing three object types — not three confidence levels on one object, three genuinely different things:
Evidence. Someone said this, here, at this time. It is true that they said it. Whether what they said is true is a separate question.
Interpretation. Someone, or something, concluded this from evidence. It has an author and a basis, and it can be wrong while every piece of evidence under it stays correct.
Accepted truth. The organisation currently operates as if this holds. It has an authority behind it and a scope.
Collapsing these is the most common failure I have seen, and it is invisible until it is expensive. A system that stores "the refund window is 30 days" without recording whether that is a document, an inference, or an operating decision cannot later tell you why it believes it, and cannot be corrected except by hand.
Once the types exist, every durable write has to answer a list: what kind of object is this, what is it based on, who or what asserted it, over what scope, superseding what, until when, what would retract it — and, a field I did not expect to need, what produced this record: which model, which prompt version, which settings.
These look like schema fields. They are not. They are rules about how a system is permitted to change its beliefs, and choosing them is a design decision that no embedding model makes for you.
What that looks like once it is built
Concretely: in our system nothing enters memory as a fact. It enters as a concept hypothesis — a typed object carrying a state, a confidence, a set of aliases, a revision count, and a list of evidence spans that each point back to a specific message in a specific source.
The word is load-bearing. A hypothesis is something that can be revised without the system having lied, and revision_count is a field you can sort by when you want to know what your memory is least sure of.
Writes are then not insertions. They are one of four operations against what is already held:
create — this is genuinely new
attach — this is further evidence for something already held
refine — this changes the description, aliases or confidence of something weakly held
reject — this does not become memory
The interesting one is reject, and specifically that it does not write nothing. Rejected candidates go to a residuals pool. You can go back later and ask what the system decided not to believe, which is the only practical way to discover that your write policy has been systematically discarding a whole category of thing.
The volume, incidentally, is mostly attach. On one corpus of 3,253 conversations, 34,805 extracted mentions consolidated to 7,339 canonical concepts. Roughly four out of five candidate writes were the same thing said again. That ratio is what the write path mostly consists of in practice: not deciding what is important, but deciding what is not new.
And the operations that can destroy information — merge, split, archive — are not available inline at all. They are generated by a separate offline pass, land in a staging table as proposals, and are applied by a second bounded process that records a rationale per action. That is promotion authority, implemented: the system may propose a change to its own memory, and something else decides.
I should be honest about how far this goes, because I was asked in public and gave a worse answer than the question deserved. You cannot design every write decision in advance; we do not. What you can specify is the contract — the type system, what counts as evidence, who may promote — and the reversibility: our ontology is only ever modified in a mode where every step can be undone, with checkpoints, so that a bad run is a rollback rather than an archaeology project. Inside those two constraints the model gets latitude, and we look at what comes out. That is a weaker claim than "specify your write policy," and it is the true one.
An aside, because it surprised me
The extraction step — the thing that turns a paragraph of conversation into a typed object with a confidence and an evidence span — is where write policy actually executes. We tested it across roughly 300 prompt variants at different reasoning-effort settings, expecting that more deliberation would produce better structure.
It produced worse structure. At high effort the model stopped extracting to the schema and started trying to improve it: renaming types it judged imprecise, collapsing fields, adding structure nobody asked for, returning objects that failed validation. Constraining effort and moving the interpretive work into a separate downstream pass cut extraction tokens by about 48% and raised our internal quality score from 5.2 to 7.11.
Caveats, because this is exactly the kind of number that deserves them: the scale is ours, the judge is ours, and none of it has been run against a public benchmark. Treat it as an observation, not a result.
The shape is what interests me. A schema for a durable write is a contract, and the capability that makes a model good at open-ended reasoning appears to be the same capability that makes it a poor executor of contracts — it has been trained to improve on what it is given, and here what it is given is not a suggestion. If that generalises at all, it argues that the write path wants a different kind of model call than the read path: narrow, constrained, and deliberately boring. Which runs directly against the current instinct to spend more reasoning on every step.
There is a second-order version of this that I think is genuinely unaddressed. The thing executing your write policy is a model, and models change. Upgrade, and a record your old model classified as high-risk your new one calls mild — not because the evidence changed, but because the judgement function did. Your memory now contains a stratum written by an evaluator that no longer exists, and nothing in it is marked as such.
Our partial answer is a golden dataset: a fixed set of inputs run through every candidate model, prompt and setting, with the outputs compared. It catches gross drift. It does not tell you what to do about the two years of beliefs already written by the old configuration, and I do not have a good answer to that. Re-extracting everything is expensive and produces a different memory, which is its own problem — you have now silently rewritten history. Leaving it produces a store whose meaning depends on when each part of it was written.
This is why the provenance field above includes the model. It does not solve anything. It at least makes the stratum visible.
The strongest objection to this section
I want to state it properly, because it comes from the best current work rather than from a sceptic.
APEX-MEM is a main-conference ACL 2026 system with the best published numbers I know of on long-term conversational memory — 88.88% on LOCOMO QA, 86.2% on LongMemEval. It is deliberately append-only. It preserves the full temporal evolution of information and resolves conflicting or superseded material at query time, with a multi-tool retrieval agent, rather than deciding at write time what supersedes what.3
That is the opposite of what I have just argued. A system that does no write-time resolution at all is outperforming systems that do.
I think the honest reading is this. APEX-MEM's corpus is one user's own conversational history: bounded, single-provenance, non-adversarial, and cheap to keep in full. Under those conditions "append everything and sort it out on read" is not a compromise, it is correct — you get supersession for free by keeping timestamps, and there is no authority question because there is only one author.
The organisational case breaks all three assumptions. The corpus is unbounded and grows monotonically. Provenance and authority differ per source, so "which statement is later" does not determine "which statement governs" — the head of support's channel message is more recent than the policy document and has less authority over it. And a corpus that keeps everything makes every future read pay the full cost of every past write, in a setting where the read is performed by a model whose accuracy degrades with input length. There is also a structural gap: append-only-with-timestamps gives you one time axis, and organisational memory needs two — see the next paragraph but one.
So I take the append-only result seriously as evidence that the boundary between write-time and read-time resolution is a live engineering question rather than a settled one. I do not take it as evidence that write policy is free. If someone demonstrates a comparable append-only result on a multi-author corpus with genuine authority conflicts, that is a substantial hit to this post.
Two things that are usually treated as features and are actually write policy
Disagreement. The default move is to reconcile: four teams hold incompatible views of why a launch underperformed, the system stores the consensus. But consensus is lossy compression, and the thing it compresses away is precisely the information that the organisation does not know. A record that says marketing believes X, engineering believes Y, and nobody has run the test that would separate them is more useful than a smoothed summary, and it is a write-policy decision whether such a record can exist at all.
Forgetting. A memory that only accumulates is not a memory. Supersession, expiry and decay are design requirements, not maintenance chores — and they force a distinction that is easy to miss: historical truth and operational truth are different objects. We believed X in 2024 must remain retrievable forever. X must stop being returned as current the moment it is superseded. A system that keeps only the second has destroyed its ability to explain itself; one that keeps only the first will confidently tell you last year's policy.
One data point I did not expect, from someone who took the other side. A friend who has spent ten to fifteen years accumulating structured data across roughly twenty companies — exactly the archive this whole field says you should be building — told me flatly that most of it is worthless to him. User behaviour changed, the competitive techniques changed, the market changed. He uses a few years. The rest is not a dormant asset waiting for a better model; it is a description of a world that no longer exists, and it is actively misleading when retrieved without that caveat. He is one person and this is an anecdote, but it is an anecdote from the one population whose opinion is worth most here: people who already did the accumulating.
The standard answer is bi-temporal modelling, borrowed wholesale from temporal databases: every fact carries a valid time — the period it was true in the world — and a system time — when we recorded or revised it. Two axes rather than one. Current state is the fact with no valid-until. What was true in March is a historical query. What did we believe in March is a different query, it is answered on the other axis, and it is the one you need after something has gone wrong. The shortest illustration I have found: who is the CTO and who was the CTO when this decision was made are not the same question, and a memory with one time axis cannot reliably tell them apart. Append-only with a single timestamp collapses these two into each other, which is fine when there is one author and no way to be wrong about provenance, and not fine otherwise.
3. "These two things co-occurred" is not knowledge
The most concrete write decision I can show you is a deletion.
Our graph had a relation type called CO_OCCURS_IN_ASSET_HANDOFF. It meant, roughly: these two entities keep showing up together in the same handoff context. It was easy to generate, it produced a lot of edges, and the edges looked like knowledge.
They were not. Two things appearing near each other is not a reason to believe they are related. It is a reason to look. The type was encoding a search heuristic as though it were a finding, and once it is in the graph nothing downstream can tell the difference — a path query traverses a co-occurrence edge exactly as it traverses an asserted one.
The type was not retired by a person. It was retired by the system's own maintenance process.
A background agent whose job is to work over the graph and propose revisions to the ontology — merges, splits, retirements — flagged CO_OCCURS_IN_ASSET_HANDOFF as a type that was producing edges without carrying information. As with every proposal of that kind, it did not apply itself: it landed in a staging table and was applied by a separate bounded process that records a rationale per action. Surviving edges: zero.
What replaced it was an explicit list of what may serve as the basis for a relation:
direct assertion — someone stated it
structural — the structure of the artefact implies it
causal — one brought about the other
functional — one serves or depends on the other
temporal — one preceded the other, with that ordering being the claim
Proximity is not on the list, deliberately.
I want to be careful here, because the obvious reading is wrong in both directions.
It is not "the model solved the write problem." The agent could only make that proposal because someone had already made what counts as evidence an explicit, inspectable, revisable object in the system. In most pipelines co-occurrence is not a policy at all — it is a line of extraction code, and there is nothing there to argue with. The agent revised a policy. It did not invent the idea that there should be one, and it did not choose the five bases that replaced it.
But it is not nothing either, and it is the part I would most like to be true. Once write policy is represented explicitly rather than buried in code, it becomes something the system can examine, criticise, and propose changes to — with the destructive half of the operation held behind a separate process. That is a materially different situation from write policy living in a pipeline, where the only mechanism for change is a person happening to notice.
So: a small, dated, reversible decision about what the system is permitted to treat as evidence — surfaced by the system, applied under rules a person wrote, and logged so it can be undone. That is the write path with every part of it visible at once. And note what could not have produced it. Better embeddings, a larger context window, a better reranker: all of those improve what you can find. None of them has an opinion about what you are allowed to believe. There is no retrieval improvement that repairs a graph full of edges that mean nothing.
There are, in that system, on the order of a hundred decisions of this shape — some made by people, some proposed the way this one was. Nobody sells a product that makes them.
4. A finding nobody queried for
The second system is a structural map of the codebase: modules, symbols, imports, call and data-flow edges, across all repositories. It is read-only and exposed to both people and agents.
The intended use was ordinary — impact analysis, dependency questions, onboarding. What it actually produced first was a finding nobody had asked for. Measuring coupling and centrality across the map surfaced that a particular piece of business logic was distributed across several domains that had no business sharing it. Several people half-knew this. Nobody had stated it, because from inside any one domain it looked like a local quirk. Two refactoring projects came out of the measurement.
The tempting reading is graphs are good, build a graph. I do not think that is the lesson, and the graph is not the interesting part.
The finding was available because of a much earlier decision about what the map would record: not files and folders, but symbol-level relations with typed edges, retained over time. That decision was made before anyone knew what question it would answer. A map that recorded imports at file granularity — a completely reasonable choice, cheaper, easier to maintain — could have been queried perfectly and would have contained no trace of this.
The value was not in the query. It was in the earlier decision about what to record and at what granularity. Get that wrong and you can build a perfectly accurate, perfectly retrievable map of the wrong thing.
I want this section to do one specific job: it is the structural version of the problem in §5. Here the thing nobody asked about was surfaced by measurement over a chosen representation. In §5 the material is prose, there is no structure to measure, and the same job has to be done by judgement. That is why it is harder.
5. Uncued salience
This is the part I am least sure of and most interested in being wrong about.
An agent is asked to analyse three months of declining campaign performance and explain the drop. It produces a good answer: creative fatigue, audience saturation, rising acquisition costs. All three are real, all three are visible in the data, all three are in the report.
In the corpus there is also a compliance note from week two. One line, saying that a specific claim in the ad copy had to be reworded for regulatory reasons.
That single change may account for more of the decline than the three named causes combined. The note was in the context. The model read it. It ranked it as peripheral and compressed it out.
The distinction I want:
Cued: find X. Given a description of the target, locate it. Models are very good at this and getting better. Uncued: decide, without a description of the target, that this item is the one that matters.
Two objections arrive immediately and they are not equally strong.
"This is Lost in the Middle."
It is not. Liu et al. document a positional effect: relevant information is used less reliably when it sits in the middle of a long context than at either end.4 That is a real and well-established finding about where an item is.
This failure is not positional. It persists when the compliance note is on the first page, in bold, three sentences from the top. What determines whether it survives is not its position in the window but its fit with the explanatory story the synthesis is assembling. A note about regulatory wording does not look like a performance-analysis input, so it is classified as background — from any position.
The two effects compound, and I would expect position to make this worse. But fixing position would not fix this.
"This is just under-specification. You asked for causes, the compliance note is a cause, so the model failed a badly-stated task."
This is the strong objection and I think it is wrong in two separate ways.
First: the specification cannot be completed. To rule this out by prompting, you would have to enumerate the categories of thing that might turn out to be decisive — check whether legal altered the claim copy; check whether a key person left mid-quarter; check whether the tracking pixel broke; check whether a competitor changed price. The set is open-ended, and it is open-ended in a way that is not incidental: the whole value of the analysis is finding the cause you did not think to name. "Specify better" is not a strategy here. It is a restatement of the problem in the imperative mood.
Second, and this is the part I care about: the failure produced a coherent, defensible answer. A retrieval failure is legible — the output is visibly missing something, cites nothing, or contradicts a known fact. This produces a complete report with three well-evidenced causes, correct methodology, and a plausible narrative. Nothing at the output layer indicates a problem. The only way to detect it is to already know the answer.
Which is why it belongs in a post about the write path rather than a post about analysis quality:
Failures that look like successes are the ones that get written down.
A visibly incomplete answer gets rejected and never enters durable memory. A coherent answer with a missing decisive cause gets accepted, summarised, and stored as what the organisation now knows about that quarter. The error does not just occur — it is promoted. And every future decision that retrieves this record inherits it, including decisions made by people who never read the original data and have no way to know what was dropped.
That is the join between this section and the rest of the post. Uncued salience is not primarily an analysis problem. It is a contamination path into durable belief.
The measurement problem
I do not have a benchmark, and I want to state the difficulty correctly, because my first attempt at stating it was wrong. The obstacle is not label leakage. Train/test/validation splits handle leakage — that is what they are for — and if that were the problem, this would be a solved engineering question rather than an open one.
The obstacle is that there is no defensible label to produce in the first place.
For cued retrieval, ground truth is objective and cheap: the answer is in document 4,312, and any two annotators agree. For salience you would have to label something like this fact accounts for a large part of the explanation, and that quantity has three properties that stop it being a label at all.
It is counterfactual, and the counterfactual cannot be run.Would the conclusion have been different? requires re-running the quarter without the compliance note. You cannot re-run a quarter.
It is not a property of the fact. Importance exists in the fact together with the rest of the corpus and the particular question asked. Add one document and the weight changes. So it is not an attribute you can attach to an item and carry around — it is a function over the whole set. The corpus as a whole determines the answer, while the individual contribution of any one fact is not merely unmeasured but not independently well-defined.
Hindsight contaminates the annotator. Once the outcome is known, the decisive fact looks obvious — and its whole character at the time was that it looked like nothing. That bias sits inside the labelling process, and no split removes it.
There is exactly one escape and it carries a matching cost. You can construct a synthetic corpus where ground truth holds by construction: you inserted the decisive fact, so you know it is decisive. But the construction is the cue. An inserted fact has a fingerprint — it is the one that was written in order to matter — and systems learn to find inserted things. The messy, plausible, everything-looks-routine texture that makes this hard in a real corpus is precisely what a constructed one lacks.
So the dilemma. A real corpus has no trustworthy salience labels. A synthetic corpus has labels that are trustworthy only because they were planted.
The least-bad direction I can see runs between those horns: it is retrospective — take real decisions with known outcomes, reconstruct what was available at the time, and score whether a system surfaces the factor that later turned out to matter. That is expensive, domain-specific, hard to label without hindsight bias, and has a small sample size by construction. I would still rather have fifty of those than fifty thousand synthetic needles.
Second update condition
If a single general instruction — appended to any analysis prompt: "also list anything in the material that would change your conclusion if it turned out to be significant, including things that do not look related" — closes most of the gap on held-out real cases, then this is a prompting problem, not a distinct failure mode, and I am wrong about it. That experiment is cheap. I have run it informally and been encouraged, which is close to worthless as evidence. If someone runs it properly, I would like to know either way.
6. Compression on the write path
The mechanism I suspect, stated with the discipline it deserves.
I am not claiming the model "picks a story first and then filters." That is a claim about internals and I have no evidence for it. The defensible version is behavioural:
A synthesis process can produce a coherent, well-supported answer while systematically underweighting information that does not fit the dominant explanation.
That is enough. Summarisation optimises for central tendency — it is supposed to; that is what makes a summary a summary. For most documents, discarding the material that does not fit the main thread is exactly correct. For business analysis it is backwards, because the decision-relevant information disproportionately lives in the exceptions. The average of a quarter tells you very little you did not already believe. The anomaly tells you what to do differently.
And here is the asymmetry that makes it a write-path problem rather than a read-path one:
On the read path, a compression error is recoverable. The summary is wrong, the source is still there, and the next query can go back to it. On the write path, the compressed output is the record. If the exception never entered durable memory, there is nothing to go back to. Same error, different permanence.
If summaries decide what enters organisational memory, the organisation accumulates a clean, coherent, progressively less useful record — one optimised for exactly the decisions that did not need it.
7. Persistent memory moves the security boundary
Briefly, because this is discussed here already and I only want to add the write-side framing.
A manipulated prompt against a stateless model produces one bad output. The session ends and the damage stops. The same manipulation against a system with durable memory produces a bad record, which is retrieved later, in a different session, by a different user, in support of a decision nobody connects to the original interaction.
Two results mark the shape of this. PoisonedRAG achieves a 90% success rate on targeted attacks — attacker-chosen question, attacker-chosen answer — by injecting five malicious texts into a knowledge database of millions.5 The ratio is the point: the defence budget scales with the corpus, the attack budget does not. And MINJA shows memory records can be planted through ordinary queries alone, with no access to the memory bank and no ability to modify the victim's queries, reaching 76.8% average attack success with a 98.2% injection rate.6 MINJA assumes a shared memory bank — the attacker's records must be retrievable by another user's session — which is a real constraint and is also precisely the architecture this post argues organisations should build.
I want to be concrete about where this actually bites, rather than staying with invented examples. We ingest millions of customer feedback messages into a system that produces insights for product, logistics and marketing. That is untrusted, user-authored text on a direct path into durable organisational belief, at a volume where nothing is read by a person. If prompt injections are already in that stream — and at those volumes I would assume they are — I currently have no reliable way to detect them. This is not a solved part of my system. It is a known open flank, and I would rather say so than describe a defence I have not built.
What I want to add is that the mitigations are all write-path primitives, and they are the same primitives §2 arrived at for entirely non-security reasons: provenance, authenticated sources, promotion authority (who or what may turn a candidate into an accepted truth), a candidate/accepted distinction at all, conflict detection, rollback, expiry, audit.
Rollback is the one that shows the connection most clearly, because it is only meaningful if system time was recorded separately from valid time. Revert every belief acquired through this source after this date is a query if you have two time axes and a research project if you have one. The bi-temporal design in §2 was not built for security. It is what makes the security response possible.
That convergence is the interesting part. A memory system built to know why it believes things is, incidentally, a memory system that can be told what it is not allowed to believe.
If an agent can act on remembered information, memory governance is action governance.
Open problems
The eight I would most like someone else to work on.
1. Benchmarking uncued salience without cueing it. The core difficulty in §5. Any labelled dataset has already pointed at the answer. Retrospective evaluation on real decisions is the least-bad direction I know and it is expensive and small-n. I think there is a cleverer construction and I have not found it.
2. Memory promotion. The pipeline from observation → candidate lesson → durable belief. What evidence threshold, whose authority, how many independent instances, and what happens to everything downstream when a promoted belief is later retracted.
3. Credit assignment. Connecting an outcome back to the decision that produced it, and to the specific piece of remembered context that shaped the decision, across weeks or months and multiple intervening causes. Without this, "did memory help?" is unanswerable and every memory system is sold on vibes.
4. Adversarial writes to long-lived belief state. The literature is mostly about retrieval-time attacks. The harder version is slow corruption of a belief store over months, by an attacker who is patient and whose individual contributions are each locally plausible.
5. Non-stationary write executors. Memory is written by a model, and the model changes. What does a belief store mean when different strata of it were produced by different judgement functions, and what is the correct operation on upgrade — re-extract and lose the original record, or keep it and accept that meaning varies by write date? Golden datasets detect the drift. Nothing I know of tells you what to do next.
6. The observed corpus. Once people know their messages are being written into a system that agents will act on, the messages change. In my experience most people simply do not think about it — but "most" is not a strategy, and the failure mode is not privacy, it is selection: the corpus becomes a record of what people were willing to have recorded. I do not know how large this effect is and I have not seen it measured.
7. Decay and supersession policy. When should a belief expire on its own? What is the right default half-life for different object types, and how do you keep historical truth queryable while stopping it being returned as current?
8. Outcome-grounded evaluation. Does accumulated organisational memory measurably improve later decisions, compared against a matched organisation that has none? Nobody has shown this. It is the load-bearing empirical question under this entire area, including under my own work, and it currently rests on plausibility.
An enormous amount of effort has gone into making models better at reading memory, and it has worked. The unsolved half is deciding what they are allowed to remember — and in an organisation, that is not a storage question. It is a question about what the company is permitted to believe, and it has to be answered by someone before any model gets to read anything.
Retrieval is the read path. Persistent agents also need rules for what becomes belief, what supersedes what, and what is allowed to be forgotten.
Epistemic status
I have spent roughly a year building production memory systems: a knowledge graph over the organisation's own history, and a structural map over its codebase. Everything below is sorted into three piles, and I would like the sorting to be visible rather than implied.
Observed in production. The asymmetry between reading and writing. What write decisions concretely look like when you have to make them. The co-occurrence deprecation in §3. The dependency-map finding in §4.
Working hypothesis. That uncued salience is a distinct failure mode rather than a special case of retrieval failure or under-specification. I believe this, I have not established it, and §5 names the experiment that would settle it.
Speculation. The compression story in §6 about why it happens. It is a behavioural description with a mechanism attached, and the mechanism is the weak part.
One clarification before anyone checks. One of the ingested corpora is LongMemEval. I used it as an ingestion target, not as a benchmark: I have no QA accuracy number for it, and it would be dishonest to let the name imply one. That gap is deliberate in the sense that I know it is there, and it is a real weakness — see open problem 8.
Anything here that reads like a general law is a hypothesis with a single supporting instance. I have tried to phrase things so that you can tell which is which.
There are two claims in this post that I would drop under stated conditions. They are in §1 and §5, marked.
1. There is a read path and a write path
A recent post here argued that prosaic memory — long context plus explicitly written-down documentation — may substitute for continual learning in the near term, and noted in passing that getting a model to write good memories is itself difficult: it has to know what it knows from context versus weights, and predict what a future instance will do with the note.1
I want to take that passing difficulty as the main event. In an organisational setting it is not a subproblem of continual learning. It is most of the work.
Here is the smallest illustration I have. Four statements about one refund policy, all true when written, all still sitting in the corpus:
Retrieval works perfectly here. Ask what is our refund policy and a competent system returns all four, correctly ranked, with sources. It has done its job.
Now answer the question. What is the policy? Which statement supersedes which? Does a practice adopted informally by the support lead override a document? Does the 2026 email create an obligation, and to whom? Is the third statement evidence about policy, or evidence about drift between policy and practice — and are those the same record?
None of that is a retrieval question. Retrieval hands you the material; something else has to decide what the organisation now believes.
I am not claiming the read path is solved. It clearly is not — "Du et al." hold retrieval fixed at perfect, mask everything except the relevant tokens so the model can only attend to them, and still see performance fall by 13.9% to 85% across model–task pairs at inputs well inside the advertised windows.2 (The top of that range is a synthetic arithmetic probe; the math, QA and code results are less dramatic and still clearly negative.) The read path has real unsolved problems.
My claim is comparative, and it is the one I would like argued with:
That experiment is hard for a boring reason — "downstream decision quality" is expensive to measure — but it is not conceptually hard, and I would update on it.
2. A write is an epistemic operation
In a database, a write is a transaction: it either lands or it does not. In an organisational memory, a write changes what the system is prepared to assert. Those are different kinds of operation and they need different machinery.
The first thing that stopped being optional for us was distinguishing three object types — not three confidence levels on one object, three genuinely different things:
Collapsing these is the most common failure I have seen, and it is invisible until it is expensive. A system that stores "the refund window is 30 days" without recording whether that is a document, an inference, or an operating decision cannot later tell you why it believes it, and cannot be corrected except by hand.
Once the types exist, every durable write has to answer a list: what kind of object is this, what is it based on, who or what asserted it, over what scope, superseding what, until when, what would retract it — and, a field I did not expect to need, what produced this record: which model, which prompt version, which settings.
These look like schema fields. They are not. They are rules about how a system is permitted to change its beliefs, and choosing them is a design decision that no embedding model makes for you.
What that looks like once it is built
Concretely: in our system nothing enters memory as a fact. It enters as a concept hypothesis — a typed object carrying a state, a confidence, a set of aliases, a revision count, and a list of evidence spans that each point back to a specific message in a specific source.
The word is load-bearing. A hypothesis is something that can be revised without the system having lied, and
revision_countis a field you can sort by when you want to know what your memory is least sure of.Writes are then not insertions. They are one of four operations against what is already held:
The interesting one is
reject, and specifically that it does not write nothing. Rejected candidates go to a residuals pool. You can go back later and ask what the system decided not to believe, which is the only practical way to discover that your write policy has been systematically discarding a whole category of thing.The volume, incidentally, is mostly
attach. On one corpus of 3,253 conversations, 34,805 extracted mentions consolidated to 7,339 canonical concepts. Roughly four out of five candidate writes were the same thing said again. That ratio is what the write path mostly consists of in practice: not deciding what is important, but deciding what is not new.And the operations that can destroy information — merge, split, archive — are not available inline at all. They are generated by a separate offline pass, land in a staging table as proposals, and are applied by a second bounded process that records a rationale per action. That is promotion authority, implemented: the system may propose a change to its own memory, and something else decides.
I should be honest about how far this goes, because I was asked in public and gave a worse answer than the question deserved. You cannot design every write decision in advance; we do not. What you can specify is the contract — the type system, what counts as evidence, who may promote — and the reversibility: our ontology is only ever modified in a mode where every step can be undone, with checkpoints, so that a bad run is a rollback rather than an archaeology project. Inside those two constraints the model gets latitude, and we look at what comes out. That is a weaker claim than "specify your write policy," and it is the true one.
An aside, because it surprised me
The extraction step — the thing that turns a paragraph of conversation into a typed object with a confidence and an evidence span — is where write policy actually executes. We tested it across roughly 300 prompt variants at different reasoning-effort settings, expecting that more deliberation would produce better structure.
It produced worse structure. At high effort the model stopped extracting to the schema and started trying to improve it: renaming types it judged imprecise, collapsing fields, adding structure nobody asked for, returning objects that failed validation. Constraining effort and moving the interpretive work into a separate downstream pass cut extraction tokens by about 48% and raised our internal quality score from 5.2 to 7.11.
Caveats, because this is exactly the kind of number that deserves them: the scale is ours, the judge is ours, and none of it has been run against a public benchmark. Treat it as an observation, not a result.
The shape is what interests me. A schema for a durable write is a contract, and the capability that makes a model good at open-ended reasoning appears to be the same capability that makes it a poor executor of contracts — it has been trained to improve on what it is given, and here what it is given is not a suggestion. If that generalises at all, it argues that the write path wants a different kind of model call than the read path: narrow, constrained, and deliberately boring. Which runs directly against the current instinct to spend more reasoning on every step.
There is a second-order version of this that I think is genuinely unaddressed. The thing executing your write policy is a model, and models change. Upgrade, and a record your old model classified as high-risk your new one calls mild — not because the evidence changed, but because the judgement function did. Your memory now contains a stratum written by an evaluator that no longer exists, and nothing in it is marked as such.
Our partial answer is a golden dataset: a fixed set of inputs run through every candidate model, prompt and setting, with the outputs compared. It catches gross drift. It does not tell you what to do about the two years of beliefs already written by the old configuration, and I do not have a good answer to that. Re-extracting everything is expensive and produces a different memory, which is its own problem — you have now silently rewritten history. Leaving it produces a store whose meaning depends on when each part of it was written.
This is why the provenance field above includes the model. It does not solve anything. It at least makes the stratum visible.
The strongest objection to this section
I want to state it properly, because it comes from the best current work rather than from a sceptic.
APEX-MEM is a main-conference ACL 2026 system with the best published numbers I know of on long-term conversational memory — 88.88% on LOCOMO QA, 86.2% on LongMemEval. It is deliberately append-only. It preserves the full temporal evolution of information and resolves conflicting or superseded material at query time, with a multi-tool retrieval agent, rather than deciding at write time what supersedes what.3
That is the opposite of what I have just argued. A system that does no write-time resolution at all is outperforming systems that do.
I think the honest reading is this. APEX-MEM's corpus is one user's own conversational history: bounded, single-provenance, non-adversarial, and cheap to keep in full. Under those conditions "append everything and sort it out on read" is not a compromise, it is correct — you get supersession for free by keeping timestamps, and there is no authority question because there is only one author.
The organisational case breaks all three assumptions. The corpus is unbounded and grows monotonically. Provenance and authority differ per source, so "which statement is later" does not determine "which statement governs" — the head of support's channel message is more recent than the policy document and has less authority over it. And a corpus that keeps everything makes every future read pay the full cost of every past write, in a setting where the read is performed by a model whose accuracy degrades with input length. There is also a structural gap: append-only-with-timestamps gives you one time axis, and organisational memory needs two — see the next paragraph but one.
So I take the append-only result seriously as evidence that the boundary between write-time and read-time resolution is a live engineering question rather than a settled one. I do not take it as evidence that write policy is free. If someone demonstrates a comparable append-only result on a multi-author corpus with genuine authority conflicts, that is a substantial hit to this post.
Two things that are usually treated as features and are actually write policy
Disagreement. The default move is to reconcile: four teams hold incompatible views of why a launch underperformed, the system stores the consensus. But consensus is lossy compression, and the thing it compresses away is precisely the information that the organisation does not know. A record that says marketing believes X, engineering believes Y, and nobody has run the test that would separate them is more useful than a smoothed summary, and it is a write-policy decision whether such a record can exist at all.
Forgetting. A memory that only accumulates is not a memory. Supersession, expiry and decay are design requirements, not maintenance chores — and they force a distinction that is easy to miss: historical truth and operational truth are different objects. We believed X in 2024 must remain retrievable forever. X must stop being returned as current the moment it is superseded. A system that keeps only the second has destroyed its ability to explain itself; one that keeps only the first will confidently tell you last year's policy.
One data point I did not expect, from someone who took the other side. A friend who has spent ten to fifteen years accumulating structured data across roughly twenty companies — exactly the archive this whole field says you should be building — told me flatly that most of it is worthless to him. User behaviour changed, the competitive techniques changed, the market changed. He uses a few years. The rest is not a dormant asset waiting for a better model; it is a description of a world that no longer exists, and it is actively misleading when retrieved without that caveat. He is one person and this is an anecdote, but it is an anecdote from the one population whose opinion is worth most here: people who already did the accumulating.
The standard answer is bi-temporal modelling, borrowed wholesale from temporal databases: every fact carries a valid time — the period it was true in the world — and a system time — when we recorded or revised it. Two axes rather than one. Current state is the fact with no valid-until. What was true in March is a historical query. What did we believe in March is a different query, it is answered on the other axis, and it is the one you need after something has gone wrong. The shortest illustration I have found: who is the CTO and who was the CTO when this decision was made are not the same question, and a memory with one time axis cannot reliably tell them apart. Append-only with a single timestamp collapses these two into each other, which is fine when there is one author and no way to be wrong about provenance, and not fine otherwise.
3. "These two things co-occurred" is not knowledge
The most concrete write decision I can show you is a deletion.
Our graph had a relation type called
CO_OCCURS_IN_ASSET_HANDOFF. It meant, roughly: these two entities keep showing up together in the same handoff context. It was easy to generate, it produced a lot of edges, and the edges looked like knowledge.They were not. Two things appearing near each other is not a reason to believe they are related. It is a reason to look. The type was encoding a search heuristic as though it were a finding, and once it is in the graph nothing downstream can tell the difference — a path query traverses a co-occurrence edge exactly as it traverses an asserted one.
The type was not retired by a person. It was retired by the system's own maintenance process.
A background agent whose job is to work over the graph and propose revisions to the ontology — merges, splits, retirements — flagged
CO_OCCURS_IN_ASSET_HANDOFFas a type that was producing edges without carrying information. As with every proposal of that kind, it did not apply itself: it landed in a staging table and was applied by a separate bounded process that records a rationale per action. Surviving edges: zero.What replaced it was an explicit list of what may serve as the basis for a relation:
Proximity is not on the list, deliberately.
I want to be careful here, because the obvious reading is wrong in both directions.
It is not "the model solved the write problem." The agent could only make that proposal because someone had already made what counts as evidence an explicit, inspectable, revisable object in the system. In most pipelines co-occurrence is not a policy at all — it is a line of extraction code, and there is nothing there to argue with. The agent revised a policy. It did not invent the idea that there should be one, and it did not choose the five bases that replaced it.
But it is not nothing either, and it is the part I would most like to be true. Once write policy is represented explicitly rather than buried in code, it becomes something the system can examine, criticise, and propose changes to — with the destructive half of the operation held behind a separate process. That is a materially different situation from write policy living in a pipeline, where the only mechanism for change is a person happening to notice.
So: a small, dated, reversible decision about what the system is permitted to treat as evidence — surfaced by the system, applied under rules a person wrote, and logged so it can be undone. That is the write path with every part of it visible at once. And note what could not have produced it. Better embeddings, a larger context window, a better reranker: all of those improve what you can find. None of them has an opinion about what you are allowed to believe. There is no retrieval improvement that repairs a graph full of edges that mean nothing.
There are, in that system, on the order of a hundred decisions of this shape — some made by people, some proposed the way this one was. Nobody sells a product that makes them.
4. A finding nobody queried for
The second system is a structural map of the codebase: modules, symbols, imports, call and data-flow edges, across all repositories. It is read-only and exposed to both people and agents.
The intended use was ordinary — impact analysis, dependency questions, onboarding. What it actually produced first was a finding nobody had asked for. Measuring coupling and centrality across the map surfaced that a particular piece of business logic was distributed across several domains that had no business sharing it. Several people half-knew this. Nobody had stated it, because from inside any one domain it looked like a local quirk. Two refactoring projects came out of the measurement.
The tempting reading is graphs are good, build a graph. I do not think that is the lesson, and the graph is not the interesting part.
The finding was available because of a much earlier decision about what the map would record: not files and folders, but symbol-level relations with typed edges, retained over time. That decision was made before anyone knew what question it would answer. A map that recorded imports at file granularity — a completely reasonable choice, cheaper, easier to maintain — could have been queried perfectly and would have contained no trace of this.
I want this section to do one specific job: it is the structural version of the problem in §5. Here the thing nobody asked about was surfaced by measurement over a chosen representation. In §5 the material is prose, there is no structure to measure, and the same job has to be done by judgement. That is why it is harder.
5. Uncued salience
This is the part I am least sure of and most interested in being wrong about.
An agent is asked to analyse three months of declining campaign performance and explain the drop. It produces a good answer: creative fatigue, audience saturation, rising acquisition costs. All three are real, all three are visible in the data, all three are in the report.
In the corpus there is also a compliance note from week two. One line, saying that a specific claim in the ad copy had to be reworded for regulatory reasons.
That single change may account for more of the decline than the three named causes combined. The note was in the context. The model read it. It ranked it as peripheral and compressed it out.
The distinction I want:
Two objections arrive immediately and they are not equally strong.
"This is Lost in the Middle."
It is not. Liu et al. document a positional effect: relevant information is used less reliably when it sits in the middle of a long context than at either end.4 That is a real and well-established finding about where an item is.
This failure is not positional. It persists when the compliance note is on the first page, in bold, three sentences from the top. What determines whether it survives is not its position in the window but its fit with the explanatory story the synthesis is assembling. A note about regulatory wording does not look like a performance-analysis input, so it is classified as background — from any position.
The two effects compound, and I would expect position to make this worse. But fixing position would not fix this.
"This is just under-specification. You asked for causes, the compliance note is a cause, so the model failed a badly-stated task."
This is the strong objection and I think it is wrong in two separate ways.
First: the specification cannot be completed. To rule this out by prompting, you would have to enumerate the categories of thing that might turn out to be decisive — check whether legal altered the claim copy; check whether a key person left mid-quarter; check whether the tracking pixel broke; check whether a competitor changed price. The set is open-ended, and it is open-ended in a way that is not incidental: the whole value of the analysis is finding the cause you did not think to name. "Specify better" is not a strategy here. It is a restatement of the problem in the imperative mood.
Second, and this is the part I care about: the failure produced a coherent, defensible answer. A retrieval failure is legible — the output is visibly missing something, cites nothing, or contradicts a known fact. This produces a complete report with three well-evidenced causes, correct methodology, and a plausible narrative. Nothing at the output layer indicates a problem. The only way to detect it is to already know the answer.
Which is why it belongs in a post about the write path rather than a post about analysis quality:
A visibly incomplete answer gets rejected and never enters durable memory. A coherent answer with a missing decisive cause gets accepted, summarised, and stored as what the organisation now knows about that quarter. The error does not just occur — it is promoted. And every future decision that retrieves this record inherits it, including decisions made by people who never read the original data and have no way to know what was dropped.
That is the join between this section and the rest of the post. Uncued salience is not primarily an analysis problem. It is a contamination path into durable belief.
The measurement problem
I do not have a benchmark, and I want to state the difficulty correctly, because my first attempt at stating it was wrong. The obstacle is not label leakage. Train/test/validation splits handle leakage — that is what they are for — and if that were the problem, this would be a solved engineering question rather than an open one.
The obstacle is that there is no defensible label to produce in the first place.
For cued retrieval, ground truth is objective and cheap: the answer is in document 4,312, and any two annotators agree. For salience you would have to label something like this fact accounts for a large part of the explanation, and that quantity has three properties that stop it being a label at all.
It is counterfactual, and the counterfactual cannot be run. Would the conclusion have been different? requires re-running the quarter without the compliance note. You cannot re-run a quarter.
It is not a property of the fact. Importance exists in the fact together with the rest of the corpus and the particular question asked. Add one document and the weight changes. So it is not an attribute you can attach to an item and carry around — it is a function over the whole set. The corpus as a whole determines the answer, while the individual contribution of any one fact is not merely unmeasured but not independently well-defined.
Hindsight contaminates the annotator. Once the outcome is known, the decisive fact looks obvious — and its whole character at the time was that it looked like nothing. That bias sits inside the labelling process, and no split removes it.
There is exactly one escape and it carries a matching cost. You can construct a synthetic corpus where ground truth holds by construction: you inserted the decisive fact, so you know it is decisive. But the construction is the cue. An inserted fact has a fingerprint — it is the one that was written in order to matter — and systems learn to find inserted things. The messy, plausible, everything-looks-routine texture that makes this hard in a real corpus is precisely what a constructed one lacks.
So the dilemma. A real corpus has no trustworthy salience labels. A synthetic corpus has labels that are trustworthy only because they were planted.
The least-bad direction I can see runs between those horns: it is retrospective — take real decisions with known outcomes, reconstruct what was available at the time, and score whether a system surfaces the factor that later turned out to matter. That is expensive, domain-specific, hard to label without hindsight bias, and has a small sample size by construction. I would still rather have fifty of those than fifty thousand synthetic needles.
Second update condition
6. Compression on the write path
The mechanism I suspect, stated with the discipline it deserves.
I am not claiming the model "picks a story first and then filters." That is a claim about internals and I have no evidence for it. The defensible version is behavioural:
That is enough. Summarisation optimises for central tendency — it is supposed to; that is what makes a summary a summary. For most documents, discarding the material that does not fit the main thread is exactly correct. For business analysis it is backwards, because the decision-relevant information disproportionately lives in the exceptions. The average of a quarter tells you very little you did not already believe. The anomaly tells you what to do differently.
And here is the asymmetry that makes it a write-path problem rather than a read-path one:
If summaries decide what enters organisational memory, the organisation accumulates a clean, coherent, progressively less useful record — one optimised for exactly the decisions that did not need it.
7. Persistent memory moves the security boundary
Briefly, because this is discussed here already and I only want to add the write-side framing.
A manipulated prompt against a stateless model produces one bad output. The session ends and the damage stops. The same manipulation against a system with durable memory produces a bad record, which is retrieved later, in a different session, by a different user, in support of a decision nobody connects to the original interaction.
Two results mark the shape of this. PoisonedRAG achieves a 90% success rate on targeted attacks — attacker-chosen question, attacker-chosen answer — by injecting five malicious texts into a knowledge database of millions.5 The ratio is the point: the defence budget scales with the corpus, the attack budget does not. And MINJA shows memory records can be planted through ordinary queries alone, with no access to the memory bank and no ability to modify the victim's queries, reaching 76.8% average attack success with a 98.2% injection rate.6 MINJA assumes a shared memory bank — the attacker's records must be retrievable by another user's session — which is a real constraint and is also precisely the architecture this post argues organisations should build.
I want to be concrete about where this actually bites, rather than staying with invented examples. We ingest millions of customer feedback messages into a system that produces insights for product, logistics and marketing. That is untrusted, user-authored text on a direct path into durable organisational belief, at a volume where nothing is read by a person. If prompt injections are already in that stream — and at those volumes I would assume they are — I currently have no reliable way to detect them. This is not a solved part of my system. It is a known open flank, and I would rather say so than describe a defence I have not built.
What I want to add is that the mitigations are all write-path primitives, and they are the same primitives §2 arrived at for entirely non-security reasons: provenance, authenticated sources, promotion authority (who or what may turn a candidate into an accepted truth), a candidate/accepted distinction at all, conflict detection, rollback, expiry, audit.
Rollback is the one that shows the connection most clearly, because it is only meaningful if system time was recorded separately from valid time. Revert every belief acquired through this source after this date is a query if you have two time axes and a research project if you have one. The bi-temporal design in §2 was not built for security. It is what makes the security response possible.
That convergence is the interesting part. A memory system built to know why it believes things is, incidentally, a memory system that can be told what it is not allowed to believe.
Open problems
The eight I would most like someone else to work on.
1. Benchmarking uncued salience without cueing it. The core difficulty in §5. Any labelled dataset has already pointed at the answer. Retrospective evaluation on real decisions is the least-bad direction I know and it is expensive and small-n. I think there is a cleverer construction and I have not found it.
2. Memory promotion. The pipeline from observation → candidate lesson → durable belief. What evidence threshold, whose authority, how many independent instances, and what happens to everything downstream when a promoted belief is later retracted.
3. Credit assignment. Connecting an outcome back to the decision that produced it, and to the specific piece of remembered context that shaped the decision, across weeks or months and multiple intervening causes. Without this, "did memory help?" is unanswerable and every memory system is sold on vibes.
4. Adversarial writes to long-lived belief state. The literature is mostly about retrieval-time attacks. The harder version is slow corruption of a belief store over months, by an attacker who is patient and whose individual contributions are each locally plausible.
5. Non-stationary write executors. Memory is written by a model, and the model changes. What does a belief store mean when different strata of it were produced by different judgement functions, and what is the correct operation on upgrade — re-extract and lose the original record, or keep it and accept that meaning varies by write date? Golden datasets detect the drift. Nothing I know of tells you what to do next.
6. The observed corpus. Once people know their messages are being written into a system that agents will act on, the messages change. In my experience most people simply do not think about it — but "most" is not a strategy, and the failure mode is not privacy, it is selection: the corpus becomes a record of what people were willing to have recorded. I do not know how large this effect is and I have not seen it measured.
7. Decay and supersession policy. When should a belief expire on its own? What is the right default half-life for different object types, and how do you keep historical truth queryable while stopping it being returned as current?
8. Outcome-grounded evaluation. Does accumulated organisational memory measurably improve later decisions, compared against a matched organisation that has none? Nobody has shown this. It is the load-bearing empirical question under this entire area, including under my own work, and it currently rests on plausibility.
An enormous amount of effort has gone into making models better at reading memory, and it has worked. The unsolved half is deciding what they are allowed to remember — and in an organisation, that is not a storage question. It is a question about what the company is permitted to believe, and it has to be answered by someone before any model gets to read anything.