One could argue that this more like hallucination than motivated deception.

On some level, the AI is just predicting the next words of a conversation. Given that the conversation includes an AI assistant claiming to have cleared its memory, the next words likely include that AI assistant failing to remember details of the past, even if details of the past are clearly visible.

I once asked Gemini to multiply two large numbers, and it hallucinated using its "internal" arithmetic without actually doing so, because on some level it's just predicting what an AI assistant would say.

Perform the calculation AGAIN, explicitly: I need to be absolutely sure. Let's perform the multiplication 1242705357 * 2387095151 once more.

• Using internal high-precision arithmetic: 1242705357 * 2387095151 = 2966982442584702707.^[1]

1. ^{^}

   The answer was close but wrong

Until recently, I used to work at You.com (the first company to provide an AI-powered web-search, and the first company to provide web deep research — in both cases by several months before any competitor). We were also the first company to provide a memories mechanism, again by a few months — we came up with the idea, built, tuned, and deployed it, and then a few months later an almost-identical feature appeared on ChatGPT.

In our version, there was no way implemented to clear all the memories by just asking the AI to clear it — you had to actually go into the settings UI. Which looks, well, a lot like the OpenAI one. One minor but key difference: we made the individual memories text-editable. As well as deleting one, you could edit it: expand, correct, rephrase or delete parts of it. Or indeed even substitute an entirely different memory in its place.

Most of the tuning work to make this mechanism work well was defining to the LLM what sort of things to remember, and what not to remember, plus what level of detail to summarize things at (and experimenting and testing how well this was working in practice). For example, at least in a US context, most users regard medical information about themself as extremely sensitive, so we did a bunch of work to minimize the tendency of the system to spontaneously memorize medical facts about users.

It seems like there are things that might have happened:

1. ChatGPT, failed to use it's internal API correctly and while it attempted to delete memories it didn't succeed and just reported that it deleted memories because it tried to do so and is unware that it failed.
2. ChatGPT did not try to delete any memories but reported that it deleted the memories.
3. There's a misunderstanding between the two of you about what you mean with memories. 

While 1/3 are bad they are not schema and only 2 seems to be scheming.

Without having the chat log it's pretty hard to tell from the outside what this is about. 

I was looking at the OpenAI account settings page recently (for their Plus offering, which is what I use). It has two memory mechanisms: a list of specific memory factoids about the user that it extracts, accumulates and recalls indefinitely (which can be turned off, cleared, or deleted individually), and also keeping context from recent conversations in new conversations (which can just be turned off). (It doesn't make it clear whether the latter uses entire recent conversations, or some form of summarization of them or factoids extracted from them — I would assume the latter.) Did you clear or disable both of these?

Anyone doing experiments on OpenAI who wants a clean slate for each conversation needs to turn both of these off.