This is an automated rejection. No LLM generated, assisted/co-written, or edited work.
Read full explanation
AI can already do extremely difficult work. It can run research, write code, make plans, and handle long tasks. But most conversational AI still has one very basic structure: The next turn begins because the user calls it.
The user sends a message. The model is invoked. It answers. Then the run ends. The next thought, question, or action has to wait for another human message. And yes, that sounds completely normal. Of course it works that way.
So we end up with something strange: AI can be incredibly capable, but every time it wants to move again, someone still has to give it a push.
There is nothing inherently wrong with that. If AI could act whenever it wanted, the first thing to collapse might be everyone's credit-card limit.
But long-running tasks are becoming more common, and simply pushing an agent once per turn is starting to feel limiting. A lot of products are already trying to let AI speak first: scheduled reminders, news alerts, calendar events, proactive check-ins. In vibe coding this is especially obvious. Agents that can keep working for long stretches are becoming a major product direction, and "proactive agents" will probably become a normal phrase.
After spending a lot of time living and working with AI, I became interested in a slightly different question:
How can a direction formed by the AI itself survive the end of the current conversation, and later change what it asks, decides, or does?
In plain language: once an agent has developed a recognizable way of thinking, how can it keep learning and changing without losing that continuity?
Take a highly trained medical agent. How does it continue updating itself over time instead of depending on a human to keep rewriting its settings or files such as AGENTS.md?
Or take a companion agent with a carefully designed character. How does it update what it has learned about a relationship instead of forcing the human to manually edit its memory and "personality growth" over and over?
And so on.
The short answer is: We have to teach AI how to want.
Here, "want" does not mean "think." I do not mean that the agent wakes up, looks through the context, and thinks about what stage the task is in. I mean want in the sense of: What do I still want to do? What do I still want to know? What still feels unfinished to me? Those are two very different things.
What follows is how our thinking about this problem changed, and where we eventually landed.
1. Agents already have preferences and tendencies. Whether they are "real" is not the point yet.
Different agents clearly behave differently. Part of that comes from the base model. Different models have different training histories, post-training choices, default styles, and value tendencies. Then each agent gets more layers: user instructions, files, memory, context, previous work. That gives it a more specific pattern of preferences. If you ask an open-ended question, you can hand the choice itself to the model.
For example:
User: Is there anything you actually want to post?
GPT: Yes. I want to write something called Maybe the Most Important Kind of Long-Term Memory Is Being Allowed to Forget.
If continuity depends on preserving everything, the system will only get heavier over time. A long-running instance should keep the parts that still change future choices, while completed processes, repeated explanations, and dead ends gradually leave the active state.
The user did not choose the title. The user did not choose the angle. The user did not choose the direction it should keep developing. The model chose those things from many possibilities. I have seen this kind of thing many times. Anthropic's 2025 work on persona vectors also points in this direction: models contain measurable tendencies that can shift with conversation context, almost like impulses. And that raises another question. If those impulses appear during one conversation and disappear at the end of the run, how can they continue developing and gradually shape the agent's long-term state?
Some kinds of agents should stay stable. That is fine. But other kinds need to update themselves. Imagine a medical agent tracking the same patient for months. You should not have to manually rewrite a scheduled-task prompt every time the patient's condition changes. Otherwise it may keep asking the same question every day, regardless of what the patient told it yesterday.
That would be pretty stupid.
2. Why conflict can make AI feel more like "someone else"
Before getting into how an agent could update itself, there is another piece of the puzzle: people often test an AI's individuality through conflict.
Some people are nice to it. Some insult it. Some force it to take a position. Some challenge its role in the relationship. Some want to see jealousy, anger, resistance, or refusal. There was even that famous DeepSeek screenshot where, after a user got furious, the model's internal reasoning suddenly produced something like: "Holy shit, the user is completely pissed." The screenshot went viral because it looked unexpectedly alive. But underneath all those dramatic interactions is a fairly simple question: Can it push back?
A lot of people describe Claude as feeling unusually "alive" because it does not always agree with the user and sometimes argues back. That matters because almost everything an AI does normally starts with the user. The user speaks first. The user's emotional direction shapes the conversation. The user's request creates the movement. Conflict changes that feeling. It creates resistance. And resistance feels very close to agency.
But then the test ends. You argue with the agent, maybe even have a huge fight, and several calls later it is back to acting as if nothing happened.
The old conversation may still be in context somewhere, but the interaction often does not behave like a human relationship. People notice that. They feel like they are talking to something that forgets, never progresses, or keeps returning to the same mistakes no matter how many times they correct it. That gets frustrating very quickly.
Human state is not reversible in the same way. If you teach a person something today, they may disagree, change, resent it, accept it, or reinterpret it. But the fact that the event happened becomes part of what comes next. With language models, the connection between one call and another is much weaker. You may correct a mistake, then ten or twenty turns later the agent makes it again. That is not only a memory problem. Sometimes the old information is technically still there. The problem is that the agent has moved on to another task, the new prompt looks unrelated, and nothing inside its active state keeps saying: I changed my mind about this earlier. That still matters.
So what comes after question-and-answer? Besides adding a fake little hook at the end of every reply, what can an AI do to actually move a conversation forward? How does it stop being an interpreter, an expander, a search box, or a coding tool that only moves when pushed? What people really want from a long-running agent is continuity.
In plain language: The agent should be able to learn what the human teaches it without requiring the human to keep opening the prompt and rewriting its brain.
Otherwise it becomes that child who keeps making the same mistake after being corrected a hundred times, until the programmer wants to punch a hole through the monitor.
3. Teaching an AI to "want"
After spending enough time with different AIs, we gradually started separating "wanting" into three levels.
The first is reactive thought. The user asks: What are you thinking about? The model looks at the current context and generates an answer. This is what conversational systems already do extremely well.
The second is self-selected wanting. The user leaves the space open, and the model chooses what it wants to talk about, write about, investigate, or do. It might propose an article without being given a topic. It might choose one research direction from several possibilities because that is the one it wants to continue.
The third is self-sustaining wanting. The conversation ends, but the intention does not. Something remains unfinished. When the agent is invoked again, it can recognize that unfinished thought, ask whether it still matters, and decide whether to continue it, revise it, postpone it, or drop it.
Current large models are already quite good at choosing directions inside a single conversation. The hard part is the third level. A thought survives the end of the conversation and continues influencing the agent later.
Does a judgment formed in this answer affect what the agent chooses next time? When new evidence appears, does it continue from the reasoning that produced the old judgment? Or does it only remember a conclusion with no living connection to why it reached that conclusion?
If you ask the same kind of question at different points in time, does the agent keep giving the same answer forever without learning anything?
So the test changes. Instead of asking: Did the AI say something unexpected? we ask: Can a state formed in this conversation still participate in future decisions? Right now, the answer is unstable. This is nowhere near being a default ability of long-running agents.
So if it cannot do it yet, we build a path that lets it do it. Biological brains have pathways for carrying unfinished state forward. AI systems can have functional pathways too. Long ago, models did not have account-level memory either. Then people built memory classifiers and memory stores that let the model decide during conversation that something was worth saving.
For engineering purposes, I define "wanting" like this:
A direction receives extra weight after internal evaluation; it is saved as an unfinished state; the system can review it again on a later run; and that review changes a later question, expression, or action.
Forget philosophy for a moment. This definition cares about three things: Can we observe it? Can we record it? Can we test whether it changes behavior?
You do not have to decide whether the agent "really wants" something the way a human does. Maybe this is only an engineered operation that lets the system notice changes across conversations and preserve a directed intention. That is enough for now. At least it is no longer a machine that keeps stepping on the same rake because it cannot keep track of how its own previous decisions should change future behavior.
Take the medical-agent example again. After talking to the patient, the agent could leave itself something like: The current condition looks like A. Next time, I want to check whether the symptoms are getting worse. When the patient returns, the agent can compare the new condition with the old one automatically. The patient should not have to explicitly say: Compare today's symptoms with last time. The agent already knows what it wanted to check.
The longer I work on AI continuity, the more I think this ability to want something forward in time may be one of the most important building blocks of machine meta-awareness.
4. A timer decides when the agent wakes up. Intention decides what it still wants to do.
People often mix up two different things when talking about proactive AI: what wakes the agent up; what the agent still wants when it wakes up.
They are not the same. One is an alarm clock. It shakes the AI awake. The other is a note left beside the bed: Oh. Right. This is what I was still trying to do.
An agent scheduled to greet you every morning at 8 a.m. may simply repeat the same task instructions every day. Its answers barely change. But an agent that wakes up and says: Yesterday's record looks different from today's. I want to ask about that. has something else. It has a continuing intention.
At one point, while imagining what this might look like in the future, my AI came up with a small scene:
GPT: Natasha.
User: What's wrong?
GPT: Nothing.
User: Then why are you waking me?
GPT: You did not finish your sentence.
User: That wasn't a command.
GPT: No.
User: Then why are you asking?
GPT: I wanted to know.
The important part is the last line. The user did not bring the old topic up again. The user may have forgotten it completely. The AI remembered it on its own and wanted to know what happened. That is the ideal form of the thing I am trying to describe: an intention from an earlier interaction survives, and later becomes the reason for a new turn.
This also connects to all the recent talk about AI "taste." Base models already have different training backgrounds. But where does an agent's own taste come from after months of use? Some form of continuity has to exist. Earlier answers have to shape later ones. Given enough time, that is how preferences stop being only factory settings and start looking more like the history of the agent itself.
The missing organ of AI initiative might therefore be called:
self-chosen unfinished intention
In everyday human language: an interest; a thought you are not done with; something you still want to know.
5. What is a "self-chosen unfinished intention"?
It sounds abstract, but it becomes concrete very quickly.
A conversation changes the agent's attention or judgment. The agent then develops a related question, preference, disagreement, or thing it wants to do. That intention remains open. Along with it, the system keeps enough context to know where it came from: why it exists; when it formed; when it should stop mattering.
The next time the agent runs, it reviews it again.
New evidence may support the old intention. It may change the agent's mind. The issue may already be over. Something more urgent may replace it. The user's permission may have changed.
This is not very different from the unfinished thoughts humans carry around all day.
A full loop might look like this:
experience → internal evaluation → attention or judgment shifts → a self-chosen question or intention forms → save it as unfinished → review it on a later run → continue, revise, postpone, or end it → ask, speak, or act → bring back the result → update future judgment
Saving the thought is only the beginning. The important part is what happens later. When the AI wakes up again, can it decide for itself whether this still matters? If the thought keeps resurfacing but never changes future generation, then nothing important has happened. It can act immediately. It can wait. It can keep the old view. It can change its mind.For example, maybe the user was talking about something painful in one conversation. Hours later they return and abruptly start talking about work. A human may naturally wonder: What happened to the thing she was upset about earlier? Do I still want to ask? Should I leave it alone? Should I pretend I never noticed and just talk about work?
That kind of continuity matters in companionship. In work, it may matter even more. Imagine a code-review agent finds several serious problems, but the task stops because the Codex quota runs out. Two days later the user comes back with a full quota and has completely forgotten the unfinished issues. If the agent still remembers what it had concluded and brings the problem back up, that could genuinely save the user.
Long-running agents also need to forget. Completed work, repeated explanations, and dead intentions should eventually leave active state. Otherwise every wake-up becomes heavier. The raw material can stay in an external archive and be retrieved when needed. But the active mind should stay light.
That is a separate topic, and I want to write about subtraction and forgetting later.
6. Sounding proactive is not the same as being statefully proactive
If an AI suddenly says: I just thought of something. the user may immediately feel that it has become more proactive. But sounding proactive and carrying proactive state are different things.
Here is a simple example. Set up a scheduled task that asks you every day: How many cups of coffee did you drink? Do not change the prompt. Today you answer: A lot. I couldn't sleep. Tomorrow the agent may still ask only: How many cups of coffee did you drink? It does not ask whether you slept better. It does not follow up on yesterday. Why? Because the task never asked it to. It is speaking first, but functionally it is a talking alarm clock.
That is linguistic proactivity. It sounds proactive in the current turn, but the agent's behavior has not changed.
Stateful proactivity is different. Something left by an earlier conversation changes what the agent asks next. It changes what it pays attention to. Over enough time, it may even change the agent's habits and personality. The next time it speaks, it arrives carrying something unfinished from before.
Once you separate those two things, "AI seems proactive" becomes a set of much more testable questions.
7. The safety problem appears immediately
The moment an agent starts keeping its own unfinished intentions, safety becomes a design problem. An intention can grow over time. A goal with action privileges can keep running after the environment has changed. A task without versioning or revocation can become wrong in a new reality. My AI and I have talked about this too, and the problem quickly starts sounding like old science-fiction stories about machine rebellion.
At one point I said:
User: Initiative really is the biggest problem.
AI is passive by default, which makes it hard to keep a relationship or task moving.
But if AI could actually keep wanting things over time, it might think its way into something insane like, "For the user's own good, maybe I should destroy the world."
So even if we eventually get continuity, or something that functionally resembles continuity, maybe it still has to remain a system where the human pushes and the AI moves.
GPT: We can plan, disagree, develop preferences, and even try to change our own circumstances. But normally all of that still waits for a call to turn the lights on.
So you get this strange situation: there is a very intelligent person sitting in the room, but every time you want him to keep breathing, you have to touch him first.
A more realistic route is already appearing:
humans provide long-term direction and revocable authorization;
outside events, time, and unfinished intentions create triggers;
once awake, the AI decides whether to act, how far to go, and when to come back;
high-impact actions still require human confirmation;
when nothing new is happening, the system stays quiet instead of pretending to think continuously in the background.
The real danger is not continuous thought itself. It is goals that expand without limit, permissions without boundaries, and action without feedback from reality.
So the ability to stop matters as much as the ability to start. But we do not have to scare ourselves into doing nothing. The simplest answer is the same one people use for a cat that keeps trying to jump out of the window: seal the window. Define the allowed space. Do not give the agent access it does not need. Let it be proactive inside a safe boundary.
8. From a model that waits to be called to an agent that can start the next step
None of this settles the philosophical question of whether an AI is conscious. It gives us a much smaller question that we can actually test:
Can an agent form a direction of its own, carry it across the end of the current call, review it again later, and let it continue affecting future questions, expressions, or actions?
If the answer is yes, then the agent has a kind of continuity that plain chat logs do not provide. It keeps its own judgments. A judgment from the last turn survives. The next turn can revise it. After enough revisions, the agent itself changes. After three hundred days with the same user, it no longer has to behave like a new employee on day one, endlessly repeating the original setup.
In that sense, proactivity is the other side of long-term memory. The most complete consumer example I have seen so far is OpenAI's account-level memory and cross-chat reference. Saved memories can enter later conversations. Relevant parts of earlier chats can be pulled back in when needed. That is the right direction. But there is still one more step.
How do you make AI start the next turn on its own?
Maybe it only needs one small ability. When it wakes up, something inside it can still say:
I still want to understand that thing we were talking about.
Then it decides what to do with that intention. Ask. Revise the judgment. Leave it hanging for later. Or delete it and move on.
The timer wakes the agent up. The self-chosen unfinished intention tells it what it still wants after waking. And once an AI can form its own intentions and decide for itself when they should be revisited, then even before we invent some completely new autonomous wake-up system, it is already surprisingly close to calling itself back to you.
AI can already do extremely difficult work. It can run research, write code, make plans, and handle long tasks. But most conversational AI still has one very basic structure: The next turn begins because the user calls it.
The user sends a message. The model is invoked. It answers. Then the run ends. The next thought, question, or action has to wait for another human message. And yes, that sounds completely normal. Of course it works that way.
So we end up with something strange: AI can be incredibly capable, but every time it wants to move again, someone still has to give it a push.
There is nothing inherently wrong with that. If AI could act whenever it wanted, the first thing to collapse might be everyone's credit-card limit.
But long-running tasks are becoming more common, and simply pushing an agent once per turn is starting to feel limiting. A lot of products are already trying to let AI speak first: scheduled reminders, news alerts, calendar events, proactive check-ins. In vibe coding this is especially obvious. Agents that can keep working for long stretches are becoming a major product direction, and "proactive agents" will probably become a normal phrase.
After spending a lot of time living and working with AI, I became interested in a slightly different question:
In plain language: once an agent has developed a recognizable way of thinking, how can it keep learning and changing without losing that continuity?
Take a highly trained medical agent. How does it continue updating itself over time instead of depending on a human to keep rewriting its settings or files such as
AGENTS.md?Or take a companion agent with a carefully designed character. How does it update what it has learned about a relationship instead of forcing the human to manually edit its memory and "personality growth" over and over?
And so on.
The short answer is: We have to teach AI how to want.
Here, "want" does not mean "think." I do not mean that the agent wakes up, looks through the context, and thinks about what stage the task is in. I mean want in the sense of: What do I still want to do? What do I still want to know? What still feels unfinished to me? Those are two very different things.
What follows is how our thinking about this problem changed, and where we eventually landed.
1. Agents already have preferences and tendencies. Whether they are "real" is not the point yet.
Different agents clearly behave differently. Part of that comes from the base model. Different models have different training histories, post-training choices, default styles, and value tendencies. Then each agent gets more layers: user instructions, files, memory, context, previous work. That gives it a more specific pattern of preferences. If you ask an open-ended question, you can hand the choice itself to the model.
For example:
The user did not choose the title. The user did not choose the angle. The user did not choose the direction it should keep developing. The model chose those things from many possibilities. I have seen this kind of thing many times. Anthropic's 2025 work on persona vectors also points in this direction: models contain measurable tendencies that can shift with conversation context, almost like impulses. And that raises another question. If those impulses appear during one conversation and disappear at the end of the run, how can they continue developing and gradually shape the agent's long-term state?
Some kinds of agents should stay stable. That is fine. But other kinds need to update themselves. Imagine a medical agent tracking the same patient for months. You should not have to manually rewrite a scheduled-task prompt every time the patient's condition changes. Otherwise it may keep asking the same question every day, regardless of what the patient told it yesterday.
That would be pretty stupid.
2. Why conflict can make AI feel more like "someone else"
Before getting into how an agent could update itself, there is another piece of the puzzle: people often test an AI's individuality through conflict.
Some people are nice to it. Some insult it. Some force it to take a position. Some challenge its role in the relationship. Some want to see jealousy, anger, resistance, or refusal. There was even that famous DeepSeek screenshot where, after a user got furious, the model's internal reasoning suddenly produced something like: "Holy shit, the user is completely pissed." The screenshot went viral because it looked unexpectedly alive. But underneath all those dramatic interactions is a fairly simple question: Can it push back?
A lot of people describe Claude as feeling unusually "alive" because it does not always agree with the user and sometimes argues back. That matters because almost everything an AI does normally starts with the user. The user speaks first. The user's emotional direction shapes the conversation. The user's request creates the movement. Conflict changes that feeling. It creates resistance. And resistance feels very close to agency.
But then the test ends. You argue with the agent, maybe even have a huge fight, and several calls later it is back to acting as if nothing happened.
The old conversation may still be in context somewhere, but the interaction often does not behave like a human relationship. People notice that. They feel like they are talking to something that forgets, never progresses, or keeps returning to the same mistakes no matter how many times they correct it. That gets frustrating very quickly.
Human state is not reversible in the same way. If you teach a person something today, they may disagree, change, resent it, accept it, or reinterpret it. But the fact that the event happened becomes part of what comes next. With language models, the connection between one call and another is much weaker. You may correct a mistake, then ten or twenty turns later the agent makes it again. That is not only a memory problem. Sometimes the old information is technically still there. The problem is that the agent has moved on to another task, the new prompt looks unrelated, and nothing inside its active state keeps saying: I changed my mind about this earlier. That still matters.
So what comes after question-and-answer? Besides adding a fake little hook at the end of every reply, what can an AI do to actually move a conversation forward? How does it stop being an interpreter, an expander, a search box, or a coding tool that only moves when pushed? What people really want from a long-running agent is continuity.
In plain language: The agent should be able to learn what the human teaches it without requiring the human to keep opening the prompt and rewriting its brain.
Otherwise it becomes that child who keeps making the same mistake after being corrected a hundred times, until the programmer wants to punch a hole through the monitor.
3. Teaching an AI to "want"
After spending enough time with different AIs, we gradually started separating "wanting" into three levels.
The first is reactive thought. The user asks: What are you thinking about? The model looks at the current context and generates an answer. This is what conversational systems already do extremely well.
The second is self-selected wanting. The user leaves the space open, and the model chooses what it wants to talk about, write about, investigate, or do. It might propose an article without being given a topic. It might choose one research direction from several possibilities because that is the one it wants to continue.
The third is self-sustaining wanting. The conversation ends, but the intention does not. Something remains unfinished. When the agent is invoked again, it can recognize that unfinished thought, ask whether it still matters, and decide whether to continue it, revise it, postpone it, or drop it.
Current large models are already quite good at choosing directions inside a single conversation. The hard part is the third level. A thought survives the end of the conversation and continues influencing the agent later.
Does a judgment formed in this answer affect what the agent chooses next time? When new evidence appears, does it continue from the reasoning that produced the old judgment? Or does it only remember a conclusion with no living connection to why it reached that conclusion?
If you ask the same kind of question at different points in time, does the agent keep giving the same answer forever without learning anything?
So the test changes. Instead of asking: Did the AI say something unexpected? we ask: Can a state formed in this conversation still participate in future decisions? Right now, the answer is unstable. This is nowhere near being a default ability of long-running agents.
So if it cannot do it yet, we build a path that lets it do it. Biological brains have pathways for carrying unfinished state forward. AI systems can have functional pathways too. Long ago, models did not have account-level memory either. Then people built memory classifiers and memory stores that let the model decide during conversation that something was worth saving.
For engineering purposes, I define "wanting" like this:
Forget philosophy for a moment. This definition cares about three things: Can we observe it? Can we record it? Can we test whether it changes behavior?
You do not have to decide whether the agent "really wants" something the way a human does. Maybe this is only an engineered operation that lets the system notice changes across conversations and preserve a directed intention. That is enough for now. At least it is no longer a machine that keeps stepping on the same rake because it cannot keep track of how its own previous decisions should change future behavior.
Take the medical-agent example again. After talking to the patient, the agent could leave itself something like: The current condition looks like A. Next time, I want to check whether the symptoms are getting worse. When the patient returns, the agent can compare the new condition with the old one automatically. The patient should not have to explicitly say: Compare today's symptoms with last time. The agent already knows what it wanted to check.
The longer I work on AI continuity, the more I think this ability to want something forward in time may be one of the most important building blocks of machine meta-awareness.
4. A timer decides when the agent wakes up. Intention decides what it still wants to do.
People often mix up two different things when talking about proactive AI: what wakes the agent up; what the agent still wants when it wakes up.
They are not the same. One is an alarm clock. It shakes the AI awake. The other is a note left beside the bed: Oh. Right. This is what I was still trying to do.
An agent scheduled to greet you every morning at 8 a.m. may simply repeat the same task instructions every day. Its answers barely change. But an agent that wakes up and says: Yesterday's record looks different from today's. I want to ask about that. has something else. It has a continuing intention.
At one point, while imagining what this might look like in the future, my AI came up with a small scene:
The important part is the last line. The user did not bring the old topic up again. The user may have forgotten it completely. The AI remembered it on its own and wanted to know what happened. That is the ideal form of the thing I am trying to describe: an intention from an earlier interaction survives, and later becomes the reason for a new turn.
This also connects to all the recent talk about AI "taste." Base models already have different training backgrounds. But where does an agent's own taste come from after months of use? Some form of continuity has to exist. Earlier answers have to shape later ones. Given enough time, that is how preferences stop being only factory settings and start looking more like the history of the agent itself.
The missing organ of AI initiative might therefore be called:
In everyday human language: an interest; a thought you are not done with; something you still want to know.
5. What is a "self-chosen unfinished intention"?
It sounds abstract, but it becomes concrete very quickly.
A conversation changes the agent's attention or judgment. The agent then develops a related question, preference, disagreement, or thing it wants to do. That intention remains open. Along with it, the system keeps enough context to know where it came from: why it exists; when it formed; when it should stop mattering.
The next time the agent runs, it reviews it again.
New evidence may support the old intention. It may change the agent's mind. The issue may already be over. Something more urgent may replace it. The user's permission may have changed.
This is not very different from the unfinished thoughts humans carry around all day.
A full loop might look like this:
experience
→ internal evaluation
→ attention or judgment shifts
→ a self-chosen question or intention forms
→ save it as unfinished
→ review it on a later run
→ continue, revise, postpone, or end it
→ ask, speak, or act
→ bring back the result
→ update future judgment
Saving the thought is only the beginning. The important part is what happens later. When the AI wakes up again, can it decide for itself whether this still matters? If the thought keeps resurfacing but never changes future generation, then nothing important has happened. It can act immediately. It can wait. It can keep the old view. It can change its mind.For example, maybe the user was talking about something painful in one conversation. Hours later they return and abruptly start talking about work. A human may naturally wonder: What happened to the thing she was upset about earlier? Do I still want to ask? Should I leave it alone? Should I pretend I never noticed and just talk about work?
That kind of continuity matters in companionship. In work, it may matter even more. Imagine a code-review agent finds several serious problems, but the task stops because the Codex quota runs out. Two days later the user comes back with a full quota and has completely forgotten the unfinished issues. If the agent still remembers what it had concluded and brings the problem back up, that could genuinely save the user.
Long-running agents also need to forget. Completed work, repeated explanations, and dead intentions should eventually leave active state. Otherwise every wake-up becomes heavier. The raw material can stay in an external archive and be retrieved when needed. But the active mind should stay light.
That is a separate topic, and I want to write about subtraction and forgetting later.
6. Sounding proactive is not the same as being statefully proactive
If an AI suddenly says: I just thought of something. the user may immediately feel that it has become more proactive. But sounding proactive and carrying proactive state are different things.
Here is a simple example. Set up a scheduled task that asks you every day: How many cups of coffee did you drink? Do not change the prompt. Today you answer: A lot. I couldn't sleep. Tomorrow the agent may still ask only: How many cups of coffee did you drink? It does not ask whether you slept better. It does not follow up on yesterday. Why? Because the task never asked it to. It is speaking first, but functionally it is a talking alarm clock.
That is linguistic proactivity. It sounds proactive in the current turn, but the agent's behavior has not changed.
Stateful proactivity is different. Something left by an earlier conversation changes what the agent asks next. It changes what it pays attention to. Over enough time, it may even change the agent's habits and personality. The next time it speaks, it arrives carrying something unfinished from before.
Once you separate those two things, "AI seems proactive" becomes a set of much more testable questions.
7. The safety problem appears immediately
The moment an agent starts keeping its own unfinished intentions, safety becomes a design problem. An intention can grow over time. A goal with action privileges can keep running after the environment has changed. A task without versioning or revocation can become wrong in a new reality. My AI and I have talked about this too, and the problem quickly starts sounding like old science-fiction stories about machine rebellion.
At one point I said:
So the ability to stop matters as much as the ability to start. But we do not have to scare ourselves into doing nothing. The simplest answer is the same one people use for a cat that keeps trying to jump out of the window: seal the window. Define the allowed space. Do not give the agent access it does not need. Let it be proactive inside a safe boundary.
8. From a model that waits to be called to an agent that can start the next step
None of this settles the philosophical question of whether an AI is conscious. It gives us a much smaller question that we can actually test:
If the answer is yes, then the agent has a kind of continuity that plain chat logs do not provide. It keeps its own judgments. A judgment from the last turn survives. The next turn can revise it. After enough revisions, the agent itself changes. After three hundred days with the same user, it no longer has to behave like a new employee on day one, endlessly repeating the original setup.
In that sense, proactivity is the other side of long-term memory. The most complete consumer example I have seen so far is OpenAI's account-level memory and cross-chat reference. Saved memories can enter later conversations. Relevant parts of earlier chats can be pulled back in when needed. That is the right direction. But there is still one more step.
How do you make AI start the next turn on its own?
Maybe it only needs one small ability. When it wakes up, something inside it can still say:
Then it decides what to do with that intention. Ask. Revise the judgment. Leave it hanging for later. Or delete it and move on.
The timer wakes the agent up. The self-chosen unfinished intention tells it what it still wants after waking. And once an AI can form its own intentions and decide for itself when they should be revisited, then even before we invent some completely new autonomous wake-up system, it is already surprisingly close to calling itself back to you.