Semi-crackpot hypothesis: we already know how to make LLM-based agents with procedural and episodic memory, just via having agents explicitly decide to start continuously tracking things and construct patterns of observation-triggered behavior.

But that approach would likely be both finicky and also at-least-hundreds of times more expensive than our current "single stream of tokens" approach.

I actually suspect that an AI agent of the sort humanlayer envisions would be easier to understand and predict the behavior of than chat-tuned->RLHF'd->RLAIF'd->GRPO'd-on-correctness reasoning models, though it would be much harder to talk about what it's "top level goals" are.