Designing AI Memory Systems: Architecture and Engineering Practice

This article examines AI memory from a systems-architecture perspective and explains what an agent actually needs to reuse the past. Here, memory is the collection of records, knowledge, and experience that an agent accumulates over extended interactions and later retrieves and applies. It directly affects personalization, continual learning, and performance on long-horizon tasks.

The author is Kuda (Chen Zikang) of Ant Group. The material was prepared as a community chapter for the DataWhale × OceanBase course Easy Data x AI (see subsection X1-4). Because the original notes go into considerable depth, some formulas and theorems have been simplified to keep the architectural argument readable. By the end, you should have a concrete picture of how memory can be implemented within an agent.

Alibaba Cloud Model Studio (Bailian) also provides dedicated free model compute for the community chapters and labs of that Easy Data x AI course. Details are in the benefits section at the end.

Note: the article mentions some 2026 papers. Treat them as references, not as required reading.

Follow the OceanBase community WeChat account “老纪的技术唠嗑局” for ongoing writing on #AI and #Data.

1. What memory really is: three core propositions

The question memory must answer within an agent is: did the past help it do the right thing today? A Markdown file is not enough. Even a database is not enough. The agent needs a small Memory OS.

Core proposition: what is memory in an agent system?

After a couple of years of thinking about this, here is a working conclusion: an agent’s Memory OS can be abstracted into three concepts—Raw Ledger, Views, and Policy. They correspond roughly to an operating system’s raw log, its indexes and caches, and its scheduling rules.

Proposition A: memory is not storage; it is external state that decisions can use

If you treat an agent as a function from input to output, storing a large amount of history is not itself a capability. That history becomes useful only when it can influence the current decision distribution.

The memory system extracts information that is useful now—such as evidence, summaries, subgraphs, and executable skills—and passes it to the reasoning layer so the two can jointly produce a decision. The value of memory lies not in how much history it stores, but in whether the channel from history to the current decision actually works.

Memory is not history itself. It is the channel that turns history into useful information. Its output either enters the context as evidence, a summary, or a subgraph, or participates directly in the decision—for example, by modulating the output distribution.

Proposition B: the minimal closure of Memory is Ledger, Views, and Policy

1. Raw Ledger. It keeps writes, edits, deletes, and feedback in the order they happened. “Ledger” already means a book of accounts; “Raw” reminds us that what lives here is unpolished fact. What the user said on which day, when the system revised which memory, and which session or task it belonged to should all be recoverable.
2. Derived Views. These are the versions of the ledger that are easy to look up. Vector indexes find things by meaning, full-text indexes find names and IDs, knowledge graphs find relationships, and timelines answer what happened last year. A view may compress or drop detail, but you should still be able to walk back to the raw ledger.
3. Policy. It decides when to read, how much to read, when to write, how to update, and how to forget. Those decisions must be made explicit as a recordable, replayable Action sequence (ADD / UPDATE / DELETE / NONE …), not hinted at by one sentence in a prompt.

The Raw Ledger is the black box or system of record. Views are caches, indexes, and materialized views. Policy is the scheduler or control loop. Remove any one of them, and the system becomes ungovernable (no ledger), unusable (no index or abstraction), or unable to improve iteratively (no control point for A/B testing).

Proposition C: the basic unit is an event sequence, but an event stream is not a usable system

Modeling the Raw Ledger as an event sequence is reasonable, because every audit property in the protocol depends on an event closure.

A general ledger event should include at least:

• Scope — which user, session, and task the event belongs to
• Timestamp — when it happened
• Input observation — messages or environment-state fragments at that moment
• System action — external outputs and Memory-tool actions
• Memory change — ADD / UPDATE / DELETE / NONE
• Feedback signal (optional) — reward, user rating, task success or failure
• Decision metadata (optional) — candidate_set, provenance of hits, early-stop thresholds, and so on

The event sequence is the source of truth, but it is too low-level. If you store only events, you get auditable history rather than usable memory. Views turn history into capability through reorganization, compression, indexing, temporal modeling, and skill extraction; policy determines when to use each view and how to update it.

In other words: events are the data form of the Ledger; views and policy are the capability form.

Why Raw Ledger + Views + Policy is the natural solution

The three propositions complete a clearer chain:

1. Memory needs event and action sequences as first-class objects. Without them, provenance and replay are impossible.
2. A single event stream is not usable. Reasoning needs dense information, indexes, temporal structure, and skills — all of which require views.
3. Views cannot generate themselves coherently. They are derived state, and derivation means approximation and conflict. Policy must decide write / update / retrieve / evict, and the decision process itself must be recorded, or the system cannot be governed or A/B tested.

Memory is therefore not a component but a closed-loop system: Raw Ledger (authority) → Views (usability) → Policy (control) → Commit (write-back) → Provenance (replay).

2. Designing System 1 + System 2

Why System 2 must be non-empty

The names come from cognitive science. This article borrows the labels and does not reopen the psychology debate.

System 1 is the fast, general-purpose agent—the large model plus tools that we usually encounter. It understands the question, plans, writes code, searches, and generates an answer.

System 2 is the slower external memory loop. It stores, retrieves, updates, compresses, and forgets. It resembles a kernel and a set of background services, remaining behind the scenes most of the time. Without it, every restart feels like a fresh installation.

If System 2 does not act, memory can only be baked into System 1 (LLM weights) by methods such as RL post-training. In that setting it is hard to keep general capability after memory-specialized training. A non-empty System 2 therefore has to own write, retrieve, and update, and to make those decisions observable and replayable.

Memory capabilities and the LLM’s other agent capabilities are also relatively orthogonal. You can trade away a small amount of the memory system’s theoretical ceiling and gain considerably more: System 1 performance remains intact, memory becomes pluggable and portable, and attribution becomes easier.

As an informal biological analogy, teaching an agent to use an external system is closer to how humans expand their capabilities than encoding everything in LLM weights. People extend their abilities quickly by using tools.

What “relatively orthogonal” means here

It does not mean strict independence. It means you can split the system into two loosely coupled modules—System 1’s general agent capabilities and System 2’s memory reading, writing, and retrieval—and optimize them separately without frequent large-scale interference or collapse.

Empirically, the same base model, without memory-specific training, can change long-horizon performance a lot when you plug in different RAG or long-term memory policies. The same memory infrastructure (raw_ledger + views + retrieval policy) can often serve many base models. Cross-model reuse usually means the capability lives in interfaces and external state, not in one particular weight file.

The non-orthogonal boundary is real: retrieval noise, misses, and temporal conflicts disrupt reasoning and cause hallucinations. Decisions about when and how much to retrieve, as well as when to write, also depend on the agent’s self-evaluation and planning. A more accurate phrase is “relatively orthogonal, with controllable cross terms.” Observability, provenance, and sandboxed A/B testing in System 2 make those cross terms explicit, diagnosable, and open to iteration.

Given that trade-off, System 2 is required (and must not be empty). Once it exists, memory update and recall can be studied independently of the LLM (System 1). The next question is how to model System 2 with one theory. Agentic Memory (AgeMem [1]) is a useful starting point: it is an independent system whose essence is an actively controlled loop.

That is enough to sketch System 2:

With the System 1 + System 2 split in place, the core question becomes: how good can a non-parametric System 2 get? That leads to parametric versus non-parametric memory, and to an analysis of the non-parametric ceiling.

Approximating parametric Memory without writing weights

Parametric and non-parametric: two carriers of online adaptation

Treat what has been learned as information that can influence the output distribution. There are two classic carriers:

• Parametric memory. Experience is written into model weights. Training / fine-tuning compiles history into those weights. At inference time you just use the updated model; no extra retrieval is required.
• Non-parametric memory. Experience lives in external state (ledger + views + skill pool + indexes). Policy decides what to write and how; inference lets that state affect output through retrieve / aggregate / inject.

The difference is not whether storage exists, but where the adaptation operator is written. Parametric memory pays the write cost up front during training. Non-parametric memory spreads it across online commits and inference-time retrieval and injection.

The purpose of System 2 is to move online adaptation from model weights to external memory state and control policy, making the process observable, replayable, and suitable for A/B testing.

How memory modulates decisions: the correction term Δ

If one LLM or agent step is expressed as logits (or an action distribution), the most general way for memory to affect the decision is for external memory to apply a controllable correction, Δ. System 1’s weights provide baseline generality, while Δ comes from external memory and supports personalization, task specialization, temporal correction, and the reuse of experience.

In plain language:

New action tendency = the model’s original judgment + the memory correction Δ

Δ is simply a correction. Think of it as an external, controllable bias: it does not change the weights, but it does change the decision distribution. If Shin-chan has said that he hates green peppers, a dining agent should push “stir-fried pork with peppers” to the bottom of the candidate list. If previous deployment logs show that a command destroys data, the system should lower its priority.

That is also why policy must take the form of explicit tool operations. If the source of Δ is unauditable—for example, if it is implicit in a prompt—the system cannot support provenance, rollback, or sandboxing. The protocol therefore makes the source of Δ explicit through retrieved evidence, candidate sets, and write actions.

JitRL [2] and UMEM [3]: how external experience transfers

Can an agent use past successes and failures at decision time without retraining the model? One line of research retrieves similar trajectories before each action, uses past outcomes to estimate whether the current action is worth taking, and then adjusts its priority. After the task, the new trajectory and result are written back to the experience store. JitRL is representative of this approach.

RL is reinforcement learning: act, look at the result, adjust the next choice. The method does not have to change model parameters. An external experience store can participate and try to approximate the effect of a just-in-time fine-tune.

Finding one similar case is not enough. Today you remember “this error means check the connection pool.” Tomorrow the error text changes a few words and the system no longer recognizes it.

The harder problem is enabling one experience to help with an entire class of problems. Some research builds neighborhoods based on semantic similarity, groups related questions, and estimates how much a memory helps the whole neighborhood. What the system retains is not merely the answer to one item, but potentially a general principle for a class of failures. UMEM studies that problem.

“Semantically similar” means the wording differs but the meaning is close. Vector retrieval represents that meaning as a list of numbers and compares distances. You do not need the formula; you only need to know that “cannot connect to the database” and “the service cannot establish a connection” get a chance to be treated as relatives.

What sets the ceiling of non-parametric Memory

Storing a hundred million records does not mean the agent can use them all for the task at hand. The goal of non-parametric memory is to approximate the effect of effective fine-tuning on a decision, not to maximize disk capacity.

1. Interface bandwidth (how much Memory can inject into System 1)

Whether the payload is a passage, a graph, a skill, or a tool description, it eventually has to reach System 1 through some Context Bridge.

The simplest bridge is “paste into the prompt.” A more aggressive one is latent / KV injection (the integration layer later). Either way, bandwidth is finite: token budget, attention capacity, or injected KV length.

In other words, the injection channel has a budget. How much consumable evidence or representation you can stuff into System 1 is bounded by tokens, latency, GPU memory, and attention length. A huge external store does not help if each step can only inject a little effective information. Consolidation / compression, hierarchical memory, and latent tokens are all attempts to raise information density per unit of budget.

2. Retrieval and aggregation error

Views are approximate structures derived from the Raw Ledger — built by indexing, consolidation, temporalization, and skill extraction, then turned into a result by recall, rerank, aggregation, and early stop.

Unless you dump the entire ledger into the model, views are approximations. Approximation produces false hits, misses, temporal conflicts, and semantic drift, all of which pollute Δ. Retrieval noise breaks reasoning directly. The ceiling therefore depends less on “how much you stored” and more on whether view error is governed: observable, attributable, and replayable, so you can iterate the error down.

3. Learning and control of Policy

Whether reads and writes are trustworthy. Store too much and you pollute; store too little and you learn nothing. Recall too much and you crowd out context; recall too little and you lack evidence. Delete one long-term preference by mistake and personalization drifts from then on.

The Memory Algorithm Protocol constrains policy output to an action sequence. UPDATE and DELETE must be limited to a candidate set, and recall must carry provenance. Policy receives the current input, retrieval results, and memory state; it then emits actions that Commit persists.

The real ceiling bottleneck of non-parametric Memory is often not the storage backend. It is policy:

• Write too much and you pollute; write too little and you do not learn.
• Recall too much and you get noise; recall too little and you lack information.
• One bad UPDATE / DELETE snowballs over the long run.

Policy must therefore be both learnable through an RL-style training paradigm and governable through candidate-set constraints, complete provenance, and sandboxed replay-based A/B testing. A rubric-based policy is not sustainable; the control loop must become an explicit, trainable object.

3. Designing the core Memory modules

The Memory System control / policy layer

The first assumption to challenge is the traditional engineering preference for a rubric-based policy—predefined rules that control memory reads, writes, and updates. A model must perform this role. There are two options: train an external neural network, or use prompting, supervised fine-tuning (SFT), or reinforcement learning (RL) with a language model. Either way, the model must treat memory operations as tools: it should actively control reads, writes, and updates rather than passively receiving context.

The first option is difficult and remains largely exploratory in academic research. If every parametric problem is answered with “train another network,” you must identify and successfully apply the right training method for each problem. That approach does not scale. GRPO, at least, is a broadly adopted training method and is much simpler than inventing a new network from scratch.

We therefore take the second approach and pursue Agentic Memory: memory operations become tools within the agent’s action space. This is a key step toward the parametric ceiling because the model controls memory as directly as it controls an arm. The trainable parameters remain in the underlying LLM agent. This choice is deliberate: we continue to rely on an LLM’s ability to generalize and transfer knowledge, while LLMs themselves are improving rapidly.

Memory class	Operation	What it does	Parametric analogue
Long-term memory (LTM)	ADD	Store new knowledge	Gradient update (training)
Long-term memory (LTM)	UPDATE	Correct old knowledge	Weight adjustment (fine-tuning)
Long-term memory (LTM)	DELETE	Remove stale information	Catastrophic-forgetting management / pruning
Short-term memory (STM)	RETRIEVE	Semantic search and inject	Activate related neurons
Short-term memory (STM)	SUMMARY	Compress conversation history	Form an abstract representation
Short-term memory (STM)	FILTER	Drop irrelevant context	Attention masking

A few 2026 parametric-memory papers are worth listing.

AgeMem RL training

When an agent must handle long-term memory and immediate context at the same time, it rarely uses the full toolset effectively from the outset. One training approach is to separate the skills and then introduce them gradually in increasingly difficult tasks. AgeMem uses this form of staged reinforcement learning.

Stage one trains long-term memory: in everyday dialogue, the agent decides what is worth adding and what constitutes a correction. Stage two trains short-term memory management: the system injects distractors so the agent learns to filter and summarize. Only in stage three does the agent combine long-term retrieval and short-term housekeeping in complex tasks.

The sequence resembles employee onboarding: first learn to file documents, then learn to organize the desk, and only then join a real project. If the first day requires serving customers, managing files, writing retrospectives, and fixing production incidents all at once, the likely result is exhaustion and little learning.

InfMem [4]: the PreThink-Retrieve-Write protocol

In addition to tool-based training, System 2’s active control loop can implement a more refined retrieval policy. InfMem addresses the “lost in the middle” problem in long-document reasoning with a PreThink-Retrieve-Write protocol that explicitly simulates slow thinking. It works as follows:

• PreThink. Before retrieval, the agent estimates whether its internal parametric knowledge is already sufficient. This reduces unnecessary external retrieval and latency.
• Adaptive early stopping. Classic RAG retrieves a fixed Top-K. InfMem trains a policy network that stops as soon as accumulated evidence confidence crosses a threshold. The paper reports a 3.9× speedup, much closer to the instant response of a parametric model.
• Training paradigm (SFT-to-RL). InfMem moves from supervised fine-tuning to reinforcement learning and directly optimizes retrieve and memory-update decisions so every external interaction maximizes final-answer accuracy.

Policy decides what to write and read. What the system can write depends on the structure and compression of the memory unit itself.

That is the next design problem.

Structure, time, and compression of memory units

We have chosen an Agentic Memory design and a rough shape for System 2. We still need the structure, temporality, and compression of the memory unit.

SimpleMem [5] attacks “context inflation” in long interaction with a mechanism inspired by biological consolidation.

SimpleMem does not store raw text. It stores compressed memory units and defines an affinity score between them, where the score is computed from the embedding vectors of those units.

Using that score, SimpleMem asynchronously runs recursive consolidation in the background and merges high-affinity units into higher-level abstractions. The process resembles hierarchical abstraction in parametric training: concrete samples are refined, layer by layer, into more general features, raising storage efficiency without throwing away the key information.

SimpleMem reports that on its long-horizon multi-turn dialogue tasks it beat a full-context model at a much lower token cost, which suggests consolidation is doing real work. (The exact ratio was lost in the source conversion.)

A major weakness of parametric memory is its static worldview. Once training is complete, weights tend to remain fixed and cannot naturally represent facts that change over time. LLMs are also relatively insensitive to time, so the real world and the world represented in memory fall out of sync. Zep [6] and its engine Graphiti [7] introduce a temporal knowledge graph (TKG) to mitigate this problem.

Zep annotates graph edges with temporal validity. “Biden is President of the United States” is no longer a static fact; it carries a validity window that distinguishes “was true” from “is true now.” (The concrete interval was lost in the source conversion.)

At retrieval time, Zep synthesizes a “now” truth graph from the query’s temporal context. Intuitively, this lets a Memory OS approximate a parametric model after unlearning: it can separate historical facts from current facts. The paper reports an 18.5% accuracy gain over traditional RAG on its long-horizon memory evaluation.

A shorter restatement of the denser material above:

Full dialogue is the most complete and the most expensive. A tiny summary is cheap, but details and sources may vanish with it. A long-lived system must keep both raw evidence and high-density abstraction.

The longer an interaction continues, the more easily its context becomes bloated. One response is to compress history into memory units, then recursively merge related content in the background based on semantic affinity until a higher level of abstraction emerges. That is the approach taken by SimpleMem.

Think of photo management. The bottom layer keeps the originals; above it are trip albums and then a year in review. You begin with the overview and return to the originals only when you need to know who said what on a particular day. More compression is not always better. A summary must retain a pointer to its source so the system can verify it when compression introduces an error.

Temporal memory: from “what happened” to “when it was true, and when it was believed”

“Biden is President of the United States” may have been fine in 2024 and wrong in 2026. Semantic similarity knows two sentences talk about the same thing. It does not automatically decide which one is valid now.

In the Memory OS triple (Raw Ledger, Views, Policy), time is not an extra field. It changes the semantic boundary of all three layers:

• Raw Ledger. Append-only records of “what write action happened.” After you add time, the ledger must answer both “when the system wrote or corrected” (transaction time) and “when the fact was true in the world” (valid time). They are not the same: you can correct last week’s fact today.
• Views. Retrieval is not “relevant is enough.” It is “holds in this query’s time context.” Without a time slice, semantic search treats old facts as current facts.
• Policy. The default time_scope=current means “prefer a miss over a wrong hit.” That only works if policy can explicitly trigger historical / all when history is needed, and if the EvidencePack can carry coexisting conflicting evidence so System 1 decides — Memory OS should not privately adjudicate.

Why time has to sit in the architectural skeleton:

1. LLMs are naturally weak at time (especially implicit time, relative time, and time zones).
2. Pure semantic retrieval mistakes a highly similar past fact for a fact that is still true.
3. The decision layer then sees “stale facts resurrected” and “corrected facts recalled again and again.”
4. The solution is not a stronger prompt. It is bitemporal, time-sliced recall, which makes “when it was true” a hard constraint on retrieval and aggregation while keeping policy controllable and replayable.

Hierarchical memory: the same history needs different abstractions

One storage shape rarely serves relational reasoning, experience transfer, and evidence checking at once. To give each task a suitable form, you can split memory into graph memory, experience memory, and episodic memory. MemWeaver [8] is a representative of this layered approach.

Graph memory stores entity relations and answers who is related to whom. Experience memory abstracts handling patterns from many interactions and helps on similar problems. Episodic memory keeps the original text for checking and provenance.

It is like keeping an architecture diagram, a retrospective, and the meeting transcript for the same project. They come from one history and serve different questions. Keeping only one form may look tidy and work poorly.

So far, the “memory unit” has mostly been declarative memory (facts / evidence / relations): it answers “what is it / what happened.” The next move is from “what to store” to “how to do it” — procedural memory.

The procedural layer: from knowledge to skill

Fact memory answers “what” and “what happened.” Long-horizon tasks also need “what to do next.”

If an agent successfully migrates a database but retains only a chat excerpt, it must still plan, retry, and verify everything again the next time. Procedural memory stores a reusable process, much like an executable in an operating system: the conditions under which it starts, the sequence of actions it performs, and the conditions under which it is complete.

Research is already exploring how agents can learn reusable skills from repeated attempts without drifting into incoherent behavior. A skill can be described as a Skill-MDP: when to start, which actions to perform, and when to stop. ProcMEM [9] is one example.

A skill contains a trigger, an action sequence, and a termination condition. The system first reviews success and failure trajectories and uses natural language to analyze what should change. That is called a semantic gradient. Model parameters do not change; the revision is written onto the skill text.

Candidate skills do not enter production as soon as they are written. ProcMEM uses PPO-style gating as a counterfactual check: does the new skill favor historically effective actions without diverging too far from the old skill? PPO is a common RL algorithm; the idea borrowed here is to make incremental changes rather than abruptly adopting entirely different behavior.

After a skill is deployed, its real-world returns must still be monitored. Skills that contribute nothing for a long time are retired; semantically duplicate skills are merged; and skills that have not been validated since the environment changed are tested again. Collecting a hundred “efficient work methods” is not procedural memory. A skill must execute at the right moment and demonstrate its value through results.

The integration layer: latent fusion and zero-shot alignment (Memory Tokens)

Text, graphs, temporal edges, and skills all have to enter the large model. The common method is to turn them into text, paste them into context, and let the model encode them again. It is readable. It also burns tokens and time.

Memory Tokens explore another route. A token is the model’s basic unit of processing; a Memory Token tries to compress external memory into something closer to the model’s internal representation, so there is less translation back and forth.

LycheeMemory [10]: latent compression and KV-cache injection

Re-encoding external text on every call costs compute and GPU memory. Some work trains a compressor that turns content into latent tokens and injects them into attention in a form close to the KV-cache. LycheeMemory explores that latent-compression route.

The KV cache stores intermediate results during inference so the model does not recompute everything from scratch for each new token. Representing memory in a similar form can reduce text encoding and decoding overhead.

The trade-off is equally clear: latent memory becomes increasingly difficult for people to interpret. If a latent representation contributes to an incorrect answer, the system must still provide access to the original evidence. Otherwise, the on-call engineer is left with an elegant sequence of numbers that offers no explanation.

MemAdapter [11]: an aligner for heterogeneous memory

Text, graphs, and skills each have their own representation. The same model may not read them directly. One idea is to take a query-relevant local subgraph, then align that structured information into the model’s semantic space, so you can fuse heterogeneous memory without fine-tuning the main model. MemAdapter studies this kind of zero-shot alignment.

An adapter is like a travel plug: making it fit is only the first step. Voltage compatibility and the ability to locate a faulty connection still matter. The alignment layer needs source tracking, rollback, and sandboxed comparison. A sandbox is an isolated environment where several policies can run the same tasks without modifying production memory.

Control and governance that Memory Tokens often skip

Latent injection does not eliminate attention limits. The system must still decide what to inject, how much budget to allocate, and when to stop. Representation, alignment, selection, governance, and provenance must be designed together. Optimizing only for compression sacrifices interpretability without solving the original problem.

A summary of the Memory System architecture

That covers a great deal of ground, and the author’s own view of a memory system continued to evolve while writing this article. The discussion uses Linux-inspired terms such as kernel, file system, and executable because a sound architecture should settle on abstractions that are not tied to one implementation. These terms describe the modules and interfaces that must exist, not a recipe for assembling a particular product.

• Kernel / control plane. System 2’s slow loop and scheduler. It decides when to retrieve, how much to retrieve, when to write, when to update, and when to forget, and it makes those decisions an explicit, trainable, evaluable policy. (AgeMem and InfMem are useful references for control protocols and training, but the abstraction does not depend on either paper.) One submodule is the planner / router — scheduler + syscall dispatcher. It compiles an input / query into an executable read/write plan (a sequence of Memory “syscalls”): which views to query, how much evidence each view may take, whether closed-loop retrieval is needed and when to stop, which units are worth writing, which view they belong to, and the candidate-set constraints that UPDATE / DELETE must cite. The implementation may be a rule-based router, an LLM-as-planner, or a learned controller. In every case the decision should be recordable, replayable, and comparable. Policy must not live only in prose.
• File system / storage plane. A data layer centered on an authoritative Raw Ledger plus derived views. The bottom must carry temporal consistency and conflict resolution (when a fact was true). The top must do semantic compression and layered consolidation (turning many events / episodes into higher representations) while always keeping a provenance path. Temporal KGs and recursive consolidation / hierarchical memory are useful analogies.
• Executable / skill plane. Experience is consolidated into executable, reusable procedural units (skill / macro / workflow) so the system reuses “how,” not only “what.” The form of the skill matters less than executability, verifiability, and governability, without which long-term evolution becomes systematic decay.
• Interface / context bridge. The interface between memory and reasoning. It injects external state into the compute core with low overhead, low distortion, and explicit control, and it supports observability and provenance. Some work uses latent tokens / memory tokens as the bridge.
• Learning engine / online adaptation. Without frequently updating weights, it continuously turns interaction feedback into useful improvements. This may take the form of advantage modulation during inference, evolution operators in the skill layer, or block-wise optimization of retrieval policy. The key is that learning occurs in external state and policy, keeping the system pluggable, reversible, and suitable for A/B testing. JitRL-style advantage modulation is one implementation.

4. Recap

Starting from propositions A / B / C and working through the argument, the core conclusions are:

1. Memory is not storage. It is a closed-loop system. Its essence is Raw Ledger (authority) → Views (usable capability) → Policy (control) → Commit (write-back) → Provenance (replay). Drop any piece and the system is ungovernable, unusable, or unable to iterate.
2. System 2 is necessary. Memory capabilities are relatively orthogonal to those of a general-purpose LLM agent. An externalized System 2 provides pluggability, portability, and attribution, even at the cost of some theoretical performance ceiling.
3. The ceiling of non-parametric Memory is set by three bottlenecks: interface bandwidth (injection capacity), retrieval and aggregation error (view approximation), and the learnability / controllability of policy. Policy is the most underestimated of the three.
4. Time is a structural dimension of the architecture, not merely metadata. Bitemporal, time-sliced recall provides the hard constraint that separates “was true” from “is true now.” Do not leave that distinction to the LLM’s self-consistency.
5. The five-part architecture—kernel, file system, executable, bus, and learning engine—is implementation-agnostic. It identifies the modules and interfaces that must exist rather than prescribing how to assemble a particular product.

5. From architecture to implementation: PowerMem [12]

The ledger, kernel, file system, and executable become useful only when they form a runnable, pluggable AI memory system that can continue to accumulate knowledge. The open-source project PowerMem (https://github.com/oceanbase/powermem) is a practical implementation of this Memory OS. It gives AI agents persistent, self-evolving memory and turns dialogue, actions, and feedback into long-lived state. The abstractions above have concrete counterparts in PowerMem.

The session ended. The state is still here

System 2’s first job is to move long-term state out of the model and beyond a single session. PowerMem provides persistent memory, user profiles, and scope management so memory can be isolated by user and agent, yet shared when needed.

The model can change, and the session can be reopened. User preferences and task progress do not have to disappear with the context window. In multi-agent collaboration, clear boundaries also define what each agent should remember and what it is allowed to see.

The same history needs more than one way to look it up

Vector search finds nearby meanings. Full-text search finds names and IDs. Graph search finds relations. Recency reminds the system not to ignore what just happened. PowerMem lets these signals join hybrid retrieval — exactly the Views design from earlier.

If the user asks about “that similar incident last time,” semantic search can help. If they ask for order OB-20260716, keyword search is more reliable. If they ask which project a customer is associated with, a relationship graph is the better tool. Sending every question to a single retrieval method is like remodeling a house with only a screwdriver: earnest, but inefficient.

A memory store also needs regular housekeeping

PowerMem uses a large model for memory extraction, update, and merge, and it adjusts priority with an Ebbinghaus-style time decay. The original forgetting curve describes human memory fading with time. What is borrowed here is “the longer something goes unused, the lower its default weight.”

A new address can replace an old one. A stable preference that appears repeatedly can gain weight. Fragments that have not been used for a long time can recede in priority. The store is no longer a chat archive that only grows; policy-driven lifecycle management begins to take effect.

Time decay is not crude deletion. Explicitly expired, still valid, and merely unused for a long time are three different states. Validity windows, retraction records, and soft ranking remain the key to long-term governance.

Retain experience, then turn it into something the agent can do

PowerMem offers two distillation layers: Experience and Skill. Experience keeps lessons extracted from interaction. Skill turns repeatedly validated lessons into more reusable methods.

That is procedural memory. In addition to knowing that the last migration succeeded, the agent should retain the prechecks, steps, and verification procedures from that run. When a similar task appears, history becomes an actionable reference rather than a collection of old chat logs.

Applications can change without forcing memory to move

The Context Bridge’s worst failure mode is for every application to invent its own private memory format. PowerMem provides a Python SDK, an HTTP service, MCP, and a CLI so chat assistants, developer tools, and agents can access the same backend through different interfaces.

MCP is the Model Context Protocol, a standard way to provide agents with tools and data sources. A CLI is a command-line interface for terminal use. These interfaces connect applications to the Memory OS while a single backend continues to manage the memory content.

Run it on your laptop first, then think about a larger stage

The storage layer leaves room to scale. PowerMem supports OceanBase, embedded seekdb, PostgreSQL, SQLite, and other mainstream databases. Local experiments can begin with a lightweight backend, while long-running services can choose a database designed for high availability. The Memory OS abstractions above do not need to be rebuilt.

You can evaluate the memory system of a long-running agent by asking a few questions: does state persist after the session ends? Is vector search the only way to retrieve history? Can old facts be updated, and does low-value content recede? Can successful experience become a skill? Can several applications share one memory backend? When a local prototype becomes a long-running service, can its storage scale accordingly?

PowerMem already brings these capabilities together in one open-source system. It provides a practical foundation for long-lived assistants, personalized applications, copilots, and multi-agent workflows, as well as a runnable example for understanding a Memory OS. The theory explains why each module exists; running the system shows that these abstractions arise from problems any long-lived agent will eventually encounter.

Closing

The next steps for AI memory are worth watching: finer-grained ledger replay and provenance, more natural bitemporal queries, policies that continuously adapt to real feedback, and more efficient latent-memory injection.

The destination remains some distance away, but the basic structure of kernel, file system, skill plane, bus, and learning engine still applies.

For engineering details, start with the PowerMem GitHub repository [13] and the PowerMem architecture notes [14]. An agent that truly remembers you will not do so because someone wrote “please remember the above.” It will remember because a system like this continues working in the background.

References

[1] AgeMem: https://arxiv.org/abs/2601.01885

[2] JitRL: https://arxiv.org/abs/2601.18510

[3] UMEM: https://arxiv.org/abs/2602.10652

[4] InfMem: https://arxiv.org/abs/2602.02704

[5] SimpleMem: https://arxiv.org/abs/2601.02553

[6] Zep: https://arxiv.org/abs/2501.13956

[7] Graphiti: https://github.com/getzep/graphiti

[8] MemWeaver: https://arxiv.org/abs/2601.18204

[9] ProcMEM: https://arxiv.org/abs/2602.01869

[10] LycheeMemory: https://arxiv.org/abs/2602.08382

[11] MemAdapter: https://arxiv.org/abs/2602.08369

[12] PowerMem: https://github.com/oceanbase/powermem

[13] PowerMem GitHub repository: https://github.com/oceanbase/powermem

[14] PowerMem architecture notes: https://github.com/oceanbase/powermem/blob/main/docs/architecture/overview.md

Benefits

Alibaba Cloud Model Studio (Bailian) is offering dedicated model-compute benefits for the community chapters and labs of the DataWhale × OceanBase course Easy Data x AI — this article comes from one of those community chapters.

Course learners and contributors can claim model-compute support on the Alibaba Cloud × OceanBase benefit page and receive 100 million free tokens (tested for both new and existing users).

New and existing Bailian users, including both individuals and companies, can also participate in token rebate programs with no minimum threshold. Additional OPC innovation support offers up to the equivalent of RMB 1 million in tokens, along with course-specific subsidies for university students. These benefits are intended to support community contributions and hands-on lab work for the course.

Course: https://github.com/datawhalechina/easy-data-x-ai

Benefits: https://opc.aliyun.com/oceanbase?utm_content=g_1000415375

Previous articles

Learn more

Welcome to join the open-source community Discord.