Explaining AI Memory in the Plainest Possible Language

Imagine this: every time you chat with a friend, the other person wipes their memory clean, and every conversation starts from zero — no memory, no context, no progress.

And unfortunately, this is exactly the state of most AI systems today. They’re smart enough, yet they’re missing a key element: the ability to remember.

What Is Memory?

At the current stage, AI applications are moving from Generative AI toward Agentic AI. 2025 is widely seen as the inaugural year of the agent market. Technical discussions around agent architecture are red-hot, and the technology is evolving rapidly. Looking at current trends, development frameworks are gradually shifting their focus — from low-level LLM integration to higher-level abstraction and integration of an agent’s internal components. Standard protocols like MCP and A2A have also been established, defining how agents call tools and interact with one another. The various components of agent architecture are being clearly defined and standardized.

The figure above (from The Rise and Potential of Large Language Model Based Agents: A Survey) defines an agent architecture, which contains several important parts:

1. Perception: gives the agent the ability to perceive its environment and accept multimodal information input.
2. Brain-Decision Making: gives the agent the ability to make autonomous decisions and plans, so it can carry out more complex tasks.
3. Brain-Memory & Knowledge: gives the agent the ability to remember; memory internally stores the agent’s knowledge and skills.
4. Action: gives the agent the ability to interact with the outside world, so that through action and perception the agent can autonomously complete more complex tasks.

What this article explores is the memory capability that is so crucial to an agent:

1. It enables continuous learning: an agent’s knowledge is primarily encoded in the LLM’s static parameters. Memory gives the agent the ability to accumulate and refine knowledge and experience over time. Research has shown that memory-equipped agents perform significantly better — they can summarize lessons from past experiences, learn from mistakes, and steadily improve at their tasks.
2. It maintains conversational coherence and consistent behavior: memory gives the agent longer-horizon context management. In extended conversations, the agent keeps a consistent context and avoids contradicting facts it stated earlier.
3. It enables personalized service: by drawing on past conversations, the agent can infer user preferences and build a mental model of its interactions, delivering experiences better tailored to each user.

How Do We Define Memory?

An LLM is essentially a simulation of the human brain’s neural network, so agent memory can also achieve stronger performance by simulating human memory.

The Structure of Human Memory

Memory is the process by which humans encode, store, and retrieve information, allowing us to retain experiences, knowledge, skills, and facts over time. It is fundamental to growth and to interacting effectively with the world. To figure out the structure of memory, you need to understand how memory is classified and what operations it provides.

Classifying Memory

There are several dimensions along which memory can be classified. A few common ones include:

Classification by Storage Duration

Humans have studied brain memory for a long time. The classification by storage duration was first proposed in the 1968 Atkinson-Shiffrin Memory Model, which divides memory into three types: Sensory Memory, Short-term Memory, and Long-term Memory. Sensory memory holds information the brain captures from the environment, such as sounds and images. This information lasts only briefly in the sensory memory area. Short-term memory holds the information the brain actively processes while thinking. It is also called working memory (proposed in the 1974 Baddeley & Hitch model) and, like sensory memory, can only retain information for a short time. Long-term memory stores knowledge and skills over extended periods.

These categories represent different memory work areas. Sensory memory serves as the brain’s information input area. Short-term memory (or working memory) acts as the brain’s staging and processing area during active thought. Long-term memory is the brain’s persistent information storage area. Short-term memory draws its content from two sources: sensory input and information retrieved from long-term memory. New information the brain generates is also staged in working memory. Information processing typically flows across all three areas — from input through processing to long-term storage.

Classification by the Nature of the Content

By the nature of the content, memory can be divided into Declarative Memory and Non-Declarative Memory; another naming convention is Explicit Memory and Implicit Memory.

The main differences between these two types are:

• Whether it can be described in language: declarative memory can be described in language, such as some piece of knowledge you’ve mastered. Non-declarative memory cannot be described in language, such as some skill you’ve mastered, like riding a bike.
• Whether it requires conscious participation: explicit memory requires conscious, active recall, whereas implicit memory involves no conscious participation, which is why it’s also called muscle memory.

Classification by Stored Content

Humans can retrieve different types of content from memory: we can recall the past — this content is called “experience”; we can also master “knowledge” through memory; and we likewise preserve “skills” through memory. So we can abstractly classify memory content into “experience,” “knowledge,” and “skills.” The more scientific names are:

• Episodic Memory: represents experience, storing events that happened in the past.
• Semantic Memory: represents knowledge, storing the knowledge you understand.
• Procedural Memory: represents skills, storing the skills you’ve mastered.

The above are a few common ways of classifying memory. These classification methods are not contradictory but represent different dimensions, and they have certain relationships with one another. For example, episodic memory is usually explicit memory, because it needs active recall and can be described in language. A skill is usually implicit memory — for instance, riding a bike is muscle memory, requiring neither active recall nor description in language.

We can combine these dimensions to define memory, describing it as a memory of “which type” that exists “in what form” and is “stored in which memory area.” For example, the skill of riding a bike is a “procedural” memory that exists in “implicit” form and is stored in the “long-term memory” area; or, the content in my head while writing this article is a “semantic” memory that exists in “explicit” form and is stored in the “short-term memory” area.

Memory Operations

Memory is the brain’s process of encoding, storing, and retrieving information, so the core operations are:

1. Encode: acquire and process information, transforming it into a form that can be stored.
2. Storage: the process of retaining encoded information in short-term or long-term memory.
3. Retrieval: also called recall — the process of accessing stored information when needed and bringing it back into consciousness.

Memory also includes some other operations:

1. Consolidation: through consolidation, short-term memory turns into long-term memory and is stored in the brain, lowering the chance of being forgotten.
2. Reconsolidation: the process whereby a previously stored memory is reactivated, enters an unstable state, and needs reconsolidation to maintain its storage.
3. Reflection: the process of actively reviewing, evaluating, and examining one’s own memory content to enhance self-awareness, adjust learning strategies, or optimize decisions.
4. Forgetting: forgetting is a natural process.

Agent Memory

As mentioned above, we can combine several dimensions to define memory, describing it as a memory of “which type” that exists “in what form” and is “stored in which memory area.” Agent memory can be classified in the same way, but because of differences in memory storage areas and storage forms, it differs slightly from the classification of human-brain memory.

Differences in Memory Storage Areas

The memory storage areas within an agent mainly include:

1. Context: context is the agent’s short-term memory or working memory area — the window is limited and it is easily forgotten.
2. LLM: contains the bulk of the agent’s knowledge and belongs to the agent’s long-term memory area, containing different types of memory.
3. External memory storage: because the knowledge inside the LLM cannot be updated, memory is extended through external storage; this part also belongs to the agent’s long-term memory area.

Compared with human-brain memory areas: sensory memory and short-term memory correspond to the agent’s context, while long-term memory corresponds to the agent’s LLM and external memory storage.

Differences in Storage Forms

Memory within an agent mainly exists in two forms, which can be simply categorized as Parametric and Non-parametric. Both forms of memory exist in the agent’s short-term and long-term memory areas. For example, the KV-Cache can be considered parametric memory in the short-term memory area, the LLM is parametric memory in the long-term memory area, and external memory storage is non-parametric memory in the long-term memory area.

Classifying Agent Memory

From the content above, we’ve learned that an agent’s memory storage areas and storage forms differ somewhat from the human brain, but they can still be mapped, as shown below.

We classify agent memory along the same dimensions but with different content, and then look at which technical implementations currently exist under each memory category (here we don’t classify by memory type, because it doesn’t affect the classification of technical implementations).

The following is the classification of existing memory implementations excerpted from the MemOS paper:

From the classification above, you can see that agent memory implementations are quite diverse and granular. A few familiar examples: Prompt Engineering optimizes explicit memory in the short-term area; a knowledge base (using RAG) optimizes explicit memory in the long-term area; and model Fine-Tuning optimizes implicit memory in the long-term area.

Agent Memory Operations

The memory operations an agent provides are fairly similar to human-brain memory, also including encoding, storage, and retrieval. Memory encoding includes the acquisition and processing of memory: by processing the content of the working memory area, new memories are discovered and encoded into storable structures. Memory is stored in parametric or non-parametric form; the non-parametric form is usually stored as a Plaintext, Graph, or Structured-Table structure. Memory retrieval is usually achieved via search, with specific techniques including full-text search, vector search, graph search, or hybrid search — the specific search method depends on the content and structure being stored.

Agent Memory Implementations

The technology of Agent Memory is evolving very fast; there are already many open-source and commercial Memory products on the market. Especially in 2025, a batch of new products has emerged at an even faster pace. A Shanghai startup, “Memory Tensor,” recently even secured a 100-million-yuan angel round and open-sourced its own Memory product, MemOS. Clearly the Agent Memory market is developing rapidly, and its commercial prospects have been recognized.

Next, we’ll analyze the implementations of the current Memory products one by one. The main information comes from the papers each has shared; the paper content may already differ from the latest version of the implementation, so if you’re interested, go track the latest open-source progress. Through this analysis, we hope to summarize some technical trends and understand what kind of structural definitions and storage demands the Memory scenario places on underlying storage.

LETTA

LETTA’s technology stems from research on MemGPT and is now an independent commercial product. The approach LETTA proposes borrows the idea of an operating system’s virtual memory paging. This technique was originally developed to let applications handle datasets far larger than the available memory by paging data between main memory and disk. LETTA simulates the implementation of virtual memory, dividing Context into a Main Context and an External Context; when the Main Context runs out of space, it can swap data with the External Context.

To understand the principle, just look at this figure:

The whole thing is divided into Main Context and External Context, analogous to memory and disk. The Main Context is the Prompt Tokens portion and is further divided into three small parts:

• System Instructions: stores the static system prompt.
• Working Context: in a conversation scenario, this part stores key facts, preferences, and other important information about the user, as well as the agent’s persona.
• FIFO: stores the rolling conversation history, including a summary of the conversation records already removed from the FIFO queue, plus the latest conversation records.

The following operations are performed on memory content:

• Recursive Summary: based on the current Recursive Summary and the messages to be removed from the queue, a new Summary is generated; this step compresses information to save Context space.
• Memory update and retrieval: this is handled by a Function Executor. Both operations are fully self-driven, triggered by prompts in the system instructions. The prompt includes two parts: (1) the memory hierarchy with a detailed description of each level, and (2) the callable functions described in natural language, enabling the LLM to access and modify memory.

ZEP

ZEP claims to better meet enterprise needs. Traditional RAG is based on static document data, whereas ZEP can obtain more real-time information based on conversation and business data. Its main innovation lies in the memory storage structure: it self-developed an underlying graph engine, Graphiti, that provides a temporally aware knowledge graph. Since the prior standout was MemGPT (the LETTA above), the paper mainly compares against it. On the DMR Benchmark, ZEP outperforms MemGPT, 94.8% vs. 93.4%. On the LongMemEval benchmark — a test more aligned with enterprise use cases that includes some complex temporal-reasoning tasks — ZEP improves accuracy by 18.5%.

Above is an architecture diagram excerpted from the official site; it contains relatively little information. The core part is all within the knowledge graph it builds. The whole knowledge graph has three levels:

1. Episodic subgraph (you can think of it as storing episodic memory in graph form): contains the original input data — messages, questions, or JSON format — serving as a lossless data store. The Nodes in the graph are episodes, and the Edges connect to the corresponding semantic entities.
2. Semantic subgraph (you can think of it as storing semantic memory in graph form): built on top of the episodic subgraph, extracting semantic entities and relationships from it. The Nodes in the graph are semantic entities, and the Edges represent the relationships between semantic entities.
3. Community subgraph (a higher-level summarization of semantic memory): the Nodes in the graph represent Clusters of strongly connected entities, forming a community; a community contains a high-level summarization of the cluster, and the community Edges connect a community to its members. This part draws on the idea of GraphRAG, achieving a more global summarization of semantic memory by building communities.

The paper explains its psychological basis: it adopts a dual-storage approach of original episodic data plus derived semantic-entity information, reflecting the human memory model in psychology. This model distinguishes memory into episodic memory — the memory of specific events — and semantic memory — the memory of associations between concepts and their meanings. This dual structure lets agents using Zep build more complex and nuanced memory structures, more closely simulating the human memory system.

ZEP’s implementation of memory update and retrieval also has some interesting aspects:

Memory update

1. Memory deduplication: the extracted semantic entities are deduplicated to avoid storing conflicting memories. Deduplication of entities is done by full-text searching on the entity name and entity summary to find similar entities, with the LLM judging duplicates; if a duplicate is determined, the relevant information of the current entity is updated.
2. Temporal information extraction and edge invalidation mechanism: this is a differentiating capability of Graphiti over other knowledge graphs. The graph records the time a fact was produced and the time it takes effect. When new fact data is introduced, the LLM judges duplicates. If contradictory facts with overlapping time are found, the fact is marked invalid and its invalidation time is recorded. This both preserves the latest fact and preserves the change history of the fact.
3. Building the community subgraph with a label-propagation algorithm: the label-propagation algorithm has the advantage of simplicity in dynamic expansion, allowing the system to maintain a stable community subgraph for a longer time as new data continuously enters the graph structure, thereby delaying the need to fully recompute communities. But over a long time, the community structure formed gradually deviates from the result of fully running the label-propagation algorithm. Therefore, the community structure still needs to be periodically recomputed.

Memory retrieval

Abstractly, an input string serves as the search condition and an output string is returned; the output string contains formatted nodes and edges. The detailed retrieval steps are divided into three:

1. Search: search by combining multiple retrieval methods; the search results contain a list of semantic edges, a list of entity nodes, and a list of community nodes — these three graph structures contain the relevant textual information.
2. Reranker: reorder the search results.
3. Constructor: convert the relevant nodes and edges into textual context.

ZEP implements three search functions: semantic-similarity search, BM25 full-text search, and breadth-first search. These three search methods target similarity at different levels: full-text search identifies word-level similarity, cosine similarity captures semantic-level similarity, and breadth-first search reveals context-level similarity — that is, nodes and edges that are closer in the graph tend to appear in more similar conversational contexts. This multi-angle candidate identification maximizes the chance of discovering the optimal context.

MEM0

MEM0 currently has the most stars among open-source Memory frameworks. It provides two implementations: one is Mem0, which does not use a graph, and the other is Mem0-G, which is graph-based. Below we introduce the two implementations separately:

The figure above is the implementation of Mem0; a few main steps include:

1. Memory generation: uses context-aware memory generation, where the context includes the current Q&A + the most recent M messages + the session’s Summary. The conversation Summary is generated asynchronously in the background, running independently of the main processing flow.
2. Memory update: the main goal is to maintain memory consistency and avoid redundancy. It retrieves the N semantically similar memories from the database and, together with the candidate Facts, provides them to the LLM, letting the LLM judge whether memory needs to be added, modified, or deleted — or whether nothing needs to be done.

The figure above is the implementation of Mem0-G; here we mainly summarize the differences from the Mem0 implementation:

• Memory generation: the generation process uses a two-stage pipeline, leveraging LLMs to transform unstructured text into a structured graph representation. First, an entity-extraction module processes the input text, identifying a set of entities and their corresponding types. Next, a relationship-generator module establishes meaningful connections between these entities, generating a set of relationship triples to capture the semantic structure of the information.
• Memory update: for each new relationship triple, we compute the embeddings of the source and target entities, then search for existing nodes whose semantic similarity is above a set threshold. To keep the knowledge graph consistent, a conflict-detection mechanism is implemented to identify potentially conflicting relationships when new information arrives. An LLM-based update resolver judges whether certain relationships should be deprecated, marking them as invalid rather than physically deleting them, thus supporting temporal reasoning. (This mechanism is a bit like ZEP’s.)
• Memory retrieval: uses a dual retrieval mechanism of an entity-centric approach and a semantic-triple approach, enabling Mem0 to efficiently handle both entity-focused specific questions and broader conceptual queries:
  • Entity-centric approach: first identifies the key entities in the query, then uses semantic similarity to locate the corresponding nodes in the knowledge graph. This method systematically explores the inbound and outbound edges of these anchor nodes, constructing a complete subgraph covering the relevant contextual information.
  • Semantic-triple approach: this method takes a more holistic view, encoding the entire query into a single Embedding vector. It then matches this query representation against the textual encoding of every relationship triple in the knowledge graph. The system computes fine-grained similarity scores between the query and all available triples, returning only the triples that exceed a configurable relevance threshold, sorted in descending order of similarity.

Let’s look at Mem0’s performance on the LOCOMO dataset: compared with the Memory frameworks above, its overall performance is better — which may also be why it has the most stars. (There’s also an interesting finding here: for Open Domain and Temporal questions, the graph approach performs better; ZEP, which specializes in graphs, achieves the best score on Open Domain, but on Temporal, Mem0-G scores higher.)

Implementation Summary

Memory is covering more and more scenarios and, at the same time, more and more memory types. Early Memory focused mainly on conversation memory; it has now expanded to many scenarios such as task execution, decision support, and personalized service, and the memory types covered are more comprehensive.

In terms of technical implementation, there are some proven techniques that effectively improve memory performance:

1. Fine-grained memory management: memory is clearly distinguished by scenario, classification, and form. The “divide and conquer” idea has been proven an effective optimization, similar to the optimization idea of Multi-Agent.
2. Combining multiple memory storage structures: the underlying storage structures of memory can be roughly divided into structured information (Metadata, Tags, etc.), plain text (Text-Chunk, Summary, episodic records, etc.), and knowledge graphs. Tag indexes, full-text indexes, vector indexes, and graph indexes are built respectively to improve retrieval. There are also scenario-specific indexes built on top of these atomic index capabilities, such as hierarchical summaries and community graphs. Different storage structures correspond to different scenarios; memory frameworks have evolved from integrating a single structure to combining multiple architectures, bringing a certain improvement in effectiveness.
3. Memory retrieval optimization: retrieval methods are also evolving step by step, from single retrieval to hybrid retrieval, along with tuning and optimization for Embedding and Reranker.

In Closing

AI memory systems represented by Mem0, Zep, and Letta have made important contributions to solving the statelessness problem of LLMs. Through vector databases and knowledge-graph technology, they achieve persistent storage and semantic retrieval of conversation history.

At the same time, they point to a direction for the future development of AI systems — AI systems should be able to accumulate knowledge and experience over time, just like humans:

• Intelligent extraction and retention: extract memory using LLMs, deciding which information is worth remembering based on importance and relevance.
• Contextual understanding: maintain context across interactions to deliver meaningful personalized experiences.
• Continuous learning: AI systems need to be able to learn from every interaction and improve over time.
• Adaptive forgetting: like human memory, an adaptive forgetting mechanism needs to be implemented to prevent information overload.