Building Persistent Memory for an AI Agent With seekdb: From "Full Context" to "Precise Recall"

This article explains how to use the seekdb-js SDK + Qwen3 Max (via OpenRouter) to implement an efficient vector memory system for a Node.js AI Agent.

Full code repository: https://github.com/kejun/seekdb-agent-memory

Background: Why Are Traditional Memory Approaches Inefficient?

When using LangGraph or a custom AI Agent, persistent memory is a core requirement. However, traditional memory approaches have an obvious efficiency problem: they always pass the entire message history as context to the LLM, even when those messages are completely irrelevant to the current question.

For example: when you simply greet the Agent with a “hello,” the system still stuffs the entire content of the past 50 conversation turns into the prompt. This redundant information not only wastes Tokens but can also interfere with the quality of the model’s answers.

Real-world consequences:

• Token costs skyrocket (measured at 10–20× the actual demand)
• Response latency increases
• The model’s attention is diluted, lowering answer quality

seekdb’s solution: Store messages as embedding vectors, and use vector similarity search to recall only the historical messages most relevant to the current question.

Explaining the Core Concepts

1. What Are Embedding Vectors?

Computers can’t understand human language; they can only process numbers. Embedding vectors convert text into a list of numeric values that capture semantic information.

• For example: “I like watching AI tutorials” → [0.12, -0.45, 0.88, ...]
• Qwen3 Embedding 8B generates 4096-dimensional vectors (note: not 1024-dimensional)
• Semantically similar sentences have vectors that are closer together in the multidimensional space

2. Vector Similarity Search

If you directly ask a computer whether “I like watching AI tutorials” and “I love watching YouTube AI videos” are similar, it can’t answer. But if you compare their embedding vectors, the computer can compute a definite similarity score.

seekdb’s advantages:

• Based on OceanBase, it supports large-scale vector data
• Native support for vector storage and similarity search
• Easier to deploy than PostgreSQL + PGVector

3. Distance Functions and Cosine Similarity

seekdb supports multiple distance computation methods:

• Cosine Similarity: the most common, ranging [-1, 1]
• L2 Distance (Euclidean Distance): the straight-line distance in vector space

Key formula:

Cosine Similarity = 1 - Cosine Distance

Meanings of cosine similarity values:

• 1.0: perfectly similar (0° angle)
• 0.0: unrelated (90° angle)
• In practice, > 0.7 usually indicates high relevance

Technology Selection

Qwen3 Max + Qwen3 Embedding

Component	Model	Dimensions	Notes
LLM	qwen/qwen3-max	-	128K context, $1.6/M input
Embedding	qwen/qwen3-embedding-8b	4096	High quality, pairs well with Max
Embedding	qwen/qwen3-embedding-0.6b	1024	Lightweight, lower latency

Comparing the Recall Strategies

Strategy 1: Fixed-Count Recall (Limit-based)

Always returns the N most similar historical messages.

Applicable scenario: cost-sensitive applications that need predictable Token costs.

Strategy 2: Threshold Recall (Threshold-based)

Returns only messages whose similarity exceeds a threshold (e.g. ≥ 0.75).

Applicable scenario: prioritizing answer quality and willing to accept a dynamic context length.

Strategy 3: Hybrid Recall (Recommended)

First filter by threshold, then cap the count. Balances quality and controllability.

Full Implementation Code

The code below comes from the actual repository: https://github.com/kejun/seekdb-agent-memory

1. Install Dependencies

npm install seekdb @seekdb/qwen dotenv

2. Environment Variable Configuration (.env)

# OpenRouter API Key
OPENROUTER_API_KEY=your_key_here

# SeekDB connection config
SEEKDB_HOST=127.0.0.1
SEEKDB_PORT=2881
SEEKDB_USER=root
SEEKDB_PASSWORD=
SEEKDB_DATABASE=test

# Embedding config
EMBEDDING_MODEL=qwen/qwen3-embedding-8b
EMBEDDING_DIMENSION=4096  # The 8B model is 4096-dimensional

# LLM config
LLM_MODEL=qwen/qwen3-max

3. Database Connection Configuration

// src/config/database.js
import { SeekdbClient } from 'seekdb';
import dotenv from 'dotenv';

dotenv.config();

/**
 * Create a SeekDB client
 */
export async function createClient() {
  return new SeekdbClient({
    host: process.env.SEEKDB_HOST || '127.0.0.1',
    port: parseInt(process.env.SEEKDB_PORT || '2881'),
    user: process.env.SEEKDB_USER || 'root',
    password: process.env.SEEKDB_PASSWORD || '',
    database: process.env.SEEKDB_DATABASE || 'test',
  });
}

/**
 * Get the Embedding dimension (supports configuration via environment variable)
 */
export function getEmbeddingDimension() {
  const DEFAULT_DIMENSION = 4096;
  const raw = process.env.EMBEDDING_DIMENSION;

  if (!raw) return DEFAULT_DIMENSION;

  const dim = parseInt(raw, 10);
  if (!Number.isInteger(dim) || dim <= 0) {
    console.warn(`[config] Invalid EMBEDDING_DIMENSION="${raw}", using ${DEFAULT_DIMENSION}`);
    return DEFAULT_DIMENSION;
  }

  return dim;
}

/**
 * Custom OpenRouter Embedding function
 */
class OpenRouterEmbeddingFunction {
  constructor(config) {
    this.apiKey = config.apiKey;
    this.modelName = config.modelName;
    this.baseUrl = 'https://openrouter.ai/api/v1';
  }

  get name() {
    return 'openrouter-qwen-embedding';
  }

  async generate(texts) {
    const embeddings = [];

    for (const text of texts) {
      const response = await fetch(`${this.baseUrl}/embeddings`, {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${this.apiKey}`,
          'Content-Type': 'application/json',
          'HTTP-Referer': process.env.APP_URL || 'http://localhost',
          'X-Title': process.env.APP_NAME || 'SeekDB Agent',
        },
        body: JSON.stringify({
          model: this.modelName,
          input: text,
        }),
      });

      if (!response.ok) {
        throw new Error(`Embedding API error: ${await response.text()}`);
      }

      const data = await response.json();
      embeddings.push(data.data[0].embedding);
    }

    return embeddings;
  }
}

export function createEmbeddingFunction() {
  return new OpenRouterEmbeddingFunction({
    apiKey: process.env.OPENROUTER_API_KEY,
    modelName: process.env.EMBEDDING_MODEL || 'qwen/qwen3-embedding-8b',
  });
}

4. The Core Memory Management Class

// src/memory/AgentMemory.js
import { createEmbeddingFunction, getEmbeddingDimension } from '../config/database.js';

export class AgentMemory {
  constructor(client, collectionName = 'chat_memory') {
    this.client = client;
    this.collectionName = collectionName;
    this.collection = null;
  }

  /**
   * Initialize the collection
   */
  async init() {
    const embeddingFunction = createEmbeddingFunction();
    const embeddingDimension = getEmbeddingDimension();

    this.collection = await this.client.getOrCreateCollection({
      name: this.collectionName,
      configuration: {
        dimension: embeddingDimension,
        distance: 'cosine',
      },
      embeddingFunction,
    });

    console.log(`Collection ready: ${this.collection.name}`);
  }

  /**
   * Store a conversation message
   */
  async store(role, message) {
    const id = `${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;

    await this.collection.add({
      ids: id,
      documents: message,
      metadatas: { role, timestamp: Date.now() },
    });
  }

  /**
   * Recall relevant history
   * @param {string} query - the query text
   * @param {Object} options - options
   *   - strategy: 'threshold' | 'limit' | 'hybrid'
   *   - threshold: similarity threshold (default 0.75)
   *   - limit: number to return (default 5)
   *   - role: optional, filter by role ('user' | 'assistant')
   */
  async recall(query, options = {}) {
    const {
      strategy = 'threshold',
      threshold = 0.75,
      limit = 5,
      role,  // new: role filter
    } = options;

    const where = role ? { role } : undefined;

    switch (strategy) {
      case 'threshold':
        return this._recallByThreshold(query, threshold, { where, limit });
      case 'limit':
        return this._recallByLimit(query, limit, { where });
      case 'hybrid':
        return this.recallHybrid(query, { threshold, limit, where });
      default:
        throw new Error(`Unknown strategy: ${strategy}`);
    }
  }

  async _recallByThreshold(query, threshold, options = {}) {
    const { where } = options;

    const results = await this.collection.query({
      queryTexts: query,
      where,
      nResults: 50,
    });

    const memories = [];
    const ids = results.ids[0];
    const documents = results.documents[0];
    const distances = results.distances?.[0] || [];
    const metadatas = results.metadatas?.[0] || [];

    for (let i = 0; i < ids.length; i++) {
      const similarity = 1 - (distances[i] || 0);

      if (similarity >= threshold) {
        memories.push({
          id: ids[i],
          role: metadatas[i]?.role || 'unknown',
          message: documents[i],
          similarity: parseFloat(similarity.toFixed(4)),
          timestamp: metadatas[i]?.timestamp,
        });
      }
    }

    return memories;
  }

  async _recallByLimit(query, limit, options = {}) {
    const { where } = options;

    const results = await this.collection.query({
      queryTexts: query,
      where,
      nResults: limit,
    });

    // Process the results...
    return memories;
  }

  async recallHybrid(query, options = {}) {
    const { threshold = 0.6, limit = 10, where } = options;
    const thresholdResults = await this._recallByThreshold(query, threshold, { where, limit });
    return thresholdResults.slice(0, limit);
  }
}

5. A Smart Agent Example

// src/demo/chat-demo.js
import { createClient } from '../config/database.js';
import { AgentMemory } from '../memory/AgentMemory.js';
import { OpenRouterClient } from '../llm/OpenRouterClient.js';

export class ChatAgent {
  constructor() {
    this.memory = null;
    this.llm = new OpenRouterClient();
    this.client = null;
  }

  async init() {
    this.client = await createClient();
    this.memory = new AgentMemory(this.client, 'chat_memory');
    await this.memory.init();
  }

  async chat(userMessage) {
    // Smart detection: is this a query about personal information?
    const isProfileQuery = /\b(I am|I'm|my name|my job|my profession|good at|what do I do|who am I)\b/i.test(userMessage);

    // Dynamically choose a recall strategy based on query type
    const recallOptions = isProfileQuery
      ? { strategy: 'limit', limit: 3, role: 'user' }  // Personal-info query: only what the user has said
      : { strategy: 'threshold', threshold: 0.65, limit: 5, role: 'user' };

    // Recall relevant history
    const relevantHistory = await this.memory.recall(userMessage, recallOptions);

    // Build the context
    const context = relevantHistory
      .map(h => `${h.role}: ${h.message}`)
      .join('\n');

    const systemPrompt = relevantHistory.length > 0
      ? `Here is the conversation history relevant to the current question:\n${context}`
      : 'You are a helpful AI assistant.';

    // Call the LLM
    const response = await this.llm.chat([
      { role: 'system', content: systemPrompt },
      { role: 'user', content: userMessage },
    ]);

    // Store the conversation
    await this.memory.store('user', userMessage);
    await this.memory.store('assistant', response);

    return response;
  }
}

// Usage example
const agent = new ChatAgent();
await agent.init();

await agent.chat('Hi, I am a programmer and I love writing code');
await agent.chat('What am I good at?');  // Can recall "programmer" and "writing code"
await agent.chat('How is the weather in Beijing?');  // Irrelevant history is filtered out

Key Feature: Role Filtering

In real-world applications, we usually care only about what the user themselves has said, not the Agent’s replies. The role parameter makes this possible:

// Recall only what the user themselves has said
const memories = await memory.recall('What is my name?', {
  strategy: 'limit',
  limit: 3,
  role: 'user',  // Key: only query messages with the user role
});

This is especially useful when handling personal-information queries, as it avoids recalling irrelevant content such as the Agent’s polite replies.

Effectiveness Comparison

Approach	Messages Passed	Token Consumption	Latency
Full context	995 messages	Baseline 100%	Slow
seekdb + Limit	5 messages	~5% (95% saved)	Fast
seekdb + Threshold	18 messages (dynamic)	~15% (85% saved)	Fast

Summary

Core insights:

1. The key to memory isn’t “how much you store,” but “how accurately you recall”
2. Vector similarity search is the ultimate solution for semantic memory
3. Dynamically choosing a recall strategy based on query type works better

Tech stack combination:

• Vector DB: seekdb (OceanBase)
• LLM: Qwen3 Max via OpenRouter
• Embedding: Qwen3 Embedding 8B (4096-dimensional)

Full code: https://github.com/kejun/seekdb-agent-memory