Goodbye to the Patchwork: A One-Stop Tech Stack for Memory, Retrieval, and the AI Data Engine (Part 2)

Author: Fu Rongfeng, Senior Technical Expert at OceanBase

🌟 Tip: The seekdb used in this article is OceanBase’s open-source AI-native database. You’re welcome to try it at https://github.com/oceanbase/seekdb — we believe it can bring a simpler, more efficient data-management approach to your AI app development!

In the previous article, we introduced seekdb, an AI-native database that can replace the “three-database parallel” setup of a relational database + vector database + document database. In real-world scenarios, with seekdb as the underlying database, the upper layers still need many capability components the business requires — such as retrieval, context engineering, and memory. Because LLMs can’t include an enterprise’s private knowledge during training, and struggle to keep up with the latest information, we introduce RAG (Retrieval-Augmented Generation) to solve this. When a user asks a question, the system first retrieves relevant documents from an external knowledge base, then feeds that content as context to the LLM to help it generate more accurate and timely answers. Put simply, RAG = “look up the references + write the answer” — so the LLM no longer “guesses from memory,” but instead has “evidence to back it up.”

RAG architecture evolution: from Naive RAG to Modular RAG

RAG’s development has gone through three typical stages: Naive RAG → Advanced RAG → Modular RAG, gradually evolving from a simple pipeline into a flexibly assembled modular system.

1. Naive RAG: the basic paradigm

The most primitive RAG architecture contains three core steps.

• Indexing: chunk documents and embed them as vectors.
• Retrieval: retrieve relevant chunks based on the user’s query.
• Generation: feed the retrieved results into the LLM to generate an answer.

This approach is simple in structure and easy to implement, performing well in general scenarios — but it lacks the ability to optimize retrieval quality.

2. Advanced RAG: retrieval enhancement

To improve recall, Advanced RAG introduces enhancement mechanisms before and after retrieval, significantly boosting retrieval accuracy and avoiding “garbage in, garbage out.”

Pre-Retrieval

• Query Rewrite: semantically rewrite the user’s question to improve match precision.
• HyDE (Hypothetical Document Embedding): first generate a hypothetical answer, then use it for retrieval to improve relevance.

Post-Retrieval

• Rerank: use a lightweight model to reorder the recalled results.
• Filter: filter out invalid or low-quality chunks to reduce noise.

3. Modular RAG: a modular redesign

As use cases grew more complex, Advanced RAG evolved into Modular RAG, enriching the five stages — Indexing, Pre-Retrieval, Retrieval, Post-Retrieval, and Generation — and modularizing them. The whole flow is decomposed into multiple pluggable modules that can be combined as needed, so developers can freely assemble the RAG pipeline best suited to their business needs. It includes the following modules.

• Chunk Optimization: optimize the text-chunking strategy to improve context completeness.
• Structural Organization: build a knowledge hierarchy to support multi-granularity retrieval.
• Query Transformation / Expansion: expand query dimensions to improve recall breadth.
• Retriever Selection: support hybrid retrieval (keyword + vector + SQL, etc.).
• Compression & Selection: compress long documents and select the best chunks.
• Verification: verify whether the output is compliant and free of hallucination or privacy leakage.
• Routing: choose different processing paths based on the question type.
• Orchestration: control the execution flow, deciding whether retrieval is needed and when to generate.
• Knowledge Guide: guide the reasoning path, performing structured reasoning together with a knowledge graph.

In summary, Naive RAG suits quick validation and simple Q&A, Advanced RAG improves retrieval quality, and Modular RAG achieves high flexibility and extensibility, able to handle complex and diverse AI application scenarios.

The challenge of putting RAG into production: a demo in a week, struggling for half a year

But once RAG goes into production, it exposes many problems, including missing content, missing highly relevant content, content lost after reranking, content that fails to be extracted, formatting errors, too little or too much detail, incomplete content, scalability problems, structured-data processing, complex PDFs, context issues, and model safety. At root, these fall into two categories: document-parsing problems and retrieval problems. PowerRAG addresses both of these and adds some new capabilities.

PowerRAG: helping improve RAG effectiveness

PowerRAG is a RAG product deeply optimized and further developed on top of the open-source project RAGFlow, released under the Apache 2.0 license. It uses OceanBase as an integrated data-processing foundation, integrating core flows such as document parsing, chunking, storage, and retrieval all within OceanBase to provide high-performance, highly available data support. Compared with the original RAGFlow, PowerRAG mainly enhances and optimizes three key modules — document processing, data retrieval, and effectiveness evaluation/feedback — and offers atomic APIs (for example, parsing and chunking).

Document parsing: building a knowledge source AI can understand

Traditional document chunking often causes semantic fractures and information loss. PowerRAG achieves high-quality knowledge input through multimodal parsing and intelligent chunking. Below, using a relatively complex document as an example, we walk through the document-processing flow and its modules.

1. Document parsing and chunking: identify different modules such as headers/footers, paragraphs, images, and tables, and process each module through a different flow.
2. Intelligent filtering: automatically identify and remove meaningless content (such as bare page numbers) to avoid polluting the knowledge base.
3. Paragraph context preservation: since paragraph content can be lengthy, introduce heading information to rebuild the logical connections between paragraphs.
4. Image semantic recognition: use a vision model to semantically query images, crop images for things like flowcharts and pie charts, and use a dedicated model to extract text descriptions.
5. Table structure recognition: convert tables into structured fields (JSON/key-value pairs) to improve searchability.
6. Finally, each chunk becomes a “semantically complete, clearly structured” knowledge unit that supports efficient subsequent retrieval and generation.

Knowledge retrieval: fully leveraging the database’s hybrid-retrieval capabilities

PowerRAG is built on OceanBase-CE/seekdb and fully supports full-text indexing, scalar + vector hybrid retrieval (including pre- and post-filtering), and other hybrid-retrieval modes, solving the insufficient retrieval capability and lagging performance of traditional approaches.

Full-text + vector

• Tokenizer: a built-in high-performance Chinese tokenizer, with plugin-based extension for minor-language tokenization such as Korean, Japanese, and Thai, meeting the needs of global scenarios.
• Real-time index updates: unlike products such as Elasticsearch that have index latency, OceanBase-CE/seekdb supports “write and it takes effect immediately,” which is crucial for the “reflect-and-write-back” mechanism in RAG — for example, when an agent finds an error and immediately updates the knowledge, the next query reflects it.
• NL Mode (natural-language query): the user’s raw question (e.g., “Is there a laptop with more than 16GB of memory?”) can be used directly for full-text retrieval without manual tokenization at the application layer. The BM25 token-scoring algorithm used by the full-text index is the same as the one used in retrieval, ensuring token alignment and avoiding “can’t find it” problems.

Scalar + vector

The system has a built-in optimizer that dynamically decides the execution order based on the filter rate: at a high filter rate, do scalar filtering first and then vector retrieval (pre-filtering); at a low filter rate, do the reverse (post-filtering). It even supports advanced strategies such as iterative filtering, maximizing performance.

JSON + vector

It supports efficient parsing and indexing of many other data types such as JSON. In RAG scenarios, each document chunk usually carries metadata (such as source, category, time). OceanBase-CE/seekdb fully supports online indexing and querying of JSON fields, avoiding the awkward situation of “can only store, can’t query.”

Effectiveness evaluation: letting RAG “evolve”

Launching a RAG system is just the starting point; continuous optimization is what matters. PowerRAG introduces a full-pipeline effectiveness-evaluation and feedback mechanism, giving the system the ability to “self-evolve.”

BadCase analysis and governance

• Discover low-quality answers through anomaly monitoring.
• Locate problems via root-cause analysis (AI classification, categorization, distribution, solutions).
• Provide task management (badcase task distribution, progress tracking) and solution configuration to drive closed-loop fixes.

GoodCase mining

• Capture answers users approve of.
• Generate representative cases for training and optimizing model preferences.
• Support data annotation and scenario accumulation.

Prompt management

• Provide a prompt library, version management, and call tracing.
• Support quick rollback to historical versions, avoiding service degradation from faulty adjustments.

Evaluation

• Support evaluation-template design, execution, and result analysis.
• Verify whether new modules and new models fit the current scenario.

Observability and visualization

• Multi-source pipeline data collection and real-time processing.
• Structured observability data with support for visual display.
• Full-pipeline observability to support rapid localization and optimization.

Typical use cases of PowerRAG

As an enterprise-grade RAG platform deeply optimized on top of RAGFlow, PowerRAG has been successfully deployed across many complex business scenarios. Its core strengths lie in deep document understanding, high-precision retrieval and recall, and atomic API integration, making it suitable for enterprise applications of different sizes and needs.

Data-intensive RAG scenarios: processing complex, high-value documents

Real case: a financial institution’s quarterly/annual financial report Q&A system. In specialized fields like finance and auditing, documents usually contain abundant tables, charts, scanned images, and complex layouts (such as multi-column layouts and nested headings). Traditional methods struggle to extract key information accurately, leading to low-quality knowledge bases.

PowerRAG’s core strengths:

• Supports industry-leading SOTA parsing models (such as dots.ocr and MinerU), accurately recognizing images, tables, and text in PDFs and scanned files.
• Can extract structured data (such as income statements and balance sheets) from complex layouts and generate searchable chunks.
• Preserves the original context relationships, ensuring generated answers have a factual basis.

Application value: turning “incomprehensible financial reports” into “queryable knowledge assets,” supporting advanced applications like intelligent Q&A, compliance review, and trend analysis.

Q&A scenarios requiring precise citations: supporting trustworthy knowledge output

Real case: a manufacturing industry knowledge base for specialized technical support and troubleshooting. In fields like industry and IT operations, users not only need answers but also require a clear source for every step (e.g., “follow Step 3 according to Chapter 5 of the Equipment Maintenance Manual”). This demands high-precision recall + traceable reasoning.

PowerRAG’s core strengths:

• Achieves multi-path recall + fused reranking, supporting hybrid retrieval over vector, BM25, and custom scoring.
• Combines semantic relevance and keyword matching to improve result accuracy.
• All recalled chunks are linked to their original document positions, supporting “citation tracing” to boost user trust.

Application value: building an “explainable, verifiable” professional-grade knowledge Q&A system that meets high-reliability business needs.

Microservice integration scenarios: a high-performance RAG microservice invoked as an upstream capability module, with API integration into platforms like Dify

Real case: a hybrid-deployment enterprise content management (ECM) system. Many enterprises already have mature content-management platforms (such as ECM, OA, and knowledge-base systems) and want to introduce AI capabilities without restructuring their existing architecture. PowerRAG provides a lightweight, low-coupling integration solution.

PowerRAG’s core strengths:

• Provides atomic API interfaces, including document parsing, intelligent chunking, and vector/full-text recall.
• Supports quick integration into mainstream AI platforms like Dify and LangChain via an SDK.
• Can be deployed as a standalone microservice, seamlessly integrating into existing systems.

Application value: empowering traditional systems in a plugin-style manner to achieve an intelligence upgrade, lowering the cost and technical barrier of transformation.

In an advanced AI agent, beyond RAG capabilities, a memory capability is also needed. If PowerRAG (Retrieval-Augmented Generation) is one important way to implement context engineering, then memory is the supporting technology that provides continuous, structured context for RAG (and for the broader agent system). The next article will tell the story of OceanBase’s practice with memory capabilities in context engineering.