The Database Evolution of the AI Era — From Vectors to Hybrid Search

Notes:

1. This article is just a personal take on database trends. It does not go deep into the implementation principles of vectors and hybrid search; it’s a very accessible, easy-to-understand explainer that requires almost no background knowledge, so read on with confidence.
2. As for articles on the principles and best practices of hybrid search, I’ll write them when the time is right. If you’re interested, follow the WeChat account [Lao Ji’s Tech Talk].

Background

Classifying Data

I generally divide the data types in a database into three simple categories:

1. Structured data: we can treat the basic data types of traditional databases all as structured data. Each value can be regarded as an “atomic” piece of data — for example, int, double, char, varchar.
2. Semi-structured data: things like JSON / GIS / XML. A semi-structured object nests and organizes many basic data types (semi-structured data is like a “molecule,” composed of “atoms” plus the corresponding “linking structures” nested together). We call this semi-structured data.
3. Unstructured data: in plain terms, this is “images, text, audio, video.” Such data shares one characteristic — it’s large. This characteristic means sorting this kind of data is meaningless, and you can’t really compute on it.

The capability boundaries of databases:

• Relational databases handle structured data well, because it’s very easy to compress and store structured data (beyond general-purpose compression algorithms, you can also apply various targeted encoding-based compressions based on the characteristics of each data type), and to do data computation well — basic comparison, sorting, aggregation, scanning, expression evaluation, and so on.
• Semi-structured data has, in practice, always relied on specialized databases, and most of it is stored in NoSQL databases. For example, Mongo’s basic building block is the Document, Redis has all sorts of complex data structures like List / Hash / Set, and GIS generally uses a specialized database such as PostGIS.

Actually, relational databases can handle semi-structured data fairly well too. Take JSON: you can access specific data based on a Path, and a JSON Path is a bit like the column concept of a relational table. Moreover, semi-structured data can be decomposed into structured data, and with the capabilities of a relational database we can store and compute on semi-structured data.

Beyond the structured and semi-structured data mentioned above, there’s another category called “unstructured data”:

• Before the AI era arrived, unstructured data, as far as databases were concerned, could only be stored, not computed on. As for storage, unstructured data is large and basically has to be stored as large objects (e.g., blob), which is also suboptimal compared to other types (and expensive).
• In the real world, the vast majority of unstructured data is stored on a local file system — for example, object storage and block storage. A typical characteristic of these stores is that they’re cheap.
• In the general sense, unstructured data can currently only be handled generically by vector databases. If you single out text, then in addition to vectors you can also do full-text processing. For more on full-text search, see: “An Introduction to Full-text Index Capabilities”.

What Changed for Databases in the AI Era?

In the real world, unstructured data accounts for over 80%. But the vast majority of this data is merely stored, not computed on.

And in the AI era, data only creates value when it can be computed on.

LLMs

GenAI brought us general-purpose LLMs.

In terms of data processing, LLMs can be summarized as having two kinds of ability:

1. Directly processing unstructured data. For example, input a piece of text and have the LLM summarize it.
2. Extracting structured features from unstructured data, then storing them in a relational database for computation. For instance, for an image, I use a prompt to have the LLM extract tags from the picture — such as “wearing a dress,” “long hair,” “woman,” “by the lake,” and so on. Once the tags are extracted, the database can perform analytical computation.

Before the GenAI era, could unstructured data be processed? Actually, yes — but it was very “complex” and required “customization.”

The core technique back then was machine learning. In the GenAI era, this technology has a lower barrier, is more general, and is cheaper.

Embedding Models

LLM dimensions are measured in B (Billions), while embedding models are generally around 1K dimensions.

An embedding model captures the “hidden” features of unstructured data and represents those features as a high-dimensional vector. By computing the distance between two high-dimensional vectors (e.g., Cosine / IP distance), it approximates the similarity between two pieces of unstructured data. For related content, see: “A Gentle Introduction to Vector Databases”.

A vector is actually a piece of semi-structured data (as discussed earlier, databases can only efficiently handle structured / semi-structured data). For example, a 1024-dimensional vector is really an Array of 1024 elements, each of which is a Float.

In traditional databases, unstructured data is not comparable — for example, a binary comparison of two photos is completely meaningless. With an embedding model, two pieces of unstructured data become comparable (though it differs somewhat from the magnitude comparisons of traditional relational databases; here it’s a “similarity” comparison). It’s precisely this “becoming comparable” that opens the door to unstructured-data processing.

Note:

For traditional relational databases, the most fundamental data processing is in fact comparison. Sorting, filtering, and even scanning are all built on the ability to compare.

The figure below is a very common two-dimensional example. Wealth and looks are the two dimensions of the vectors. Each dimension of a vector is one “hidden” feature of the unstructured data. As you can see, in the spatial coordinate system, the closer two creatures are, the more similar they are (this is also the principle behind the simplest vector database).

With LLMs, Why Do We Still Need Vector Databases?

You can use proof by contradiction. First imagine: in the AI era, if there were only LLMs and no vector databases, what would happen?

Take Image-to-Image Search as an Example

Suppose I have 1 million images, and every time I want to find similar images, I bypass the vector database and pass everything straight to the LLM. You’d quickly discover: the LLM is super slow. Even DeepSeek R1 without reasoning can only produce results on the order of seconds. (Traditional relational databases process data on the order of milliseconds — that’s three orders of magnitude apart.) So in the entire data chain of an AI application, the vector database is not the bottleneck; the LLM is.

• LLM: even a lightweight model with 1 billion parameters has to traverse all parameters and do matrix operations to generate each token (word/character), and producing a passage repeats this dozens of times — it’s like “hiring 1 billion accountants, each doing one step, just to produce a single sentence.” The room for hardware and algorithmic optimization is far smaller than for data retrieval.
• Vector database: even handling billions of vectors, using the database’s most common “index + parallelism,” you can keep retrieval latency under a hundred milliseconds.

So LLMs are not suited to real-time processing of large volumes of data; vector databases are a better fit. When dealing with massive data, the only sensible approach is: the vector database does the initial filtering, and the LLM does the summarization / secondary processing.

Take a Knowledge Base as an Example

LLMs hallucinate — this is universally acknowledged. An LLM can answer a lot, but much of it may be “nonsense.” The main reason is that an LLM’s knowledge comes from its training dataset, yet a lot of data inside an enterprise is not public.

The context an LLM uses to process data has a “window size” — for example, GPT-4 is 8K. On one hand, there’s no way to stuff an entire enterprise’s data into the LLM at once for processing; on the other, when an LLM tackles a specific problem, it also needs to be more “focused.” In other words: a small amount of highly relevant information is enough, and you don’t want a lot of irrelevant information.

In this process, you can’t help but bring in the vector database, letting it filter out — as fast as possible — the knowledge most similar to the user’s question, and then hand it to the LLM to summarize.

Summary

In short, the most common need in the AI era is near-real-time retrieval over massive unstructured data — and right now, only vector databases can do this. A typical AI solution is “vector database + LLM” working together: the vector database’s “near-real-time retrieval” + the LLM’s “general intelligence.”

So without a doubt, the vector database is already a key link in the evolutionary path of databases in the AI era.

Lately, beyond dedicated vector databases, other relational databases, NoSQL databases, and various search engines have also begun gradually supporting vector storage and retrieval.

At present, different databases may take slightly different evolutionary paths for their vector capabilities. But in the near future, they’ll most likely converge — that is, the vector type will become a basic data type in databases, just like the number type and the string type.

One more word of explanation: in the figure above, the vector index is pulled out separately from ordinary secondary indexes mainly because vector retrieval is expensive and the results only need to be approximate rather than exact. So there’s a distinctive form of index — the vector index.

By the same token, full-text indexes, spatial indexes, and the like exist for roughly similar reasons.

What Is Hybrid Search?

Lately, many companies and experts in the database industry have been publishing articles on the principles and practice of hybrid search. Why? Because databases that support hybrid search have gradually become a must-have in the AI era.

(I especially hope Brother De can get some good rest and stop grinding out mind-expanding hybrid-search principle articles on the weekends. One of Brother De’s sleepless weekend nights may create many, many sleepless nights for fellow editors in the industry.)

Why am I bold enough to say this? (I mean: hybrid search is a must-have — not that Brother De is the king of grinding.)

1. When using a database in production, permissions are almost always a consideration. The main way to handle permissions is to Join the document table, the employee permission table, and various other tables. I’ve hardly ever seen a user or workload use “pure” vector retrieval; there’s almost always scalar & vector hybrid retrieval. That scalar part relies on the retrieval capability of a traditional database.
2. To make retrieval more accurate, for RAG solutions the current industry consensus is: full-text + vector hybrid retrieval. In the future, database vendors without full-text search capability will struggle to meet RAG-related needs.
3. Other approaches such as GraphRAG are also quite active in both academia and industry. So the need for vector + graph hybrid retrieval is becoming increasingly common.

So what is hybrid search? Rather than giving a definition, it’s easier to just use an example.

For instance, a restaurant-recommendation AI Agent system built on Ant Group’s Baibaoxiang turns a user’s natural-language question into a search over the knowledge base.

In the question shown in the figure above:

• “within 500 meters” is a query based on spatial location (GIS).
• “average spend of 25 yuan, rating of 4.5 or above” is a query based on traditional scalars.
• “no queue” is a semantic retrieval based on vectors, drawing on users’ reviews of the shop.

Hybrid Search — The Engine of AI Applications

AI Applications Rely Heavily on the Knowledge Base

The figure above is the overall agent architecture of a well-known insurance-industry ISV.

The dependencies at each level can be simplified as follows:

The architecture of the vast majority of AI applications can be simplified into the diagram above — for example: general-purpose knowledge-base platforms (all kinds of Q&A assistants), agent recommendation systems (Fliggy’s travel recommendations), Cursor-like intelligent coding assistants, industry AI assistants (database operations and monitoring agents), and so on.

Now, a couple more words about the diagram above:

• An agent’s effectiveness rests on the “knowledge base,” the “model,” and the “business logic.” In a specific agent application, the business logic is fixed, and the model can only be chosen from a general-purpose model matrix — so the knowledge base becomes the most critical factor in an agent’s effectiveness.
• The effectiveness of a knowledge base depends on many aspects:
  • Whether the data is high quality — the degree of data cleaning, labeling, and extraction. Good data is the beginning of good results.
  • Whether the query statement is high quality — whether it has been adequately rewritten (e.g., fixing typos / substituting synonyms), expanded (asking from multiple angles), and filled in semantically/contextually.
  • Whether data retrieval is accurate — the industry has reached a consensus that combining multiple retrieval methods (vector retrieval, keyword retrieval, etc.) is an effective way to improve the final results.
  • Whether data retrieval is efficient — introducing multiple retrieval methods brings far more computation. How to return results to the application faster while ensuring accuracy, shortening retrieval latency, is an important part of the application experience.

Based on the discussion above, we can essentially reach a consensus: the knowledge base is a very important factor in the effectiveness of an AI application, and the effectiveness of a knowledge base depends heavily on the quality and efficiency of data retrieval. So for a database in the AI era, it must provide users with accurate & efficient real-time hybrid search and data-processing capabilities.

A Knowledge Base Relies Heavily on Hybrid Search

The ultimate goal of a knowledge base (RAG, retrieval-augmented generation) is still the quality of the model’s generated output. The request context window of various language LLMs is generally 128K or less, and the request fed to the model contains a lot of information — prompts, memory, and information retrieved from the knowledge base.

LLMs have fairly strong generalization, but the final result still depends on the context information in the whole request. Ideally, we’d like to stuff more information into the LLM (if possible, it would be best to force-feed the entire knowledge base to the LLM), which would yield the best results.

But the reality is that an LLM’s context window is fairly small, and the fuller you stuff it, the worse the model may perform. Constrained by the LLM’s context window size, every token is precious, so the data ultimately fed to the LLM should be as relevant, concise, and accurate as possible.

Below is a line chart from a report by Chroma founder Jeff Huber when he shared the concept of Context Engineering. You can intuitively see that as the context length keeps increasing, the model’s performance degrades noticeably.

Although the DeepSeek-OCR model recently redefined AI’s input and output (from text to pixels) and can compress data to a certain extent (see: “Parsing DeepSeek OCR in the Plainest Possible Language”), knowledge bases are generally very large. So efficiently finding the information an LLM relies on from a huge dataset remains especially important.

As the figure above shows, between the full dataset and the model window there is a large funnel. A few concepts here need a bit more explanation:

• Data filtering (circling the data): based on the user’s explicit requirements (e.g., “average spend of 25 yuan, rating of 4.5 or above, within 500 meters of me”), quickly screen out the data that doesn’t meet the requirements. This part relies heavily on the basic capabilities of a traditional relational database.
• Coarse ranking: after obtaining the dataset that meets the basic requirements, you need to rank the candidate dataset by relevance to the query request. The focus here is using the database to find the most relevant data; a very important reason to use a database is speed — it can basically guarantee sub-hundred-millisecond latency, which is critical for online queries.
• Fine ranking: the speed of the coarse-ranking stage sacrifices some accuracy (for the vector path, because it’s a dual-tower model, the relevance between the request and the candidate dataset isn’t strong enough; for the full-text path, which relies on keyword matching, semantic relevance is weak). The idea of coarse ranking is to quickly recall a batch of results, on the assumption that the desired results are most likely within that batch. In scenarios with high precision requirements, you often need some reranking models to cross-capture the deeper relationships between the query request and the data, producing a more precise ordering of the dataset. Take the Reranker model: it needs to perform real-time inference on every “query-document” pair, which significantly increases the system’s response time (from the millisecond level possibly up to hundreds of milliseconds or even seconds) and the compute cost per query, so the volume of data to be fine-ranked must be further reduced.

Data Filtering

Data filtering is very important, and many databases support it — for example, ES / Mongo / relational databases — though each is suited to somewhat different scenarios.

For NoSQL databases like ES, generally only single-table operations are supported, and the design is often denormalized: all the data fields of a business scenario are put into a single table, frequently forming one big wide table.

For some normalization-dependent “one-to-many” capabilities — for example, one person having multiple phone numbers — ES often uses a nested structure. But the denormalized big-wide-table + nested-structure approach often brings high update costs and data consistency problems. So it generally appears in edge systems with low consistency / real-time requirements; core systems still tend to choose relational databases.

Relational databases are based on the relational normal forms: a business scenario often has multiple tables, each with its own responsibility, linked by foreign keys and so on. An important capability of a relational optimizer is choosing the lowest-cost index across the tables and selecting the optimal Join order and algorithm among multiple tables, giving better overall scalability and flexibility.

Coarse Ranking

A single coarse-ranking method often leaves many scenarios poorly covered. For instance, recalling a batch of template-generated data — a single vector database can’t distinguish it well; or recalling multilingual data with synonymous / near-synonymous meanings — a keyword-only search engine can’t solve it well either. So in real engineering, multiple recall methods (dense vectors, sparse vectors, search engines, graphs, etc.) are often used in combination.

Coarse ranking based on multi-path retrieval brings several problems:

• Coarse-ranking precision: ranking ultimately orders data rows, and the ranking score is generally the sum of the scores from the various scoring columns of a data row — which can be seen as global ranking. “Siloed” multi-path retrieval generally takes TopK from each path and then fuses (multi-way merge), which can be seen as each path doing its own local ranking before fusing. Compared with global ranking, this loses precision, greatly affecting the effectiveness of the AI application.
• Ease of use: the AI application has to handle the multi-path retrieval and the fusion of results itself, making the whole process fairly cumbersome.
• Consistency: if multiple databases carry the multiple retrieval paths, the data consistency among the multiple systems requires extra handling.
• Efficiency & cost: multi-path retrieval requires maintaining multiple copies of the data, each retrieved independently, so the compute resources consumed also multiply.

Therefore, the advantages of hybrid search in the coarse-ranking stage are also very clear. In other words, if a database can support multi-modal data types, it can support vector, full-text, and other ranking methods over the same copy of data simultaneously, and fuse multi-path retrieval into a single scoring-and-ranking framework — bringing higher precision to coarse ranking while avoiding data consistency / cost / ease-of-use problems.

Fine Ranking

When multi-path retrieval involves vectors and reranking (fine filtering), because vectors depend on an embedding model and fine filtering depends on a reranking model — and current vector / relational databases don’t support invoking models inside the DB — the whole process becomes very complex, involving multiple round-trips between the application and the DB. As shown below:

So in the hybrid-search framework of future databases, a built-in AI Function capability is needed, supporting calls to embedding models, reranking models, and LLMs.

Whether it’s generating vector embeddings in the index-building flow triggered by data writes, or generating the query statement’s vector embedding at query time, the database should trigger it automatically. Once the Embedding AI Function is integrated, the user can be unaware of the existence of vectors — the vectors are hidden in the database’s own index tables.

The reasons a database needs to integrate Embedding are:

• The embedding model must be applied to both the query and the dataset to be retrieved. If the two use inconsistent model types / versions, the vector retrieval spaces become inconsistent and the recalled data will be problematic. Database systems often take a hands-off “not my problem” attitude and throw this to the user to guarantee. Although that’s understandable, if Embedding can be managed by the database — with the model information becoming metadata of the data and the metadata version managed by the database — data consistency can be better safeguarded internally.
• Models evolve rapidly. When experimenting with models or when a more capable new model is released, there’s often a need to switch models. Previous solutions required the user to delete all the old vectors, scan the Chunks stored in the database, generate new vectors, and insert them into the database — a very cumbersome process, and the alternation of old and new indexes also affected business usage. Now that Embedding is managed by the database, switching models is just a Rebuild Index DDL command; the database internally handles the vector-index switchover and the reclamation of the old index, transparent to the business — and most importantly, the whole process barely affects business access.

Beyond that, users can also consider explicitly invoking a Rerank model: after coarse ranking, simply call the Reranker AI Function.

That concludes the main text. This article didn’t cover too much of the low-level implementation principles or usage best practices of hybrid search; I’ll keep updating later.

---

And now, the advertisement time you’ve all been waiting for~

Commercial Break

0x00. OceanBase’s Performance in Hybrid Search

Since I said up front that the main text wouldn’t include anything related to database vendors, the capabilities and performance comparisons of database vendors in hybrid search had to be placed in this closing “advertisement time.”

The Coarse-Ranking Stage

• OceanBase supports multi-modal data types and, over the same copy of data, simultaneously supports vector, full-text, and other ranking methods, fusing multi-path retrieval into a single scoring-and-ranking framework — bringing higher precision to coarse ranking while avoiding data consistency / cost / ease-of-use problems.
• In terms of performance, OceanBase’s coarse-ranking capability has been steadily catching up to the best databases in the industry with comparable capabilities (here we won’t pass judgment on the various database products; see the figure below for performance data).

• In terms of related features:
  • OceanBase’s vector capability can currently match the various algorithms and features of Milvus, the most popular vector-database forerunner.
  • On the full-text side, OceanBase supports features for RAG scenarios such as BM25, Multi Match, sparse pruning algorithms, and RankFeature. For knowledge-base scenarios, it can basically serve as a drop-in replacement for another old forerunner, ElasticSearch.

The Fine-Ranking Stage

OceanBase has a built-in AI Function capability, supporting calls to embedding models, reranking models, and LLMs.

OceanBase has now woven the AI Function capability into the entire hybrid-search framework.

The data filtering, coarse ranking, and fine ranking of the whole hybrid-search process can be fully integrated into the OceanBase kernel. Users can insert text Chunks directly into OceanBase and, at query time, query directly using the raw natural-language string — achieving true Data In, Data Out, and making the whole AI multi-path retrieval simpler and more efficient.

0x01. OceanBase 4.4.1 CE Is Officially Released!

OceanBase 4.4.1 Community Edition was officially released on October 24. Its hybrid-search capabilities have been further enhanced — you’re welcome to download and try it.

The following is excerpted from the release notes of OceanBase V4.4.1 CE[1]:

• V4.4.1 upgrades vector retrieval, supporting Hybrid Search to improve recall.
• Adds AI function capability, supporting access to LLMs from SQL.
• Further improves vector retrieval performance through ARM-architecture adaptation, primary-key-less table optimization, IVF index optimization, and more.
• Adds views to show vector index memory usage and asynchronous task status, improving usability.
• ……

For documentation, see: OceanBase official docs “Vector Index Hybrid Search”[2].

Download link: OceanBase Software Download Center[3].

0x10. An AI-Native Database Product Is Coming Soon!!

To be continued. (This product will make its official debut in the “Open Source Night” segment of the OceanBase 2025 Annual Conference[4] — stay tuned.)

References

[1] Release notes of OceanBase V4.4.1 CE: https://www.oceanbase.com/product/oceanbase-database-community-rn/releaseNote#V4.4.1

[2] OceanBase official docs “Vector Index Hybrid Search”: https://www.oceanbase.com/docs/common-oceanbase-database-cn-1000000004020382#3-title-%E6%99%AE%E9%80%9A%E6%A0%87%E9%87%8F%E6%A3%80%E7%B4%A2

[3] OceanBase Software Download Center: https://www.oceanbase.com/softwarecenter

[4] OceanBase 2025 Annual Conference: https://www.oceanbase.com/conference2025