OceanBase Vector Search at Lalamove: Exploration and Practice

Author: Chen Quan, Senior Big Data Engineer in Lalamove’s Big Data Technology and Product Department

Founded in 2013 and grown out of the Guangdong–Hong Kong–Macao Greater Bay Area, Lalamove is an internet logistics marketplace engaged in intra-city/inter-city freight, enterprise logistics services, moving, less-than-truckload (LTL) freight, errand running, cold-chain transport, vehicle sales and rental, and aftermarket services. As of 2024, Lalamove had 16.7 million monthly active users and 1.68 million monthly active drivers worldwide, with operations covering 11 markets and 400+ cities globally and 6 data centers around the world.

1. The Challenges of LLM Application Scenarios

Building on its deep accumulation of AI implementation in logistics, Lalamove has explored and deployed LLM applications across 14+ business units or departments and 50+ real business scenarios. In the process of adopting LLMs, it faced challenges such as their lack of vertical-domain knowledge, insufficient timeliness, and data-security risks. To address these problems, it adopted a fairly common industry solution—Retrieval-Augmented Generation (RAG)—which, by bringing in external data, turns the LLM’s answers from a “closed-book” exam into an “open-book” one. By integrating domain-specific knowledge, private data, and real-time data, RAG significantly reduces the uncertainty of generated answers and strengthens data security, thereby effectively solving the LLM’s inherent problems and improving the accuracy and usefulness of its answers.

The core of RAG lies in combining the powerful capabilities of language models with those of a vector database. When implementing a RAG solution, enterprises typically need to pair it with a vector database. Vector databases have unique advantages in handling multimodal data and semantic search, specifically in the following respects:

• Storing unstructured data: Vector databases can effectively store and manage multimodal data such as audio, video, images, and text. Such data is usually large in scale, high in information density, and costly to process.
• Vectorized representation: Neural networks extract data features and convert them into coordinate points in a high-dimensional space. Vectorized representation gives data semantic expressiveness, making it suitable for similarity search.
• Retrieving unstructured data: By computing the distance between vectors (such as inner product or Euclidean distance), the most similar vectors are identified. The retrieval process involves traversing a proximity graph and requires a large number of floating-point operations to achieve efficient similarity matching.

2. Considerations in Selecting a Vector Database

(1) The Existing Architecture and Its Pain Points

The existing architecture consists of an infrastructure layer (two machine types, CPU and GPU), a storage layer (vector database, ES, etc.), a retrieval layer (mainly graph indexes, with multiple retrieval types), an access layer, and an entry layer. It comprises 5 clusters across China and abroad, with per-cluster memory of 380+ GB and a maximum single-table data size of 20 million rows.

Pain Point 1: Dynamic Schema

With rapid business growth, frequent additions and deletions of fields have become the norm. The current solution is to create a new table, import the existing data, and finally rebuild the index—a relatively cumbersome process. For some tables with large data volumes, rebuilding the index can take more than ten hours. In addition, the index-rebuilding process is extremely demanding on CPU and memory, easily causing online business jitter.

Pain Point 2: Hybrid Search

Vector search has significant advantages in similar-semantic retrieval and multimodal data understanding, while full-text search excels at exact matching and at retrieving short texts and low-frequency words. In enterprise applications, relying on a single retrieval method alone struggles to meet the business’s high requirements for retrieval precision. To compensate for the shortcomings of full-text search, Elasticsearch was introduced as the full-text search engine, which in turn increased the complexity of the overall architecture and raised the system’s maintenance difficulty. For users, complex reranking logic had to be implemented at the application layer, and the resulting similarity scores were hard to unify, increasing the cost of use. As a result, the business hoped to introduce a one-stop hybrid-index capability.

Pain Point 3: High Operational Difficulty

• Weak stability: The vector database itself was unstable and bug-prone; the lack of expert experience made troubleshooting difficult; and limited monitoring metrics made problems hard to pinpoint.
• Insufficient scalability: The horizontal scaling of nodes was poor, data migration relied on manual work, and the management and operation of data shards was complex.
• Weak access control: The existing authentication mechanism was not robust enough, easily leading to data leakage and security issues; access control had to be implemented in-house, increasing development and operational complexity.
• Poor community activity: Although the project was still maintained, updates were infrequent, community contributions and developer participation were limited, and the community’s features and ecosystem evolved slowly, unable to meet the business’s future needs.

(2) Selection Criteria and Process

Based on the pain points above, we re-evaluated our vector database selection at the end of 2024. The selection criteria were considered mainly from two angles—business requirements and operational requirements—as shown in the figure below.

During the selection process, we shortlisted 10 vector databases and, through a detailed multi-dimensional comparison, conducted a first round of filtering based on our business and operational pain points. First, since our company uses a multi-cloud architecture, we wanted a database that could be deployed across clouds, which ruled out cloud-vendor databases. Second, given the business’s higher requirements for vector dimensions, we ruled out PostgreSQL. In addition, considering stability and access control, we ruled out Weaviate.

After the preliminary filtering, Milvus, Elasticsearch, and OceanBase made it onto the shortlist. In the second round of filtering, we focused on stability and operational cost:

• Milvus: Real-time risk-control scenarios demand extremely high stability from a vector database. Because Milvus’s overall architecture is fairly complex, ensuring its stability requires more operational investment. In addition, Zilliz’s cloud version and Lalamove’s online services are deployed across regions, which poses certain stability risks, so Milvus was ruled out for now.
• OceanBase: After setting up an OceanBase Community Edition environment, we conducted a comprehensive test of its vector capabilities and, together with the business team, performed stress testing and comparison against real online scenarios. The results showed that OceanBase could meet the business’s needs in both functionality and performance. Moreover, OceanBase has been refined over many years by major enterprises, and its stability has been proven. At the same time, the OceanBase community is highly active, regularly updating its vector capabilities and performance and providing technical support.
• Elasticsearch: Elasticsearch performs quite well in full-text and hybrid search, but given the actual situation of our internal team, we ultimately chose OceanBase over Elasticsearch.

After completing the selection, the key decision we faced was whether to self-host or go to the cloud. We first compared these two options in detail. Then, considering that a large number of databases within the company are trending toward the cloud—where elastic scaling works well and SLA guarantees are more reliable—and that at this stage we are more focused on business onboarding and do not want to invest too much manpower in operations, we ultimately chose to build the vector database foundation on the cloud.

3. Production Scenarios for the Vector Database

(1) Financial-Loss Code Detection

Financial-loss code detection is an important application scenario for OceanBase vector search at Lalamove. R&D quality issues or potential vulnerabilities in code can cause the company to suffer serious financial losses. In the past, identifying financial-loss code relied mainly on manual review, which was inefficient and hard to apply comprehensively across online services, leaving financial-loss risks impossible to fully avoid. To solve this problem, we combined LLM capabilities with OceanBase vector search to develop an automated code-risk detection system. By vectorizing historical case data and retrieving similar code, the system uses an LLM to analyze and judge financial-loss risk, thereby improving the efficiency and accuracy of code review and controlling risk during development.

The specific flow is as follows: First, based on historical financial-loss code scenarios and real case data, an LLM performs classification and labeling to produce a dataset, which—after a second round of manual confirmation—is loaded into the vector database. When a developer submits code to be built, a code-detection process is triggered: the submitted code is compared against the financial-loss code stored in the vector database via vector similarity search, and the retrieval results and related data are provided to the LLM to judge the financial-loss risk. If the code is judged to be risky, the build process is interrupted to prevent the code from being released to the online platform. Implementing this project improved the efficiency and accuracy of financial-loss code detection and effectively helped the company avoid potential financial-loss risks.

(2) Data Warehouse AI Q&A Assistant

The data warehouse AI Q&A project is another important production scenario for OceanBase vector search at Lalamove, and it is also a very typical application scenario. Lalamove’s big data warehouse is enormous, with hundreds of thousands of Hive tables, and a large number of users need to query data every day. However, users often lack sufficient business background knowledge and struggle to quickly find the data they need, so they can only turn to data-warehouse developers for help, placing a huge workload on those developers. To solve this problem, we applied vector search to a data warehouse AI Q&A assistant, improving data-query efficiency and easing the workload of data-warehouse developers.

The specific flow is as follows: First, the schema information of databases and tables, the chat Q&A records, and internally maintained documents are processed—for example, table information is turned into field-mapping relationships, and chat Q&A records are converted into QA pairs. Then, an Embedding model converts this data into vectors and stores them in the OceanBase vector database. When a user asks a question, the system first performs intent recognition to determine whether the user wants to find data, ask about metric definitions, or get general knowledge Q&A. Next, it understands the user’s question, breaks a complex question into multiple sub-questions, performs entity recognition, and—when necessary—uses multi-turn dialogue to clarify the user’s intent. It then performs knowledge recall; because this scenario demands high query precision, multiple retrieval methods are used, such as vector search, scalar search, and full-text keyword search. The recalled knowledge data is fed to a reranking model for reordering, and the most relevant answer is passed to the LLM for summarization and generation, ultimately providing the user with accurate data-query results. Implementing this project lowered the barrier for users to find data, eased the hidden communication burden, significantly improved data-warehouse query efficiency, reduced labor costs, and enhanced the user experience.

4. Future Plans

As OceanBase runs stably in Lalamove’s online business, there will be deeper and richer application plans in the future.

• Business migration: This involves supporting one-stop fusion-retrieval capabilities, adapting business systems, and migrating data.
• Performance and cost: As the number of users grows, performance and cost become key concerns. We will consider introducing technologies such as the scalar-quantization index HNSW_SQ or the disk-based index IVF, while also supporting table-level TTL and hot-cold data tiering.
• Internal system integration: We will integrate OceanBase into internal systems, such as the big data platform, the monitoring and alerting system, and the DMS database management system, to provide a smoother user experience.
• Exploring more scenarios: We are exploring more OceanBase application scenarios, such as OLAP and OBKV, in the hope of solving online-storage pain points.