Dify + OceanBase: Multi-Scenario Practice for Landing AI Workloads
Over the past year, OceanBase’s AI capabilities have advanced rapidly. From June 2024, when community users built a RAG assistant on OceanBase’s basic vector capabilities, to October, when the first official vector release, OceanBase 4.3.3, became production-ready. We then quickly adapted to mainstream ecosystem products such as LlamaIndex, DB-GPT, and Dify, while also open-sourcing a RAG Demo and running rich AI Workshops at offline events. Then in November, we released a Demo for multi-modal fusion queries and opened up an online AI Demo experience for community users, including: an image-to-image search application based on vector retrieval, an AI knowledge base based on vector retrieval, and an AI assistant based on multi-modal fusion queries.
Through 2025, we successively released OceanBase MCP, Power RAG, and the OceanBase AI Appliance, enhanced the AI capabilities of tools such as OCP, ODC, and OMS, and announced that we are fully embracing AI.
Choosing an AI Application Development Platform
To realize and validate the feasibility of feature ideas through fast iteration — beyond the team’s deep involvement — choosing the right tool can make the work far more efficient. When selecting an AI application development platform, we settled on three key points:
- Visual orchestration. Quickly validate the feasibility of ideas through a no-code, visual interface.
- Programmable nodes. Since many platforms provide nodes that can’t fully meet our needs, we wanted a platform with open nodes that supports custom programming.
- Reusable flows. For example, in a ChatBI scenario, the LLM often fails to understand a question correctly because it lacks business-specific knowledge. In that case, we can reuse an already-built RAG flow to retrieve the relevant business knowledge for the user’s question, so the LLM can understand the question correctly and generate the correct SQL.
Based on these three selection criteria, we felt Dify was the best fit. Dify (Define + Modify) is a forward-looking open-source LLM application development platform that combines Backend as a Service with the LLMOps philosophy to give developers and enterprises production-grade capabilities for building generative AI applications.
First, Dify’s workflow orchestration is very powerful. Compared with other platforms, it offers complete logic control — including conditional branches, iteration, and loops — and is highly usable.
Second, Dify’s programmable nodes fully open up the inputs, outputs, and intermediate process definitions, with full Python function capabilities. During our early research, we found that on other platforms, using a programming node’s capabilities required restarting the service — something we found completely unacceptable.
Finally, on the reusability front, Dify lets you add existing flows directly from the tools panel in the bottom-left corner during orchestration, which is very convenient.
In addition, Dify provides a plugin feature that lets us download and directly use third-party flows and third-party plugins from the community.
Challenges in Using Dify and How We Solved Them
After confirming that Dify met our needs, we began rolling it out and using it, and inevitably ran into some challenges.
Challenge 1: High maintenance cost of multiple storage components
The Business Challenge
The figure below is the architecture diagram from Dify’s official website. As you can see, it has three databases: by default the architecture uses PostgreSQL as the metadata database, Redis as the cache, and Weaviate as the vector database. Such a multi-component architecture increases maintenance complexity and the learning curve.
The Solution
OceanBase has a complete tool ecosystem and supports scalar, KV, and vector storage and queries all at once. Given the problem above, our idea was: can we replace all of them with OceanBase tools? Use OBKV-Redis to replace Redis, and OceanBase’s vector capabilities to directly replace Weaviate. The biggest challenge was that OceanBase isn’t compatible with PostgreSQL and has no plans to be. So we modified a Dify branch to be compatible with MySQL — and because OceanBase is MySQL-compatible, we could use OceanBase as both the metadata database and the vector database (project: the oceanbase/dify-on-mysql branch; scan the QR code below to go there).
After switching the original three databases to OceanBase, we not only unified the tech stack and made operations more efficient, but also gained hybrid search capabilities.
Challenge 2: Weak high availability and rapid scaling
The Business Challenge
During deployment, since the Redis and PostgreSQL that Dify uses by default don’t support native distribution, and Weaviate officially offers only a K8s private distributed deployment option, this architecture carries a single-point-of-failure risk when facing high concurrency in an on-premises private deployment. The performance limits of a single machine become a bottleneck for business growth.
The Solution
OceanBase is a natively distributed cluster that supports high-availability deployment across two regions with three centers, or three regions with five centers; it also supports fast, transparent horizontal scaling that is transparent to the business. Therefore, after replacing all three database components with OceanBase, we avoided the frequent crashes under high concurrency and achieved stable business operation. We also eliminated the inability to migrate smoothly and automatically after a failure, achieving transparent horizontal scaling, and improved backup and recovery capabilities.
Challenge 3: No business isolation; high maintenance cost for multiple workloads
The Business Challenge
Dify’s license shows that the open-source version does not allow the multi-tenancy feature, so workloads aren’t isolated and may interfere with one another. You can achieve isolation by deploying multiple clusters, but the maintenance cost of multiple clusters is high.
The Solution
We use OceanBase’s multi-tenant resource isolation to deploy multiple business clusters, achieving business-level resource isolation. One tenant corresponds to one workload, and a single cluster can support multiple workloads at the same time.
In summary, after replacing Dify’s original three databases with OceanBase, we achieved: storage system OceanBase = PostgreSQL + vector store + Redis, with the following benefits:
- Improved stability. 24/7 enterprise-grade high availability with cross-data-center disaster recovery.
- Improved scalability. Supports unlimited data growth, from standalone to distributed, effectively avoiding memory OOM.
- Stronger multi-tenancy. Multi-tenancy works even on the community edition.
- Cost savings. One system replaces multiple storage systems; through resource pooling, we save 30% of resources.
- A unified tech stack, lowering both operations cost and the learning curve.
Landing Dify + OceanBase and Its Application Scenarios
The Dify + OceanBase solution has already been applied to several scenarios. Here are a couple of simple examples for your reference.
AI Agent: A Conversational Scale-Based Psychological Assessment Tool for People in Drug Rehabilitation
At the 2025 OceanBase AI Hackathon, an officer from a drug rehabilitation center with no technical background submitted his work — a conversational, scale-based psychological assessment tool for people in rehabilitation. The motivation: judicial and administrative agencies are responsible for helping people in rehabilitation through compulsory isolated rehabilitation and recovery. Compulsory isolated rehabilitation is a rehabilitation measure with distinctive Chinese characteristics; centered on the rehabilitation work of educating people in rehabilitation, it applies psychological principles and methods to carry out psychological detoxification and recovery — an important responsibility. This process involves psychological assessment, that is, quantifying the cognition, behavior, and emotions of people in rehabilitation according to rules and procedures, one of the methods often used in psychological counseling at rehabilitation facilities. Assessment is usually done in two ways: one-on-one interviews and scale tests. One-on-one interviews can gather a larger amount of information and yield better assessment results, but they are time-consuming and labor-intensive (around an hour); scale tests can be completed quickly and are convenient and efficient, but they gather more limited information and yield poorer assessment results.
The officer tried to use LLM capabilities to automate this — judging and interpreting a person’s situation from their answers — so he chose the Tongyi Qianwen LLM with OceanBase Cloud as the underlying database, plus Dify, deployed in Docker.
From the flowchart, you can see the process isn’t complicated. First, the prepared scale is uploaded to the database, and the conversation begins. The LLM then scores each answer, decides what information to gather in the next stage, and automatically selects a more appropriate question, guiding the person to answer the scale questions step by step. This loop continues, ultimately collecting the relevant information about the person.
A Southeast Asian Courier Company: Intelligent Customer Service
Against the backdrop of a massive order volume, a Southeast Asian courier company’s customer service team faces an enormous daily volume of inquiries. To improve answering efficiency and reduce the pressure on agents, the company used LLM capabilities to build intelligent customer service. Initially, they used Dify’s RAG capabilities with multi-way recall enabled, which worked well for Chinese; but because Thai is a lesser-used language and the default tokenizer doesn’t support it, the results for Thai were poor. After introducing OceanBase, they could quickly implement a tokenizer for the target language via a plugin, solving the problem. In addition, this avoided the frequent crashes they had previously seen under high concurrency and improved backup and recovery. On the vector side, introducing HNSW_SQ enabled a single partition to support the workload, with no need for sharding.
Finally, I’d like to recommend the WeChat official account “Lao Ji’s Tech Talk,” run by Lao Ji, the OceanBase open source lead. It continuously publishes all kinds of technical content related to #databases, #AI, and #tech architecture. Friends who are interested are welcome to follow!
“Lao Ji’s Tech Talk” hopes not only to keep bringing you valuable technical insights, but also to contribute to the open source community together with you. If you appreciate the OceanBase open source community, please light up a little star ✨! Every Star you give is motivation for our efforts.