Safeguarding Performance in High-Concurrency Scenarios with OceanBase

This article is excerpted from the e-book “Case Studies of OceanBase Community Edition in Pan-Internet Scenarios”. Click the link to get the full version.

As data volumes keep surging, more and more business systems face the pressure of high-concurrency, high-performance access, and enterprises’ demand for performance assurance grows ever stronger. As the foundational component of business systems, a database that delivers high concurrency and high performance is key to supporting business systems and meeting customer needs. By explaining the design of database components, this article unpacks the key factors in delivering solid system performance.

Safeguarding System Performance Through Sound Kernel Component Design

As the database that has safeguarded Tmall’s “Double 11” promotion for 12 consecutive years and powers Alipay’s core systems, OceanBase has been battle-tested time and again in high-concurrency scenarios. In March 2025, OceanBase’s performance was upgraded once more. At the same hardware scale (a 16-core configuration), in actual testing with the standard Sysbench test set (see Figure 1), its standalone edition comprehensively outperformed MySQL 8.0 on overall performance — covering queries, batch reads, writes, mixed read-write, inserts, and updates. In high-concurrency write scenarios in particular, throughput improved markedly, peaking at a 214.99% gain, meeting business needs under heavy load.

Figure 1: Sysbench performance benchmark comparison (OceanBase standalone edition vs. MySQL 8.0)

OceanBase’s high performance is inseparable from the design and implementation of its various kernel components, mainly covering efficient plan management, concurrency control, multi-dimensional cache optimization, hot-row optimization, log aggregation optimization, and compilation optimization. Below we analyze each module to lift the veil on this high performance.

1. Efficient Plan Management

1. Plan cache.

Optimizing a SQL statement is a fairly time-consuming process, and as a statement grows more complex, optimization takes longer. To avoid repeatedly running the optimization process, the generated execution plan is added to the plan cache so it can be reused the next time the SQL runs. Each tenant has an independent plan cache on each server, caching the SQL plans processed on that server.

In application systems, the same SQL may run with different parameters each time. To reduce the plan cache’s size, the system first parameterizes the user’s SQL to obtain a SQL string independent of the specific parameters, and uses that string as the plan-cache key. The plan cache is a classic key-value structure: the key is the parameterized SQL string, and the value is the execution plan for that SQL. In OceanBase’s plan cache, a SQL execution plan can be of three types: local, remote, or distributed. Depending on which data a given SQL needs to access, all three plan types may coexist in the cache for the same SQL. For a particular execution plan of a particular SQL, by default OceanBase keeps only one plan — generated the first time the SQL runs. But in some cases the parameter values of the same SQL can affect plan choice, so the plan cache may, as needed, keep different execution plans for different parameter values, ensuring the most appropriate plan is used on each run.

2. Fast parameterization.

With a plan cache in place, the next question is how to fetch the cached data quickly. Traditional databases generally parameterize at the syntax-tree level when parameterizing, then use the parameterized syntax tree as the key to fetch a plan from the plan cache. OceanBase, by contrast, uses lexical analysis: it parameterizes the text string directly and uses that as the plan-cache key — hence the name “fast parameterization.” The detailed flow is shown in Figure 2.

Figure 2: The process of fetching an execution plan based on fast parameterization

Plan-cache optimization eliminates the repeated work of plan generation. Fast parameterization skips the syntax-analysis step, and at the same time, hashing and MemCmp on a text string are more efficient than hashing and comparing a parameterized syntax tree — improving the efficiency of fetching plans from the cache. Both optimizations fundamentally reduce CPU overhead, thereby boosting system throughput.

2. Concurrency Control

1. Data management.

The OceanBase storage engine uses an LSM-tree architecture, splitting data into static and dynamic data. Dynamic data is held in the Memtable and periodically dumped to disk. The Memtable uses a dual B+tree-and-hash index structure to store data, where the B+tree serves range queries and the hash serves single-row lookups. The leaf nodes of the B+tree hold metadata for the row data, with three key fields: primary key, lock, and a linked-list pointer, as shown in Figure 3.

Figure 3: Memtable in-memory data structure (multiple modifications on a row)

• The lock information indicates whether a transaction holds the row lock; a transaction must acquire the row lock before modifying data.
• The linked-list information points to multiple versions of the data. Each version stores only the delta — for example, a modification that changes a single field records only that field’s change.
• For rows that have not yet been committed but have already been dumped into static data, the static data is specially marked, used to determine whether a row lock exists.

2. Concurrency control.

OceanBase implements concurrency control based on MVCC and per-row mutex locks, with the following main characteristics.

• A snapshot version is a timestamp; transaction visibility can be determined simply by comparing timestamps. As a result, unlike other database systems, OceanBase does not need to maintain a global transaction manager, so there is no global-transaction-manager access bottleneck under high concurrency.
• Lock information is stored in the metadata of OceanBase rows, so no separate lock manager is needed either.
• Reads take no row lock, while writes take a per-row mutex lock — so reads and writes don’t block each other, improving throughput under high concurrency.

Figure 4: Memtable in-memory data structure (read-write request logic)

3. Multi-Dimensional Caching

As mentioned above, the OceanBase storage engine uses an LSM-tree architecture, whose overall storage-engine architecture is shown in Figure 5.

Figure 5: OceanBase storage engine architecture

Generally, a query in an LSM architecture needs to merge static and dynamic data, then project the result before returning it to the client. With multi-level SSTables, this inevitably lengthens the execution path. To improve query efficiency, OceanBase designed a multi-level caching strategy, comprising the following five caches.

(1) Block Cache: Similar to Oracle’s Buffer Cache, it caches specific data blocks. In fact, the Block Cache caches decompressed micro-blocks, which are variable in size.

(2) Block Index Cache: Caches the index of micro-blocks, akin to a B-tree’s intermediate layer. It differs somewhat from the Block Cache in data structure; since the intermediate layer is usually small, the Block Index Cache’s hit rate is typically high.

(3) BloomFilter Cache: A BloomFilter is a structure that helps accelerate the filtering of empty queries and improve insert performance. OceanBase’s BloomFilter is built on macro-blocks, constructed automatically on demand: when the number of empty queries on a macro-block exceeds a certain threshold, a BloomFilter is built automatically and placed into the cache.

(4) Row Cache: The Row Cache caches specific data rows. During Get/MultiGet queries, the matching rows may be placed into the Row Cache, which can dramatically improve performance for hot-row lookups.

(5) Fuse Row Cache: The difference from the Row Cache is that the Fuse Row Cache caches the value after merging the current system’s dynamic and static data. For high-frequency data access, the fuse row cache can be accessed directly.

4. Hot-Row Optimization

With the growth of online transactions and e-commerce, the hot-concurrency pressure on business systems has gradually become a challenge. A flurry of balance updates on a hot account in a short span, or a flash sale of a popular product during a promotion, are direct examples of this scenario. The essence of hot updates is highly concurrent modification of certain field values (balance, inventory, and so on) on the same row in a short span. The bottleneck lies mainly in the fact that, to maintain transactional consistency, a relational database must put each row update through a “lock → update → write-log commit → release lock” process — and this process is essentially serial. Therefore, the key to improving hot-row update capability is to shorten the lock-holding time as much as possible. To ease the hot-row update problem, OceanBase proposed the Early Lock Release (ELR) optimization, whose principle is shown in Figure 6.

Figure 6: The principle of early lock release

1. Before optimization.

After the user issues a COMMIT, the database (DB) side triggers the log persistence flow. This process includes the following four steps.

(1) Serialize the in-memory data and submit it to the local Log buffer.

(2) The database sends the log data to all standby machines.

(3) Only after a majority of the standby machines have synchronized the log successfully does the database consider the log persisted.

(4) Finally, unlock the transaction and return a commit-success response to the client.

In this flow, a transaction’s lock-holding time includes: data writing, log serialization, the network communication to sync the standby machines, and the time to flush the log to disk. For a three-region, five-center deployment or a setup with poor disk performance, the lock-holding time is long and tends to have a significant performance impact on hot rows.

2. After optimization.

In the optimized design, the overall commit flow stays largely the same, but the timing of unlocking is adjusted. In the new flow, as soon as log serialization completes and is submitted to the Log buffer, the unlock operation is triggered immediately — no longer waiting for a majority of standby machines to finish flushing the log to disk. This effectively shortens the transaction’s lock-holding time. Once the transaction is unlocked, subsequent transactions are allowed to operate on the same row, achieving concurrent updates of the same row by multiple transactions and thereby boosting system throughput.

This optimization takes effect on the premise that the OceanBase kernel guarantees the following key properties.

• If an early-unlocked transaction ultimately rolls back, every subsequent transaction that read this transaction’s information must also roll back — what we call cascading rollback.
• Before an early-unlocked transaction has responded to the client, any subsequent transaction that read this transaction’s information cannot respond to the client early either.
• Only single-machine transactions support the ELR optimization, to reduce the probability of cascading rollback.

3. Optimization results.

Based on the optimization above, performance in hot-row scenarios can be computed with the formula: TPS = 1 / {the lock-holding time of a hot row within one transaction}, where lock-holding time refers to the interval from acquiring the lock to the transaction’s commit completing.

In a three-region, five-center scenario, because the SQL’s overall execution time is 30 ms, the transaction’s COMMIT response time (RT) is about 30 ms. With this optimization, performance can essentially match that of a same-city deployment.

Based on Sysbench single-row updates, stress-tested on a 16c/64g Alibaba Cloud ECS environment, the hot-row optimization results are shown in Table 1.

Table 1: Hot-row optimization results

Client concurrency	Before (TPS)	After (TPS)	Performance gain
1	2270	2245	0%
5	2489	8247	231%
15	2470	8458	242%
25	2527	10588	319%
50	2502	10641	325%

5. Log Aggregation Optimization

In database systems, flushing the WAL to disk is a common action. A large number of high-frequency, small-volume log writes hitting the disk generate heavy I/O, which in turn hurts business-request RT and system throughput. To solve or mitigate this, a common approach in the database field is group commit — in short, aggregating multiple log records and performing a single I/O for the batch. Under high-concurrency writes, this reduces the IOPS for persisting logs and the related CPU overhead, ultimately improving system throughput.

OceanBase 4.x redesigned the 3.x log-aggregation scheme, reducing the number of buffer copies for a single log record and further enhancing log-aggregation capability at the log-stream level, with the architecture shown in Figure 7.

Figure 7: Logical management structure of the leader and follower replicas in the OceanBase log engine

In Figure 7, the transaction module commits logs to the GroupBuffer for aggregation. Through a certain aggregation policy, the generated logs are submitted to a ringbuffer-implemented sliding window — the FixedSlidingWindow. This structure is responsible for syncing the aggregated logs to the standby machines, triggering the transaction module’s log callbacks, and advancing information such as the local maximum continuous committed lsn.

To meet log-sync efficiency across different concurrent-write scenarios, OceanBase 4.x supports two log-aggregation policies: periodic and feedback-based.

• Periodic aggregation: A dedicated background thread triggers log aggregation every 1 ms. This policy is friendly to sustained high-concurrency writes.
• Feedback-based aggregation: For low-concurrency write scenarios, periodic aggregation introduces a 1 ms delay to transaction logs, with a significant business impact. To address this, OceanBase 4.x supports an I/O-thread feedback-based aggregation policy: after the log flush thread completes its current I/O action, it proactively triggers a log aggregation before the next I/O action.

These two policies are mutually exclusive; the system adaptively chooses between them based on the current write traffic, balancing the latency of a single commit request under low concurrency against the overall system throughput under high concurrency. Based on a 32c × 3 three-replica environment, with the client simulating a single committed log size of 512B each time, we compared the maximum log-persistence throughput of three systems: OceanBase’s log module Palf, Etcd, and Braft. The results confirmed that at the same concurrency, the throughput of OceanBase’s Palf log module is far higher than the other two systems (see Figure 8).

Figure 8: Comparison of the maximum log-persistence capability of OceanBase PALF, Etcd, and Braft

6. Compilation Optimization

In 2024, OceanBase carried out extensive performance optimizations for domestic and Intel chips. These optimizations are now applied in real business scenarios and have achieved significant performance gains.

PGO (Profile Guided Optimization), also known as FDO (Feedback Directed Optimization), is a class of feedback-based compilation optimization. Its core principle is to collect profile data based on real production traffic, process it to obtain information such as hot functions and branch-execution frequencies, and finally feed the result file into the compiler and linker to recompile and produce an optimized binary. This optimization improves instruction-execution efficiency on multiple fronts (icache, itlb, page faults, and so on).

LTO (Link Time Optimization) is another compiler-optimization technique. It lets the compiler perform global, cross-compilation-unit optimization at the link stage, enabling whole-program inlining and virtual-method optimization, thereby improving the CPU icache hit rate and the system’s overall throughput.

Beyond compilation optimization, OceanBase used optimized assembly instructions to resolve the performance bottleneck of glibc’s native load128 atomic operation on ARM CPUs; based on the differences across CPU architectures, it adaptively sets the CACHE ALIGN size, achieving adaptive alignment of hot variables on both x86 and ARM CPUs, effectively reducing false sharing. This optimization markedly improved OceanBase’s performance on domestic chip environments, lowering implementation costs in the rollout of domestic-innovation (Xinchuang) initiatives across industries.

Summary

This article discussed safeguarding database performance in high-concurrency scenarios, highlighting the importance of a high-performance database system in supporting the business under the pressure of massive data and high-concurrency access. Taking OceanBase as an example, it explained — from the SQL engine, transaction engine, storage engine, compilation optimization, and other angles — the key designs of a high-performance distributed database in high-concurrency scenarios. Thanks to these designs and optimizations, OceanBase can support the high-concurrency, high-performance needs of diverse business scenarios. We hope this article offers valuable reference for the performance optimization of high-concurrency systems.

Finally, I’d like to recommend the WeChat official account “Lao Ji’s Tech Talk” run by Lao Ji, the head of OceanBase Open Source. It continuously publishes all kinds of technical content related to #Databases, #AI, and #Tech Architecture. If you’re interested, feel free to follow!

“Lao Ji’s Tech Talk” hopes not only to keep bringing you valuable technical content, but also to contribute to the open source community together with everyone. If you appreciate the OceanBase open source community, light up a little star ✨! Every Star you give is fuel for our efforts.