Longda's Interesting World

Distributed Databases & AI Agent Engineering Practice

OceanBase-4.2.x Experience Notes (1)

Hands-on notes based on OceanBase 4.2.1 LTS (part 1): a complete installation guide covering OBD three-replica cluster pre-checks, disk and user planning, OBServer/OBProxy offline deployment, and tenant verification.

Summary

On November 16, 2023, OceanBase held its annual conference and officially announced the 4.2.1 LTS version. Starting from 4.2.1, 4.2.x will be a Long Term Support release, with subsequent ongoing releases such as 4.2.2, 4.2.3, 4.2.4, 4.2.5, and so on—very friendly to users. Future system upgrades can be performed as rolling upgrades, with no need to worry about reworking and upgrading the application layer; data can also be upgraded locally, even smoothly upgraded without service interruption. More importantly, future users will all migrate to 4.2.x, and as the user base grows, stability will keep being strengthened.

Once the kernel version is basically stable, and the tooling ecosystem—such as obd, ocp, and oms—is also basically stable in terms of framework and process, it is worth running a comprehensive round of testing to uncover some usability issues and help future users onboard more easily.

Test Outline

  1. OBD installation and deployment
  2. OBD inspection optimization
  3. tpcc testing
  4. sysbench testing
  5. tpch testing
  6. obdiag log collection
  7. Using OBD, scale up a single node into cluster mode
  8. Using OBD, vertically scale up the cluster (add machines within a single replica)
  9. Using OBD, horizontally scale out the cluster (add replicas)
  10. Using OBD, horizontally scale in the cluster (remove replicas)
  11. Using OBD, vertically scale down the cluster (remove machines within a single replica)
  12. failover test, when running sysbench, restart one server.
  13. ocp installation and deployment
  14. ocp takes over an OBD-deployed cluster
  15. ocp installs an OB cluster (single-node primary-standby mode)
  16. Using OBD, scale up a single node into cluster mode
  17. Using OBD, vertically scale up the primary cluster (add machines within a single replica)
  18. Using OBD, horizontally scale out the primary cluster (add replicas)
  19. Using OBD, horizontally scale in the primary cluster (remove replicas)
  20. Using OBD, vertically scale down the primary cluster (remove machines within a single replica)
  21. Using NFS, back up the primary cluster and restore the standby cluster
  22. Using NFS, back up the standby cluster; after a switchover, restore the original primary cluster
  23. oms installation and deployment
  24. tpch data synchronization to MySQL
  25. binlog service installation
  26. Under a sysbench test, configure the binlog service to synchronize to MySQL

Software Versions

Kernel version: 4.2.1_CE_BP1_HF1
OBD version: 2.4.0
OCP version: 4.2.1
OMS version: 4.2.0
obdiag diagnostic tool: 1.4.0
OBLogProxy (binlog service): v2.0.0

Hardware Configuration

The machine configuration running observer (since this is only a functional test, no higher-spec hardware was chosen)
aliyun ecs:
4c/32g amd instance
3 disks

Deployment Topology

This example uses the classic three-replica deployment topology with 4 machines:

  • 1 machine deploys OBProxy – it is recommended to deploy the client application together with OBProxy to reduce first-hop network latency.
  • 1-1-1 deploys a three-replica OceanBase cluster, where each zone represents one replica; in this example, a zone contains only one machine. The three zones are often deployed in production as a “three data centers across two regions” topology, with three data centers and three replicas—one replica per data center—where two of the data centers are relatively close to each other.

​

Software and Hardware Requirements

Item Description
OS Red Hat Enterprise Linux Server 7.x (kernel Linux 3.10.0 or above)
CentOS Linux 7.x (kernel Linux 3.10.0 or above)
Anolis OS 8.x (kernel Linux 3.10.0 or above)
CPU Enterprise users: minimum 16 cores, 32 cores or more recommended
Personal testing: minimum 2 cores, 8 cores or more recommended
Memory Enterprise applications: minimum 64G, 256G or more recommended
Personal testing: minimum 8G, 64G or more recommended
Disk type SSD recommended
Disk space 4x the memory size or more
File system ext4 or xfs; when the data volume exceeds 16TB, use xfs
Network card Gigabit interconnect or above

Set Up Passwordless SSH Login

Before installation, you need to configure the environment on each machine. These settings must be performed as the superuser. It is recommended to set up trusted login (i.e., passwordless login) from the control machine to the OBServer and OBProxy machines. For how to set up passwordless SSH login, see https://open.oceanbase.com/docs/community/oceanbase-database/V3.1.0/optional-set-password-free-ssh-logon
​

Here are two recommended scripts that make it easy to run commands and copy files in bulk across the cluster.
Bulk file copy—you can replace the host list with your own actual list of machines:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#/usr/bin/bash


hosts=(
    "ob001"
    "ob002"
    "ob003"
    "obdriver"
    )
    for host in "${hosts[@]}"
    do

        echo "begin to scp " $@ " on " $host
        scp -r $1 $host:$2
    done

Bulk command execution—you can replace the host list with your own actual list of machines:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
/usr/bin/bash



hosts=(
    "ob001"
    "ob002"
    "ob003"
    "obdriver"
    )
    for host in "${hosts[@]}"
    do
        echo "begin to run " $@ " on " $host
        ssh $host $@
    done

Create a User

For personal testing, you can use the root account directly. For enterprise users, it is recommended to create a regular user to avoid security risks to the system. In this example, admin is used as a demonstration; enterprise users can use whichever account they commonly use, according to their own needs.
​

1
2
3
useradd -U admin -d /home/admin -s /bin/bash
mkdir -p /home/admin
sudo chown -R admin:admin /home/admin

Set the password

1
passwd admin

​

Set sudo privileges
vi /etc/sudoers # add one line for oceanbase

1
2
3
4

## Same thing without a password
# %wheel        ALL=(ALL)       NOPASSWD: ALL
admin       ALL=(ALL)       NOPASSWD: ALL

Disk Planning

The OceanBase database server relies on 3 directories. For personal testing, you can place all data on a single disk, but for enterprise users, you must mount 3 disks separately: the data disk, the transaction log disk, and the OBServer installation disk. When a machine does not have 3 disks, or when a RAID disk array is used, you need to partition the disk or the logical volume of the disk array into 3 partitions; for partition sizes, refer to the description below:

  • Data disk

    • The configuration parameter is data_dir. Plan the data disk according to business needs. The data disk holds the baseline data; physically there is only one baseline data file, block_file, located in the installation directory under store/sstable. It is created in one shot when the OBServer process starts, and its size follows a disk pre-allocation strategy based on the startup parameter datafile_disk_percentage (default 95%); the size cannot be adjusted after creation. OceanBase scales in and out by adding or removing machines, and currently does not support disk-level scaling up or down on a single machine.

  • Transaction log disk

    • The configuration parameter is redo_dir. The recommended size is 3 to 4 times the OBServer memory or more. The transaction log disk contains multiple fixed-size small files, located in the installation directory under store/{clog,ilog,slog}, created and purged automatically as needed. When the disk reaches 80%, self-purge logic is triggered, but only on the condition that the in-memory data corresponding to this portion of log data has already been merged and fused into the baseline data—only then can it be deleted. For the same amount of data, the size of the transaction logs is about three times the size of the in-memory data. Therefore, the upper bound of space required by the transaction log disk is proportional to the total volume of business data between two merge operations; the empirical formula is: transaction log file size = 3 to 4 times the in-memory upper limit of incremental data.

  • OBServer installation disk

    • The configuration parameter is home_path. 200G or more is recommended (to retain 7 or more days of logs). OceanBase’s rpm package installs under /home/admin/oceanbase, where the baseline data file and the transaction log file are symlinked to the two separate disks mentioned above. There is also another continuously growing file—the OB runtime log—located in the installation directory under log. The OB process itself cannot self-delete the runtime logs; a scheduled task or operations script is needed to handle the deletion logic.

      After dividing the disks, you can check with the df -h command; the result looks like this:

​
In this example: /data is the data disk with a size of 1TB, /redo stores redo logs, and /home/admin/oceanbase stores the oceanbase binary and runtime logs.
​

Check directory permissions

1
2
3
4
5
6
7
8
ls –al # run this command
drwxr-xr-x 2 admin admin 4096 Feb 9 18:43 
drwxr-xr-x 2 admin admin 4096 Feb 9 18:43 log1

If the admin user has no permission, run the following commands as the root user 
chown -R admin:admin /data
chown -R admin:admin /redo
chown -R admin:admin /home/admin

​

Pre-checks

For enterprise users, it is recommended that all OBServer machines have consistent hardware and software configurations (operating system, OS kernel, glibc, python, and other software packages), and that the OBProxy machines and OBServer machines have consistent software configurations (operating system, OS kernel, glibc, python, and other software packages).

Check the Operating System

The currently supported operating systems are:
​

Red Hat Enterprise Linux Server 7.x (kernel Linux 3.10.0 or above)
CentOS Linux 7.x (kernel Linux 3.10.0 or above)
​

  1. Log in to the server as the root user
  2. Check the OS version

A RedHat 7 system displays as follows 

1
2
[root@redhat-04 /root]#cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.2 (Maipo)

A CentOS 7 system displays as follows: 

1
2
[root@centos-01 /root]#cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)

On an anolis system:

1
2
3
4
5
6
7
8
9
10
[root@anolis ~]# cat /etc/os-release
NAME="Anolis OS"
VERSION="8.2"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.2"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.2"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.org/"

​

On other systems, such as Debian 9, it displays as follows:

1
2
3
4
5
6
7
8
9
10
root@ob001:~# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
VERSION_CODENAME=stretch
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

On an ubuntu system:

1
2
3
4
5
6
7
8
9
10
11
12
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal

For some systems, such as Ubuntu/Debian, you need to install yum:

1
2
3
4
5
6
7
sudo apt-get update
sudo apt-get install build-essential 
sudo apt-get install yum -y
sudo apt install yum-utils -y
sudo ln -s /bin/bash /bin/sh
apt-get install alien -y
apt-get install rpm
	3. Check the kernel version; the OS is required to be 3.10.0 or above

1
2
[root@centos-01 /root]#uname -r 
3.10.0-327.el7.x86_64

Check Memory

1
free -g

Enterprise applications require a minimum of 64G, with 256G or more recommended
If, with free -g
​
the memory shown in the free column is less than the memory_limit setting in the configuration file, you need to clear the cache or modify the memory_limit setting, lowering memory_limit below the value in the free column. The cache-clearing operation is as follows

1
# echo 3 > /proc/sys/vm/drop_caches

​

Check Disks

​

Ensure that the disks corresponding to data_dir, redo_dir, home_path in the configuration file have been mounted, that the directories for data_dir and redo_dir are empty, and that the disk usage of the directory corresponding to data_dir is below 4%.
​

Check the Network Card Name

​
In the configuration file, there is a configuration item “devname” that needs to specify the network card. When starting the OBServer service, you need to specify the network card via the “-i” parameter. A server may have multiple network cards and multiple IPs, and communication between OBServers depends on the specified network card and IP. You can use the ifconfig command to view network card names (you need to install the net-tools dependency package first); just make sure a valid network card exists.
In this example:
​

Configure limits.conf

ulimit is used to limit the resources consumed by processes started from the shell. For personal testing, you can skip this setting, but enterprise users must set it.
There are two ways to modify resource limits: one is to specify them at the session level at startup, and the other is to modify the /etc/security/limits.conf configuration file, which takes effect globally.
The limits involved for the OBServer process include the maximum thread stack size (stack), the maximum number of file handles (open files), and the core file size (core file size).
As shown below, when starting the OBServer process, set the maximum stack size to unlimited, the maximum number of file handles to 655350, and the core file size to unlimited at the session level

1
2
3
4
5
6
7
8
9
10
11
$vi /etc/security/limits.conf to add
root soft nofile 655350
root hard nofile 655350
* soft nofile 655350
* hard nofile 655350
* soft stack 20480
* hard stack 20480
* soft nproc 655360
* hard nproc 655360
* soft core unlimited
* hard core unlimited
	Exit the current session and log in again

​

Check whether the configuration has taken effect

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
ulimit -a
# run this command; the resource limit details are as follows (blocks, -c) 0
core file size
data seg size scheduling priority file size
pending signals max locked memory
(kbytes, -d) unlimited (-e) 0
(blocks, -f) unlimited (-i) 772861
(kbytes, -l) 64
max memory size
open files
pipe size
POSIX message queues real-time priority
stack size
cpu time
max user processes virtual memory file locks
(kbytes, -m) unlimited (-n) 1024
(512 bytes, -p) 8
(bytes, -q) 819200
(-r) 0 (kbytes, -s) 8192
(seconds, -t) unlimited (-u) 655360
(kbytes, -v) unlimited (-x) unlimited

​

Configure the “sysctl.conf” File

To ensure OceanBase runs properly, please modify the “/etc/sysctl.conf” configuration on all physical machines before installing OceanBase (to improve Linux system performance).

Some parameters have already been set in advance by the operating system

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# for oceanbase
## Modify the kernel asynchronous I/O limit
fs.aio-max-nr=1048576

## Network optimization
net.core.somaxconn = 2048
net.core.netdev_max_backlog = 10000 
net.core.rmem_default = 16777216 
net.core.wmem_default = 16777216 
net.core.rmem_max = 16777216 
net.core.wmem_max = 16777216

net.ipv4.ip_local_port_range = 3500 65535 
net.ipv4.ip_forward = 0 
net.ipv4.conf.default.rp_filter = 1 
net.ipv4.conf.default.accept_source_route = 0 
net.ipv4.tcp_syncookies = 0 
net.ipv4.tcp_rmem = 4096 87380 16777216 
net.ipv4.tcp_wmem = 4096 65536 16777216 
net.ipv4.tcp_max_syn_backlog = 16384 
net.ipv4.tcp_fin_timeout = 15 
net.ipv4.tcp_max_syn_backlog = 16384 
net.ipv4.tcp_tw_reuse = 1 
net.ipv4.tcp_tw_recycle = 1 
net.ipv4.tcp_slow_start_after_idle=0

vm.swappiness = 0
vm.min_free_kbytes = 2097152

# this is oceanbase's data directory
kernel.core_pattern = /data/core-%e-%p-%t

Here, in “kernel.core_pattern = /data/core-%e-%p-%t”, /data is oceanbase’s data directory. Also, for personal testing, you can set only “fs.aio-max-nr=1048576”.
​

1
2
# apply the configuration
sysctl -p

​

Disable the Firewall and SELinux

You can skip this for personal testing, but it is recommended for enterprise users.

Disable firewalld

1
2
3
4
# run these 3 commands in order 
systemctl disable firewalld 
systemctl stop firewalld
systemctl status firewalld

Disable selinux

vi /etc/selinux/linux

1
2

SELINUX=disabled

Run this command

1
2
3
setenforce 0
# verify the configuration has taken effect
cat /etc/selinux/config

Set Up Clock Synchronization

You can skip setting up clock synchronization in any of the following situations:

  1. The NTP clock is already in a synchronized state
  2. Deploying a standalone (single-machine) version
  3. Personal testing

​

The time on all servers in an OceanBase cluster must be kept consistent; otherwise the OceanBase cluster will fail to start and faults will occur at runtime. For enterprise users, clock synchronization is extremely, extremely important. A deviation between the physical machine and the clock server of under 50ms can be considered synchronized; the maximum tolerable deviation must not exceed 200ms. When it exceeds 200ms, a no-leader situation occurs; after clock synchronization is restored, restarting the observer can restore the state.

Check Clock Synchronization

1
sudo clockdiff  $IP

Configure Clock Synchronization

https://open.oceanbase.com/docs/community/oceanbase-database/V3.1.0/optional-configuring-clock-sources
​

Installation

Installation Package Components

Download all the installation packages from https://open.oceanbase.com/softwareCenter/community. The package versions shown in this article may already be outdated; please download the latest versions of the packages from the open-source OceanBase official site.
​
If your machine can access the public network and can add third-party YUM software sources, you can run the following commands to install OBD using OceanBase’s official software source:

1
2
3
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://mirrors.aliyun.com/oceanbase/OceanBase.repo
sudo yum install -y ob-deploy

​

scp all the software packages to the control machine

​

Install OBD

Use the root user for now; this operation is performed only on the control machine

Online Installation

1
yum install -y ob-deploy

Local Installation

centos or redhat

1
yum install ob-deploy-1.1.0-1.el7.x86_64.rpm

Ubuntu/Debian

1
alien -i ob-deploy-1.1.0-1.el7.x86_64.rpm

Install OBLibs

Use the root user for now; this needs to be run on every machine.

Online Installation

1
yum install -y oceanbase-ce-libs

Local Installation

First copy oceanbase-ce-libs-3.1.0-3.el7.x86_64.rpm to every machine
​

centos or redhat or anolis

1
yum install oceanbase-ce-libs-3.1.0-3.el7.x86_64.rpm

Ubuntu/Debian

1
alien -i oceanbase-ce-libs-3.1.0-3.el7.x86_64.rpm

Install OBServer & OBProxy

Switch to the admin user
​

Add the OceanBase database offline packages to the local mirror

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
admin@obdriver:/data/rpm$ obd mirror clone *.rpm
name: libobclient
version: 2.0.0
release:2.el7
arch: x86_64
md5: f73cae67e2ff5be0682ac2803aba33a7ed26430e
add libobclient-2.0.0-2.el7.x86_64.rpm to local mirror
name: obclient
version: 2.0.0
release:2.el7
arch: x86_64
md5: 1d2c3ee31f40b9d2fbf97f653f549d896b7e7060
add obclient-2.0.0-2.el7.x86_64.rpm to local mirror
name: ob-deploy
version: 1.1.0
release:1.el7
arch: x86_64
md5: c01dbbebc7f44b700833ce6846df09f20033675c
add ob-deploy-1.1.0-1.el7.x86_64.rpm to local mirror
name: obproxy
version: 3.1.0
release:1.el7
arch: x86_64
md5: 0b17cf0459a3b53c5a2febb6572894d183154c64
add obproxy-3.1.0-1.el7.x86_64.rpm to local mirror
name: oceanbase-ce
version: 3.1.0
release:3.el7
arch: x86_64
md5: b73bcd531bdf3f087391991b290ff2cbcdaa0dc9
add oceanbase-ce-3.1.0-3.el7.x86_64.rpm to local mirror
name: oceanbase-ce-libs
version: 3.1.0
release:3.el7
arch: x86_64
md5: 528144ec7ff0194a8b326491a396b8f5c87b1eaa
add oceanbase-ce-libs-3.1.0-3.el7.x86_64.rpm to local mirror


1
2
3
4
5
6
7
8
9
10
11
12
13
admin@obdriver:~$ obd mirror list local
+-------------------------------------------------------------------------------------------+
|                                     local Package List                                    |
+-------------------+---------+---------+--------+------------------------------------------+
| name              | version | release | arch   | md5                                      |
+-------------------+---------+---------+--------+------------------------------------------+
| libobclient       | 2.0.0   | 2.el7   | x86_64 | f73cae67e2ff5be0682ac2803aba33a7ed26430e |
| obclient          | 2.0.0   | 2.el7   | x86_64 | 1d2c3ee31f40b9d2fbf97f653f549d896b7e7060 |
| ob-deploy         | 1.1.0   | 1.el7   | x86_64 | c01dbbebc7f44b700833ce6846df09f20033675c |
| obproxy           | 3.1.0   | 1.el7   | x86_64 | 0b17cf0459a3b53c5a2febb6572894d183154c64 |
| oceanbase-ce      | 3.1.0   | 3.el7   | x86_64 | b73bcd531bdf3f087391991b290ff2cbcdaa0dc9 |
| oceanbase-ce-libs | 3.1.0   | 3.el7   | x86_64 | 528144ec7ff0194a8b326491a396b8f5c87b1eaa |
+-------------------+---------+---------+--------+------------------------------------------+

Download the Configuration Files

Download all the configuration files from https://github.com/oceanbase/obdeploy/tree/master/example/autodeploy
There are currently several configuration files:
​

​

In this example, we use the distributed example; we scp the distributed configuration file to the control machine.

Modify the Configuration File

In this example, we use distributed-with-obproxy-example.yaml as the example

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
## Only need to configure when remote login is required
# user:
#   username: your username
#   password: your password if need
#   key_file: your ssh-key file path if need
#   port: your ssh port, default 22
#   timeout: ssh connection timeout (second), default 30
oceanbase-ce:
  servers:
    - name: z1
      # Please don't use hostname, only IP can be supported
      ip: 192.168.1.2
    - name: z2
      ip: 192.168.1.3
    - name: z3
      ip: 192.168.1.4
  global:
    # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /root/observer
    # The directory for data storage. The default value is $home_path/store.
    # data_dir: /data
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    # redo_dir: /redo
    # External port for OceanBase Database. The default value is 2881.
    # mysql_port: 2881
    # Internal port for OceanBase Database. The default value is 2882.
    # rpc_port: 2882
    # Defines the zone for an observer. The default value is zone1.
    # zone: zone1
    # The maximum running memory for an observer. When ignored, autodeploy calculates this value based on the current server available resource.
    # memory_limit: 58G
    # The percentage of the maximum available memory to the total memory. This value takes effect only when memory_limit is 0. The default value is 80.
    # memory_limit_percentage: 80 
    # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G. Autodeploy calculates this value based on the current server available resource.
    # system_memory: 22G
    # The size of a data file. When ignored, autodeploy calculates this value based on the current server available resource.
    # datafile_size: 200G
    # The percentage of the data_dir space to the total disk space. This value takes effect only when datafile_size is 0. The default value is 90.
    # datafile_disk_percentage: 90
    # System log level. The default value is INFO.
    # syslog_level: INFO
    # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true. The default value for autodeploy mode is false.
    # enable_syslog_wf: false
    # Enable auto system log recycling or not. The default value is false. The default value for autodeploy mode is on.
    # enable_syslog_recycle: true
    # The maximum number of reserved log files before enabling auto recycling. When set to 0, no logs are deleted. The default value for autodeploy mode is 4.
    # max_syslog_file_count: 4
    # Cluster name for OceanBase Database. The default value is obcluster. When you deploy OceanBase Database and obproxy, this value must be the same as the cluster_name for obproxy.
    # appname: obcluster
    # Password for root. The default value is empty.
    # root_password:
    # Password for proxyro. proxyro_password must be the same as observer_sys_password. The default value is empty.
    # proxyro_password:
  z1:
    zone: zone1
  z2:
    zone: zone2
  z3:
    zone: zone3
obproxy:
  servers:
    - 192.168.1.5
  global:
    # The working directory for obproxy. Obproxy is started under this directory. This is a required field.
    home_path: /root/obproxy
    # External port. The default value is 2883.
    # listen_port: 2883
    # The Prometheus port. The default value is 2884.
    # prometheus_listen_port: 2884
    # rs_list is the root server list for observers. The default root server is the first server in the zone.
    # The format for rs_list is observer_ip:observer_mysql_port;observer_ip:observer_mysql_port.
    # Ignore this value in autodeploy mode.
    # rs_list: 127.0.0.1:2881
    # Cluster name for the proxy OceanBase Database. The default value is obcluster. This value must be set to the same with the appname for OceanBase Database.
    # cluster_name: obcluster
    # Password for obproxy system tenant. The default value is empty.
    # obproxy_sys_password:
    # Password for proxyro. proxyro_password must be the same with proxyro_password. The default value is empty.
    # observer_sys_password:
1
2
3
4
5
6
7
## Only need to configure when remote login is required
# user:
#   username: your username
#   password: your password if need
#   key_file: your ssh-key file path if need
#   port: your ssh port, default 22
#   timeout: ssh connection timeout (second), default 30

Modify the username and password
​

Typically, a few of these variables need to be set by hand: the IP of each machine, plus home_path, data_dir, and redo_dir. In this example, they are changed to /home/admin/oceanbase/ob, /data/ob, and /redo/ob respectively—the disks mounted earlier.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
oceanbase-ce:
  servers:
    - name: z1
      # Please don't use hostname, only IP can be supported
      ip: 172.30.62.200
    - name: z2
      ip: 172.30.62.201
    - name: z3
      ip: 172.30.62.202
  global:
    # The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /home/admin/oceanbase/ob
    # The directory for data storage. The default value is $home_path/store.
    data_dir: /data/ob
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    redo_dir: /redo/ob

Configure the proxy, modifying the IP and home_path

1
2
3
4
5
6
obproxy:
  servers:
    - 172.30.62.203
  global:
    # The working directory for obproxy. Obproxy is started under this directory. This is a required field.
    home_path: /home/admin/oceanbase

I also recommend a website, https://www.bejson.com/validators/yaml_editor/, which can perform YAML validation on the configuration file. Very often, an extra space or a missing space in a configuration file is extremely hard to spot.
​

​

Start the Installation

For an offline installation, there is one step to perform: when you cannot connect to the server, you need to delete the remote repo configuration to avoid wasting time on connecting to the remote repo.

1
rm -fr ~/.obd/mirror/remote/*.repo

Start the installation

1
admin@obdriver:~$ obd cluster autodeploy obtest -c distributed-with-obproxy-example.yaml

Check whether the installation succeeded

1
2
3
4
5
6
7
8
admin@obdriver:~$ obd cluster list
+------------------------------------------------------------+
|                        Cluster List                        |
+--------+---------------------------------+-----------------+
| Name   | Configuration Path              | Status (Cached) |
+--------+---------------------------------+-----------------+
| obtest | /home/admin/.obd/cluster/obtest | running         |
+--------+---------------------------------+-----------------+
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
admin@obdriver:~$ obd cluster display obtest
Get local repositories and plugins ok
Open ssh connection ok
Cluster status check ok
Connect to observer ok
Wait for observer init ok
+-------------------------------------------------+
|                     observer                    |
+---------------+---------+------+-------+--------+
| ip            | version | port | zone  | status |
+---------------+---------+------+-------+--------+
| 172.30.62.200 | 3.1.0   | 2881 | zone1 | active |
| 172.30.62.201 | 3.1.0   | 2881 | zone2 | active |
| 172.30.62.202 | 3.1.0   | 2881 | zone3 | active |
+---------------+---------+------+-------+--------+

Connect to obproxy ok
+-------------------------------------------------+
|                     obproxy                     |
+---------------+------+-----------------+--------+
| ip            | port | prometheus_port | status |
+---------------+------+-----------------+--------+
| 172.30.62.203 | 2883 | 2884            | active |
+---------------+------+-----------------+--------+

Modify the Configuration

The OceanBase database has hundreds of configuration items, some of which are coupled. Until you are familiar with the OceanBase database, we do not recommend modifying the settings in the example configuration file. This example is intended to illustrate how to modify a configuration and make it take effect.
​

For an introduction to all parameters, refer to

1
https://github.com/oceanbase/obdeploy/blob/master/plugins/oceanbase/3.1.0/parameter.yaml
1
2
3
4
5
6
7
# Use the edit-config command to enter edit mode and modify the cluster configuration
obd cluster edit-config lo
# Change sys_bkgd_migration_retry_num to 5
# Note: the minimum value of sys_bkgd_migration_retry_num is 3
# After saving and exiting, OBD will tell you how to make this change take effect
# This configuration item only needs a reload to take effect
obd cluster reload lo

Verification​

Install obclient

obclient is typically installed on the control machine; you need to switch to the root account

Online Installation

1
2
yum install -y libobclient
yum install -y obclient

Local Installation

​

centos or redhat or anolis

1
2
yum install libobclient-2.0.0-2.el7.x86_64.rpm
yum install obclient-2.0.0-2.el7.x86_64.rpm

Ubuntu/Debian

1
2
alien -i libobclient-2.0.0-2.el7.x86_64.rpm
alien -i obclient-2.0.0-2.el7.x86_64.rpm

On debian, you need to add the path to the system environment variables

1
export PATH=/app/mariadb/bin:$PATH

​

On ubuntu, you need to add the path to the system environment variables

1
export PATH=/u01/obclient/bin:$PATH

​

​

Install the MySQL Development Packages

If you need to run sysbench, tpch, or similar programs, you need to install the MySQL development packages

​

centos or redhat or anolis

1
2
3
yum install mariadb
yum install mariadb-libs
yum install mariadb-devel

Ubuntu

1
apt-get install mariadb-server

Debian

1
apt-get install mysql-server mysql-client libmariadbd18 libmariadbd-dev

Check Tenants

To use oceanbase, you need to create a tenant; real user applications must run within a tenant.
There are 2 ways to create a tenant:
You can use obd to create a tenant; when creating a tenant with obd, it allocates all the resources

1
obd cluster tenant create ${cluster_name} -n ${tenant_name}

​

To create a tenant, refer to
https://open.oceanbase.com/docs/community/oceanbase-database/V3.1.0/create-a-user-tenant
​

In this example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
admin@obdriver:~$ mysql -h${obproxy_ip} -P${obproxy_port} -uroot
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.25 OceanBase 3.1.0 (r3-b20901e8c84d3ea774beeaca963c67d7802e4b4e) (Built Aug 10 2021 07:51:04)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [(none)]> use oceanbase;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MySQL [oceanbase]> select * from gv$tenant;
+-----------+-------------+-------------------+-------------------+----------------+---------------+-----------+---------------------------------------------+
| tenant_id | tenant_name | zone_list         | primary_zone      | collation_type | info          | read_only | locality                                    |
+-----------+-------------+-------------------+-------------------+----------------+---------------+-----------+---------------------------------------------+
|         1 | sys         | zone1;zone2;zone3 | zone1;zone2,zone3 |              0 | system tenant |         0 | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 |
|      1001 | mytest      | zone1;zone2;zone3 | RANDOM            |              0 |               |         0 | FULL{1}@zone1, FULL{1}@zone2, FULL{1}@zone3 |
+-----------+-------------+-------------------+-------------------+----------------+---------------+-----------+---------------------------------------------+
2 rows in set (0.00 sec)

MySQL [(none)]>

​

Best Practices for Enterprise Users

Disks

The OB runtime logs, transaction logs, and data files must be kept separate. If there are not 3 disks available, you can partition a single disk into 3 partitions.
​