OceanBase 内存研究(OceanBase 3.2.4.5)

内存结构

从官网的结构图可以看出，一台observer可使用的总内存(memory_limit)包括 系统内存(system_memory) 和 租户内存(sys租户与普通租户)

系统内存

系统内存system_memory 属于 observer 的内部内存，允许其它租户共享使用该内存资源

(root@10.0.0.61:2883) [oceanbase]> show parameters like 'system_memory';
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name          | data_type | value | info                                                                                                                                                                    | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | system_memory | NULL      | 1G    | the memory reserved for internal use which cannot be allocated to any outer-tenant, and should be determined to guarantee every server functions normally. Range: [0M,) | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
1 row in set (0.018 sec)

租户内存

指 sys租户内存 和 创建的普通租户内存，OceanBase集群的租户 真正可分配使用的内存 为 可使用的总内存(memory_limit) — 系统内存(system_memory)，每个租户内存 包含 增量数据的 MemStore（tenant memstore）以及 KVCache 缓存（tenant cache memory）。

注:以下实验环节使用OceanBase 3.2.4.5企业版本，均使用sys租户登录查看

查看OBserver内存

可以查看当前 OBserver 总内存 mem_total 为15G，已经分配了10G，总的CPU为14，目前CPU已经全部分配

(root@10.0.0.61:2883) [oceanbase]> select zone,cpu_total,cpu_assigned,mem_total/1024/1024/1024,mem_assigned/1024/1024/1024 from __all_virtual_server_stat;
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone  | cpu_total | cpu_assigned | mem_total/1024/1024/1024 | mem_assigned/1024/1024/1024 |
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone1 |        14 |           14 |          15.000000000000 |             10.000000000000 |
+-------+-----------+--------------+--------------------------+-----------------------------+
1 row in set (0.015 sec)


(root@10.0.0.61:2883) [oceanbase]>

注意

mem_total值(租户可使用内存) 为 总内存(memory_limit) — 系统内存(system_memory)

(root@10.0.0.61:2883) [oceanbase]> show parameters like 'system_memory';
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name          | data_type | value | info                                                                                                                                                                    | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | system_memory | NULL      | 1G    | the memory reserved for internal use which cannot be allocated to any outer-tenant, and should be determined to guarantee every server functions normally. Range: [0M,) | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+---------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
1 row in set (0.011 sec)


(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit';
+-------+----------+-----------+----------+--------------+-----------+-------+-------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name         | data_type | value | info                                                                                | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+--------------+-----------+-------+-------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit | NULL      | 16G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,) | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+--------------+-----------+-------+-------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
1 row in set (0.019 sec)


(root@10.0.0.61:2883) [oceanbase]>

查看租户内存

查看每个租户可用内存

这里查看的是 每个租户可使用的最大内存，包含memstore。 其实就是创建租户时指定的max_memory的值

(root@10.0.0.61:2883) [oceanbase]> select tenant_id,tenant_name,max_cpu,min_cpu,max_memory/1024/1024/1024 from v$unit;
+-----------+-------------+---------+---------+---------------------------+
| tenant_id | tenant_name | max_cpu | min_cpu | max_memory/1024/1024/1024 |
+-----------+-------------+---------+---------+---------------------------+
|      NULL | NULL        |       2 |       2 |            2.000000000000 |
|         1 | sys         |       4 |       4 |            2.000000000000 |
|      1001 | obmysql     |       2 |       2 |            4.000000000000 |
|      1002 | oboracle    |       2 |       2 |            2.000000000000 |
|      1006 | tnt_pay     |       2 |       2 |            1.000000000000 |
|      1009 | oboms       |       2 |       2 |            1.000000000000 |
+-----------+-------------+---------+---------+---------------------------+
6 rows in set (0.032 sec)


(root@10.0.0.61:2883) [oceanbase]>

查看每个租户 memstore 内存

通过 tenant_id(租户标识字段) 与 memstore_limit(可使用的memstore内存) 来查看租户内存情况。500租户(系统内存存放的租户)可使用内存最大，为4G

(root@10.0.0.61:2883) [oceanbase]> select tenant_id,round(active/1024/1024/1024,2) active,round(total/1024/1024/1024,2) total,round(freeze_trigger/1024/1024/1024,2)
    -> freeze_trigger,round(mem_limit/1024/1024/1024,2) memstore_limit from v$memstore;
+-----------+--------+-------+----------------+----------------+
| tenant_id | active | total | freeze_trigger | memstore_limit |
+-----------+--------+-------+----------------+----------------+
|         1 |   0.20 |  0.20 |           0.30 |           1.00 |
|       500 |   0.00 |  0.00 |  1288490188.80 |  4294967296.00 |
|      1001 |   0.13 |  0.13 |           0.60 |           2.00 |
|      1002 |   0.11 |  0.11 |           0.30 |           1.00 |
|      1006 |   0.10 |  0.10 |           0.15 |           0.50 |
|      1009 |   0.10 |  0.28 |           0.15 |           0.50 |
+-----------+--------+-------+----------------+----------------+
6 rows in set (0.010 sec)


(root@10.0.0.61:2883) [oceanbase]>

修改OBserver内存

可以看到 memory_limit_percentage(可使用服务器物理内存最大百分比) 与 memory_limit(OBserver总内存) 分别为 90% 与 16G

(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit%';
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name                    | data_type | value | info                                                                                   | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit_percentage | NULL      | 90    | the size of the memory reserved for internal use(for testing purpose). Range: [10, 90] | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit            | NULL      | 16G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,)    | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
2 rows in set (0.005 sec)


(root@10.0.0.61:2883) [oceanbase]>

修改 memory_limit_percentage

将百分比值改为60

(root@10.0.0.61:2883) [oceanbase]> alter system set memory_limit_percentage=60;
Query OK, 0 rows affected (0.389 sec)


(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit%';
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name                    | data_type | value | info                                                                                   | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit_percentage | NULL      | 60    | the size of the memory reserved for internal use(for testing purpose). Range: [10, 90] | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit            | NULL      | 16G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,)    | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
2 rows in set (0.011 sec)

再次查看OBserver总内存

可以发现 即使修改 memory_limit_percentage 的值，但是mem_total总内存依然不变

(root@10.0.0.61:2883) [oceanbase]> select zone,cpu_total,cpu_assigned,mem_total/1024/1024/1024,mem_assigned/1024/1024/1024 from __all_virtual_server_stat;
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone  | cpu_total | cpu_assigned | mem_total/1024/1024/1024 | mem_assigned/1024/1024/1024 |
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone1 |        14 |           14 |          15.000000000000 |             10.000000000000 |
+-------+-----------+--------------+--------------------------+-----------------------------+
1 row in set (0.021 sec)


(root@10.0.0.61:2883) [oceanbase]>

修改 memory_limit

将 memory_limit 值改为13

(root@10.0.0.61:2883) [oceanbase]> alter system set memory_limit='13G';
Query OK, 0 rows affected (0.175 sec)


(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]>
(root@10.0.0.61:2883) [oceanbase]> show parameters like 'memory_limit%';
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone  | svr_type | svr_ip    | svr_port | name                    | data_type | value | info                                                                                   | section  | scope   | source  | edit_level        |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit_percentage | NULL      | 60    | the size of the memory reserved for internal use(for testing purpose). Range: [10, 90] | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
| zone1 | observer | 10.0.0.61 |     2882 | memory_limit            | NULL      | 13G   | the size of the memory reserved for internal use(for testing purpose). Range: [0M,)    | OBSERVER | CLUSTER | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+-----------+----------+-------------------------+-----------+-------+----------------------------------------------------------------------------------------+----------+---------+---------+-------------------+
2 rows in set (0.006 sec)


(root@10.0.0.61:2883) [oceanbase]>

再次查看OBserver总内存

修改 memory_limit 的值，mem_total总内存发生变化

(root@10.0.0.61:2883) [oceanbase]> select zone,cpu_total,cpu_assigned,mem_total/1024/1024/1024,mem_assigned/1024/1024/1024 from __all_virtual_server_stat;
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone  | cpu_total | cpu_assigned | mem_total/1024/1024/1024 | mem_assigned/1024/1024/1024 |
+-------+-----------+--------------+--------------------------+-----------------------------+
| zone1 |        14 |           14 |          12.000000000000 |             10.000000000000 |
+-------+-----------+--------------+--------------------------+-----------------------------+
1 row in set (0.003 sec)

总结

1.当 memory_limit_percentage(可使用服务器物理内存最大百分比) 值 与 memory_limit(OBserver总内存) 值 均不为空时，OBserver总内存以 memory_limit为主，不受 memory_limit_percentage 影响。如果 memory_limit值为0，OBserver总内存 受

memory_limit_percentage 改变影响。

2.每个租户可用内存 包含 增量数据的 MemStore（tenant memstore）以及 KVCache 缓存（tenant cache memory）

3.OceanBase集群的租户 真正可分配使用的内存 为 可使用的总内存(memory_limit) — 系统内存(system_memory)，但系统内部内存允许其它租户共享使用该内存资源