2023-01-13 LightDB分布式部署.md

LightDB分布式部署

• 分布式部署模式

# 1.常规模式
    1台服务器作为协调者节点，N(N>1)台服务器作为工作节点,每个节点都按照高可用方式进行部署
    比如，一个协调节点，2个工作节点，
    * 工作节点按照一主一从高可用方式部署，则一共需要[(1+2)*2=6台]服务器
    * 工作节点按照一主一从一witness方式部署，则一共需要[(1+2)*3=9台]服务器    
# 2.多机单实例
    1台服务器作为协调者节点，N(N>1)台服务器作为工作节点,每个节点都按照单机模式部署
# 3.单机多实例
    使用1台服务器同时安装协调者节点和工作节点的实例，所有实例均按照单机模式部署

生产环境仅建议使用常规模式部署。

• 常规模式部署

1.服务器列表

1. 182.168.121.211
2. 182.168.121.212
3. 182.168.121.213
4. 182.168.121.214
5. 182.168.121.215
6. 182.168.121.216

• 命令行安装

[lightdb@localhost lightdb-x-13.8-22.3-7953-el7.x86_64]$ pwd
/home/lightdb/lightdb-x-13.8-22.3-7953-el7.x86_64
[lightdb@localhost lightdb-x-13.8-22.3-7953-el7.x86_64]$ ls
install.sh  jdk  lightdb-installer-13.8-22.3.jar  lightdb-uninstaller-13.8-22.3.jar  lightdb-x  log  script  system-lib
[lightdb@localhost lightdb-x-13.8-22.3-7953-el7.x86_64]$ 

# 1.执行install.sh
[lightdb@localhost lightdb-x-13.8-22.3-7953-el7.x86_64]$ ./install.sh
Whether to use the graphical user interface (GUI, Make sure DISPLAY is configured, Such as [export DISPLAY=127.0.0.1:0.0])?(Yes or No)
no
Choice a kind of configuration mode!
1: Only install.
2: Install database and Create instance.
3: Developer
Please enter 1 2 or 3(The default is 1):
2
Choice a kind of install mode!

1: Single Mode.
2: High Availability Mode
3: Distributed Mode
Please enter 1, 2 or 3:(The default is 1)
3
Please select distributed mode!
Please enter distributed mode:
1 Normal mode;
2 Multi-server single instance;
3 Single server multi-instance.(Default 1)
1
Config Server for High Availability Mode!
Please enter coordinator node:----配置CN节点
Please enter a virtual IP address with network, such as (192.168.217.234/16):
192.168.121.201/24
The cluster need a primary server(required), one or more standby server(optional) and a witness server(optional).
Please input a primary server ip which is actually the ip of the current server(required). such as (192.168.217.234/16):
192.168.121.211/24 ---CN主节点
Please input a standby server ip(required), such as (192.168.217.234/16):
192.168.121.212/24 ---CN从节点
Please enter status of the standby server [192.168.121.212]
1 sync:
2 async:
1
Continue to add standby nodes?(enter [exit] to left.)：
exit
Please input a witness server ip(optional), such as (192.168.217.234/16):
                    ----此处先不配置witness节点
Witness is empty
Please enter a port of the cluster. (Default port 5432):

======================================= Servers =======================================
Vip: 192.168.121.201
Port: 5432
Ip: 192.168.121.211 Role: primary Status: none
Ip: 192.168.121.212 Role: standby Status: sync
======================================= Servers =======================================
Please check the servers of the cluster. 1 continue; 2 clear server config (Default 1)
Please check the servers of the cluster. 1 continue; 2 clear server config (Default 1)
1
Please enter work node: ----添加一个work节点
Please enter a virtual IP address with network, such as (192.168.217.234/16):
192.168.121.202/24
The cluster need a primary server(required), one or more standby server(optional) and a witness server(optional).
Please input a primary server ip which is actually the ip of the current server(required). such as (192.168.217.234/16):
192.168.121.213/24 
Please input a standby server ip(required), such as (192.168.217.234/16):
192.168.121.214/24
Please enter status of the standby server [192.168.121.214]
1 sync:
2 async:
1
Continue to add standby nodes?(enter [exit] to left.)：
exit
Please input a witness server ip(optional), such as (192.168.217.234/16):

Witness is empty
Please enter a port of the cluster. (Default port 5432):

======================================= Servers =======================================
Vip: 192.168.121.202
Port: 5432
Ip: 192.168.121.213 Role: primary Status: none
Ip: 192.168.121.214 Role: standby Status: sync
======================================= Servers =======================================
Please check the servers of the cluster. 1 continue; 2 clear server config (Default 1)
Please enter work node: ---添加第二个work
Please enter a virtual IP address with network, such as (192.168.217.234/16):
192.168.121.203/24
The cluster need a primary server(required), one or more standby server(optional) and a witness server(optional).
Please input a primary server ip which is actually the ip of the current server(required). such as (192.168.217.234/16):
192.168.121.215/24
Please input a standby server ip(required), such as (192.168.217.234/16):
192.168.121.216/24
Please enter status of the standby server [192.168.121.216]
1 sync:
2 async:
1
Continue to add standby nodes?(enter [exit] to left.)：
exit
Please input a witness server ip(optional), such as (192.168.217.234/16):

Witness is empty
Please enter a port of the cluster. (Default port 5432):

======================================= Servers =======================================
Vip: 192.168.121.203
Port: 5432
Ip: 192.168.121.215 Role: primary Status: none
Ip: 192.168.121.216 Role: standby Status: sync
======================================= Servers =======================================
Please check the servers of the cluster. 1 continue; 2 clear server config (Default 1)
1
Continue to add worker node(1[yes],2[no])?
2
====================================== Generate pg_hba ================================
====================================== Generate end ===================================
====================================== Copying files takes time =======================
192.168.121.212: start copy
192.168.121.212: end copy
192.168.121.213: start copy
192.168.121.213: end copy
192.168.121.214: start copy
192.168.121.214: end copy
192.168.121.215: start copy
192.168.121.215: end copy
192.168.121.216: start copy
192.168.121.216: end copy
====================================== Copying files end ==============================
Check system parameters and dependency packages!
========================================= 192.168.121.211 =========================================
NETWORK
name: sem, recommend value: 500,2048000,200,4096, current value: 500,2048000,200,4096, status: OK
name: aio_max_nr, recommend value: 1048576, current value: 1048576, status: OK
name: somaxconn, recommend value: 2000, current value: 2000, status: OK
name: tcp_max_syn_backlog, recommend value: 2000, current value: 2000, status: OK
name: tcp_tw_reuse, recommend value: 1, current value: 1, status: OK
name: tcp_syn_retries, recommend value: 3, current value: 3, status: OK
name: tcp_retries2, recommend value: 5, current value: 5, status: OK
name: tcp_slow_start_after_idle, recommend value: 0, current value: 0, status: OK
PAGE_CACHE
name: dirty_background_ratio, recommend value: 5, current value: 5, status: OK
name: dirty_ratio, recommend value: 40, current value: 40, status: OK
name: dirty_expire_centisecs, recommend value: 500, current value: 500, status: OK
name: dirty_writeback_centisecs, recommend value: 250, current value: 250, status: OK
MEMORY
name: shmmni, recommend value: 4096, current value: 4096, status: OK
name: shmmax, recommend value: 953868288, current value: 1976979456, status: WARNING
name: shmall, recommend value: 232878, current value: 482661, status: WARNING
name: swappiness, recommend value: 5, current value: 5, status: OK
name: overcommit_memory, recommend value: 2, current value: 2, status: OK
name: overcommit_ratio, recommend value: 75, current value: 75, status: OK
FILE_HANDLER
name: file_max, recommend value: 524288, current value: 524288, status: OK
ULIMIT
name: ulimit_core, recommend value: unlimited, current value: unlimited, status: OK
name: ulimit_nofile, recommend value: 8192, current value: 524288, status: OK
Dependency Package
name: JSON-C-0.11 is existed: yes
name: C-ARES-1 is existed: yes
name: LIBNL3 is existed: yes
name: LIBPCAP-1 is existed: yes
name: LIBZSTD-1 is existed: yes
name: LZ4-1 is existed: yes
name: NCURSES-LIBS-5 is existed: yes
name: READLINE-6 is existed: yes
name: SNAPPY-1 is existed: yes
name: UUID-1.6 is existed: yes
name: LIBICU-50 is existed: yes
System service
name: ntp is install: no
========================================= 192.168.121.212 =========================================
.....
========================================= 192.168.121.213 =========================================
.....
========================================= 192.168.121.214 =========================================
.....
========================================= 192.168.121.215 =========================================
.....
========================================= 192.168.121.216 =========================================
NETWORK
name: sem, recommend value: 500,2048000,200,4096, current value: 500,2048000,200,4096, status: OK
name: aio_max_nr, recommend value: 1048576, current value: 1048576, status: OK
name: somaxconn, recommend value: 2000, current value: 2000, status: OK
name: tcp_max_syn_backlog, recommend value: 2000, current value: 2000, status: OK
name: tcp_tw_reuse, recommend value: 1, current value: 1, status: OK
name: tcp_syn_retries, recommend value: 3, current value: 3, status: OK
name: tcp_retries2, recommend value: 5, current value: 5, status: OK
name: tcp_slow_start_after_idle, recommend value: 0, current value: 0, status: OK
PAGE_CACHE
name: dirty_background_ratio, recommend value: 5, current value: 5, status: OK
name: dirty_ratio, recommend value: 40, current value: 40, status: OK
name: dirty_expire_centisecs, recommend value: 500, current value: 500, status: OK
name: dirty_writeback_centisecs, recommend value: 250, current value: 250, status: OK
MEMORY
name: shmmni, recommend value: 4096, current value: 4096, status: OK
name: shmmax, recommend value: 1976979456, current value: 1976979456, status: OK
name: shmall, recommend value: 482661, current value: 482661, status: OK
name: swappiness, recommend value: 5, current value: 5, status: OK
name: overcommit_memory, recommend value: 2, current value: 2, status: OK
name: overcommit_ratio, recommend value: 75, current value: 75, status: OK
FILE_HANDLER
name: file_max, recommend value: 524288, current value: 524288, status: OK
ULIMIT
name: ulimit_core, recommend value: unlimited, current value: unlimited, status: OK
name: ulimit_nofile, recommend value: 8192, current value: 524288, status: OK
Dependency Package
name: JSON-C-0.11 is existed: yes
name: C-ARES-1 is existed: yes
name: LIBNL3 is existed: yes
name: LIBPCAP-1 is existed: yes
name: LIBZSTD-1 is existed: yes
name: LZ4-1 is existed: yes
name: NCURSES-LIBS-5 is existed: yes
name: READLINE-6 is existed: yes
name: SNAPPY-1 is existed: yes
name: UUID-1.6 is existed: yes
name: LIBICU-50 is existed: yes
System service
name: ntp is install: no
Choice a kind of Compatible Type!
1: LightdDB(Compatible with PostgreSQL).
2: ORACLE(Compatible with ORACLE).
3: MYSQL(Compatible with MYSQL).
Please enter 1, 2 or 3:(The default is 1)
1
Choice a kind of LightDB workload!
1: OLTP(On-line Transaction Processing).
2: OLAP(On-Line Analytical Processing).
Please enter 1 or 2:(The default is 1)
1
Specify a path for installing all LightDB software and storing configuration information.
Please enter base location(The default is /usr/local/lightdb):

Base Location: /usr/local/lightdb
Install Location: /usr/local/lightdb/lightdb-x/13.8-22.3
Please enter instance location(The default is /usr/local/lightdb/lightdb-x/13.8-22.3/data/defaultCluster/):

Instance location: /usr/local/lightdb/lightdb-x/13.8-22.3/data/defaultCluster/
Please configure memory(MB) and character set!
Please enter shared_buffers, Default value is (454):

Please enter effective_cache_size, Default value is (1273):

Please choice a kind of Character Set.
1. UTF8
2. GBK
3. SQL_ASCII
4. LATIN1
The default choice 1(UTF8)
1
Please enter LightDB password!
Please enter original password:
Please enter confirm password:

Do you want to deploy immediately?(Yes or No, The default is yes)
yes

• 问题

#ssh超时
packet_write_wait: Connection to 192.168.121.214 port 22: Broken pipe
解决；~/.ssh/config文件中添加
$ vi ~/.ssh/config
ServerAliveInterval 60
IPQoS lowdelay throughput