GreenPlum数据库对于数据的冗余,也支持类似Oracle数据库物理DataGuard的数据冗余机制,Master的数据库镜像称为Standby,Segment节点的数据库镜像称之为Mirror,本文主要介绍如何为没有做Standby的Master节点添加Standby。
需要注意,在为Master节点搭建Standby的过程中,GreenPlum会自动关闭数据库,并以utility模式打开Master节点,然后修改gp_segment_configuration字典中增加Standby的信息,然后再关闭Master节点,将Master的数据拷贝到Standby节点,最后启动数据库,所以,在为Master节点添加Standby时,需要在空闲时段进行,否则会影响业务。
在搭建Standby的时候,Standby主机需要先安装GreenPlum数据库软件,这和正常安装GreenPlum数据库软件没有区别,从规范化考虑,尽量使用和Master主机同样的安装路径,下面简单结束安装过程。
1.创建gpadmin用户及安装目录
01
[root@mdw-std ~]
# groupadd -g 520 gpadmin
02
[root@mdw-std ~]
# useradd -u 520 -g gpadmin gpadmin
03
[root@mdw-std ~]
# passwd gpadmin
04
Changing password for user gpadmin.
06
BAD PASSWORD: it is based on a dictionary word
07
BAD PASSWORD: is too simple
09
passwd: all authentication tokens updated successfully.
10
[root@mdw-std ~]
# mkdir -p /gpdb/app
11
[root@mdw-std ~]
# mkdir -p /gpdb/gpdata
12
[root@mdw-std ~]
# chown -R gpadmin:gpadmin /gpdb/
2.配置hosts文件,添加所有主机的信息
1
[gpadmin@mdw config]$ vi /etc/hosts
除Standby主机外,其他所有主机(Master节点和所有Segment节点)的hosts文件均需要添加Standby主机的信息,也就是所有GreenPlum数据库包含的主机的hosts文件都需要包含以上hosts文件中的所有主机信息。
3.上传安装文件及解压
01
[gpadmin@mdw-std gpdb]$ scp 10.9.15.20:/gpdb/*.zip .
02
The authenticity of host '10.9.15.20 (10.9.15.20)' can't be established.
03
RSA key fingerprint is 61:72:68:57:16:28:40:d4:bc:9e:68:f0:bc:ac:65:e9.
04
Are you sure you want to continue connecting (yes/no)? yes
05
Warning: Permanently added '10.9.15.20' (RSA) to the list of known hosts.
06
gpadmin@10.9.15.20's password:
07
greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip 100% 134MB 133.7MB/s 00:01
08
[gpadmin@mdw-std gpdb]$ unzip greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip
09
Archive: greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.zip
10
inflating: README_INSTALL
11
inflating: greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.bin
4.配置内核参数,添加以下内容
01
[root@mdw-std ~]
# vi /etc/sysctl.conf
03
kernel.shmmax = 500000000
04
kernel.shmall = 4000000000
06
kernel.sem = 250 512000 100 2048
08
kernel.core_uses_pid = 1
12
net.ipv4.tcp_syncookies = 1
13
net.ipv4.ip_forward = 0
14
net.ipv4.conf.default.accept_source_route = 0
15
net.ipv4.tcp_tw_recycle = 1
16
net.ipv4.tcp_max_syn_backlog = 4096
17
net.ipv4.conf.all.arp_filter = 1
18
net.ipv4.conf.default.arp_filter = 1
19
net.ipv4.ip_local_port_range = 1025 65535
20
net.core.netdev_max_backlog = 10000
21
vm.overcommit_memory = 2
通过sysctl命令使之生效。
1
[root@mdw-std ~]
# sysctl -p
5.配置资源限额(limit)
1
[root@mdw-std ~]
# vi /etc/security/limits.conf
2
gpadmin soft nofile 65536
3
gpadmin hard nofile 65536
4
gpadmin soft nproc 131072
5
gpadmin hard nproc 131072
6.修改磁盘模式,/boot/grub/menu.lst文件增加elevator=deadline。
01
[root@mdw-std ~]
# echo deadline > /sys/block/sda/queue/scheduler
02
[root@mdw-std ~]
# cat /sys/block/sda/queue/scheduler
05
[root@mdw-std ~]
# vi /boot/grub/menu.lst
06
# grub.conf generated by anaconda
08
# Note that you do not have to rerun grub after making changes to this file
09
# NOTICE: You have a /boot partition. This means that
10
# all kernel and initrd paths are relative to /boot/, eg.
12
# kernel /vmlinuz-version ro root=/dev/sda3
13
# initrd /initrd-[generic-]version.img
17
splashimage=(hd0,0)/grub/splash.xpm.gz
19
title Oracle Linux Server Unbreakable Enterprise Kernel (3.8.13-16.2.1.el6uek.x86_64)
21
kernel /vmlinuz-3.8.13-16.2.1.el6uek.x86_64 ro root=UUID=1fb9c4cf-3cf4-47db-bf4a-0a87958d477d rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
22
initrd /initramfs-3.8.13-16.2.1.el6uek.x86_64.img
23
title Oracle Linux Server Red Hat Compatible Kernel (2.6.32-431.el6.x86_64)
25
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=1fb9c4cf-3cf4-47db-bf4a-0a87958d477d rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM elevator=deadline rhgb quiet
26
initrd /initramfs-2.6.32-431.el6.x86_64.img
7.配置磁盘预读块
1
[root@mdw-std ~]
# /sbin/blockdev --setra 65535 /dev/sda3
8.安装GreenPlum数据库软件
1
[gpadmin@mdw-std gpdb]$ /bin/bash greenplum-db-4.3.6.2-build-1-RHEL5-x86_64.bin
跳过一堆的许可信息后,会提示license相关的信息。
1
********************************************************************************
2
Do you accept the Pivotal Database license agreement? [yes|no]
3
********************************************************************************
输入yes确认后,会提示安装路径相关信息。
3
********************************************************************************
4
Provide the installation path for Greenplum Database or press ENTER to
5
accept the default installation path: /usr/local/greenplum-db-4.3.6.2
6
********************************************************************************
这里使用和Master相同的安装路径。
3
********************************************************************************
4
Install Greenplum Database into </gpdb/app>? [yes|no]
5
********************************************************************************
输入yes确认安装路径。
03
Extracting product to /gpdb/app
05
********************************************************************************
07
Greenplum Database is installed in /gpdb/app
09
Pivotal Greenplum documentation is available
11
********************************************************************************
GreenPlum数据库软件安装完成,需要配置gpadmin用户的环境变量,增加以下内容。
1
[gpadmin@mdw-std app]$ vi /home/gpadmin/.bash_profile
2
source /gpdb/app/greenplum_path.sh
使之生效。
1
[gpadmin@mdw-std app]$ . /home/gpadmin/.bash_profile
9.Standby主机创建数据库初始化数据存放目录及文件空间目录,需要和Master目录一样。
1
[gpadmin@mdw-std gpdata]$ mkdir /gpdb/gpdata/master
2
[gpadmin@mdw-std gpdata]$ mkdir /gpdb/gpdata/fspc_master
以上均是在Standby节点的主机上操作,接下来是在Master节点的主机上操作。
10.在Master节点配置所有节点于Standby节点的ssh互信。
01
[gpadmin@mdw config]$ vi hostlist
09
[gpadmin@mdw config]$ gpssh-exkeys -f hostlist
10
[STEP 1 of 5] create local ID and authorize on local host
11
... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped
13
[STEP 2 of 5] keyscan all hosts and update known_hosts file
15
[STEP 3 of 5] authorize current user on remote hosts
18
*** Enter password for mdw-std:
23
[STEP 4 of 5] determine common authentication file content
25
[STEP 5 of 5] copy authentication files to all remote hosts
26
... finished key exchange with mdw-std
27
... finished key exchange with sdw1
28
... finished key exchange with sdw2
29
... finished key exchange with sdw3
31
[INFO] completed successfully
通过gpssh工具测试互信设置是否成功。
01
[gpadmin@mdw config]$ gpssh -f hostlist -e 'date'
03
[mdw-std] Tue Feb 16 17:50:16 CST 2016
05
[ mdw] Tue Feb 16 17:50:16 CST 2016
07
[ sdw2] Tue Feb 16 17:50:16 CST 2016
09
[ sdw1] Tue Feb 16 17:50:16 CST 2016
11
[ sdw3] Tue Feb 16 17:50:16 CST 2016
11.在Master节点通过gpinitstandby命令添加Standby。
01
[gpadmin@mdw config]$ gpinitstandby -s mdw-std
03
The filespace locations on the master must be mapped to
04
locations on the standby. These locations must be empty on the
05
standby master host. The default provided is the location of
06
the filespace on the master (except if the master and the
07
standby are hosted on the same node or host). In most cases the
10
Enter standby filespace location for filespace fspc1 (default: /gpdb/gpdata/fspc_master/gpseg-1):
这里会提示Master节点中记录的文件空间的目录,默认和Master相同,这里可以使用和Master节点不同的路径,建议使用一样的路径。需要注意,在这个步骤之前,一定要创建和Master一样的数据库初始化目录,这里是/gpdb/gpdata/master目录,否则会报这个路径找不到的错误。如果文件空间直接使用默认路径,直接回车就可以。
02
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Validating environment and parameters for standby initialization...
03
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Checking for filespace directory /gpdb/gpdata/master/gpseg-1 on mdw-std
04
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Checking for filespace directory /gpdb/gpdata/fspc_master/gpseg-1 on mdw-std
05
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------
06
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master initialization parameters
07
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------
08
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum master hostname = mdw
09
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum master data directory = /gpdb/gpdata/master/gpseg-1
10
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum master port = 5432
11
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master hostname = mdw-std
12
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master port = 5432
13
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum standby master data directory = /gpdb/gpdata/master/gpseg-1
14
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Greenplum update system catalog = On
15
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------
16
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:- Filespace locations
17
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:------------------------------------------------------
18
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-pg_system -> /gpdb/gpdata/master/gpseg-1
19
20160216:17:54:26:011967 gpinitstandby:mdw:gpadmin-[INFO]:-fspc1 -> /gpdb/gpdata/fspc_master/gpseg-1
20
Do you want to continue with standby master initialization? Yy|Nn (default=N):
输入Y确认搭建Standby。
02
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Syncing Greenplum Database extensions to standby
03
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-The packages on mdw-std are consistent.
04
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Adding standby master to catalog...
05
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Database catalog updated successfully.
06
20160216:17:54:56:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Updating pg_hba.conf file...
07
20160216:17:55:02:011967 gpinitstandby:mdw:gpadmin-[INFO]:-pg_hba.conf files updated successfully.
08
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Updating filespace flat files...
09
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Filespace flat file updated successfully.
10
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Starting standby master
11
20160216:17:55:04:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Checking if standby master is running on host: mdw-std in directory: /gpdb/gpdata/master/gpseg-1
12
20160216:17:55:05:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Cleaning up pg_hba.conf backup files...
13
20160216:17:55:11:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Backup files of pg_hba.conf cleaned up successfully.
14
20160216:17:55:11:011967 gpinitstandby:mdw:gpadmin-[INFO]:-Successfully created standby master on mdw-std
搭建完成,可以使用gpstate命令查看Standby的状态。
01
[gpadmin@mdw config]$ gpstate -s
02
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-Starting gpstate with args: -s
03
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.3.6.2 build 1'
04
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-master Greenplum Version: 'PostgreSQL 8.2.15 (Greenplum Database 4.3.6.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Nov 12 2015 23:50:28'
05
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-Obtaining Segment details from master...
06
20160216:17:55:37:012095 gpstate:mdw:gpadmin-[INFO]:-Gathering data from segments...
08
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-----------------------------------------------------
09
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:--Master Configuration & Status
10
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-----------------------------------------------------
11
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master host = mdw
12
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master postgres process ID = 2474
13
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master data directory = /gpdb/gpdata/master/gpseg-1
14
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master port = 5432
15
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master current role = dispatch
16
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Greenplum initsystem version = 4.3.6.2 build 1
17
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Greenplum current version = PostgreSQL 8.2.15 (Greenplum Database 4.3.6.2 build 1) on x86_64-unknown-linux-gnu, compiled by GCC gcc (GCC) 4.4.2 compiled on Nov 12 2015 23:50:28
18
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Postgres version = 8.2.15
19
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Master standby = mdw-std
20
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Standby master state = Standby host passive
21
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-----------------------------------------------------
22
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-Segment Instance Status Report
23
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-----------------------------------------------------
24
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Segment Info
25
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Hostname = sdw1
26
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Address = sdw1
27
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Datadir = /gpdb/gpdata/primary/gpseg0
28
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Port = 40000
29
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Status
30
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- PID = 2378
31
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Configuration reports status as = Up
32
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Database status = Up
33
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-----------------------------------------------------
34
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Segment Info
35
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Hostname = sdw2
36
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Address = sdw2
37
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Datadir = /gpdb/gpdata/primary/gpseg1
38
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Port = 40000
39
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Status
40
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- PID = 2362
41
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Configuration reports status as = Up
42
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Database status = Up
43
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:-----------------------------------------------------
44
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Segment Info
45
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Hostname = sdw3
46
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Address = sdw3
47
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Datadir = /gpdb/gpdata/primary/gpseg2
48
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Port = 40000
49
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Status
50
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- PID = 2384
51
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Configuration reports status as = Up
52
20160216:17:55:38:012095 gpstate:mdw:gpadmin-[INFO]:- Database status = Up
也可以在Standby节点的主机上查看启动的进程来查看Standby的状态。
1
[gpadmin@mdw-std gpdata]$ ps -ef | grep gpadmin
2
gpadmin 4362 1 0 17:55 ? 00:00:00 /gpdb/app/bin/postgres -D /gpdb/gpdata/master/gpseg-1 -p 5432 -b 5 -z 3 --silent-mode=true -i -M master -C -1 -x 0 -y -E
3
gpadmin 4378 4362 0 17:55 ? 00:00:00 postgres: port 5432, master logger process
4
gpadmin 4379 4362 0 17:55 ? 00:00:00 postgres: port 5432, startup process recovering 000000010000000000000002
5
gpadmin 4390 4362 0 17:55 ? 00:00:00 postgres: port 5432, wal receiver process
第一个进程是Standby的启动进程也就是守护进程,第二个进程是日志传输进程,第三个进程是数据同步进程也就是数据恢复进程,第四个进程是Standby与Master之间的通信检测进程,相当于心跳,以上进程的作用均是个人的猜测,有可能不准。
Standby搭建完成,即可在数据库中查询到Standby的信息。
01
[gpadmin@mdw config]$ psql -d dbdream
05
dbdream=
# select * from gp_segment_configuration;
06
dbid | content | role | preferred_role | mode | status | port | hostname | address | replication_port | san_mounts
07
------+---------+------+----------------+------+--------+-------+----------+---------+------------------+--------
08
1 | -1 | p | p | s | u | 5432 | mdw | mdw | |
09
2 | 0 | p | p | s | u | 40000 | sdw1 | sdw1 | |
10
3 | 1 | p | p | s | u | 40000 | sdw2 | sdw2 | |
11
4 | 2 | p | p | s | u | 40000 | sdw3 | sdw3 | |
12
5 | -1 | m | m | s | u | 5432 | mdw-std | mdw-std | |