Centos7上安装Greenplum6.18.1集群

1. 服务器分布情况

hostname	安装服务	备注
bigdata001	master、segment
bigdata002	segment
bigdata003	master、segment

2. 关闭selinux和防火墙(3台)

1. 修改/etc/selinux/config，修改内容如下：

SELINUX=disabled

然后重启服务器

2. 关闭防火墙

[root@bigdata001 ~]#
[root@bigdata001 ~]# systemctl stop firewalld.service
[root@bigdata001 ~]# systemctl disable firewalld.service
[root@bigdata001 ~]#

3. 设置linux系统参数(3台)

3.1 修改/etc/sysctl.conf

添加内容如下，有注释的按注释来，没注释的按默认值即可

kernel.shmall = 2033412    # echo $(expr $(getconf _PHYS_PAGES) / 2)
kernel.shmmax = 8328855552    # echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE))
kernel.shmmni = 4096
vm.overcommit_memory = 2    # 对于greenplum必须为2
vm.overcommit_ratio = 95    # 默认为50

net.ipv4.ip_local_port_range = 10000 65535    # 不要设置greenplum的PORT_BASE和MIRROR_PORT_BASE在这个范围内
kernel.sem = 500 2048000 200 4096
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.swappiness = 10
vm.zone_reclaim_mode = 0
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.dirty_background_ratio = 3 
vm.dirty_ratio = 10

• 对于64GB内存以下的服务器，删除下面两个参数

vm.dirty_background_bytes = 1610612736
vm.dirty_bytes = 4294967296

修改下面两个参数

vm.dirty_background_ratio = 3 
vm.dirty_ratio = 10

• 对于64GB内存以上的服务器，修改下面4个参数

vm.dirty_background_ratio = 0 
vm.dirty_ratio = 0
vm.dirty_background_bytes = 1610612736    # 1.5GB
vm.dirty_bytes = 4294967296    # 4GB

最后用命令sysctl -p是设置生效

3.2 修改/etc/security/limits.conf

添加内容如下：

* soft nofile 524288
* hard nofile 524288
* soft nproc 131072
* hard nproc 131072

然后退出当前会话，重新连接就会生效

4. 创建gpadmin用户(3台)

不能用root用户运行greenplum数据库服务，所以创建用户组和用户，并设置用户密码

[root@bigdata001 ~]#
[root@bigdata001 ~]# groupadd gpadmin
[root@bigdata001 ~]# useradd gpadmin -r -m -g gpadmin
[root@bigdata001 ~]# passwd gpadmin
[root@bigdata001 ~]#

设置3台服务器之间gpadmin用户的无密码访问

[root@bigdata001 ~]# 
[root@bigdata001 ~]# su gpadmin
[gpadmin@bigdata001 root]$ 
[gpadmin@bigdata001 root]$ cd ~
[gpadmin@bigdata001 ~]$
[gpadmin@bigdata001 ~]$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/gpadmin/.ssh/id_rsa): 
Created directory '/home/gpadmin/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/gpadmin/.ssh/id_rsa.
Your public key has been saved in /home/gpadmin/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:jsekKptPaHWFVV8CATwOWLRcp7SgscyZWZTFfvjaZSg gpadmin@bigdata001
The key's randomart image is:
+---[RSA 4096]----+
|    .=*=*o=o. .  |
|   o.XoB++ . o   |
|    O +++o  .    |
|       .+ .      |
|    . . So .     |
|   o . *E o o    |
|  o . o ++ o     |
| ..o . .. .      |
|  o+o            |
+----[SHA256]-----+
[gpadmin@bigdata001 ~]$ 
[gpadmin@bigdata001 ~]$ ll .ssh
总用量 8
-rw------- 1 gpadmin gpadmin 3243 11月  9 11:06 id_rsa
-rw-r--r-- 1 gpadmin gpadmin  744 11月  9 11:06 id_rsa.pub
[gpadmin@bigdata001 ~]$ 
[gpadmin@bigdata001 ~]$ cat .ssh/id_rsa.pub >> .ssh/authorized_keys
[gpadmin@bigdata001 ~]$ 

将3台服务器的id_rsa.pub的内容都放到authorized_keys中，然后将authorized_keys分发给3台服务器，最后设置authorized_keys权限为600

5. 安装greenplum(3台)

5.1 下载上传

从github上下载open-source-greenplum-db-6.18.1-rhel7-x86_64.rpm

用gpadmin用户进行上传

[gpadmin@bigdata001 ~]$ 
[gpadmin@bigdata001 ~]$ ll
总用量 65928
-rw-r--r-- 1 gpadmin gpadmin 67509712 11月  9 12:55 open-source-greenplum-db-6.18.1-rhel7-x86_64.rpm
[gpadmin@bigdata001 ~]$ 

5.2 安装rpm文件

切换到root用户下，安装rpm文件

[gpadmin@bigdata001 ~]$ 
[gpadmin@bigdata001 ~]$ exit
exit
[root@bigdata001 ~]# 
[root@bigdata001 ~]# cd /home/gpadmin/
[root@bigdata001 gpadmin]# 
[root@bigdata001 gpadmin]# yum install open-source-greenplum-db-6.18.1-rhel7-x86_64.rpm 
[root@bigdata001 gpadmin]#

• 这里yum不能使用–installroot参数指定安装目录，否则会报异常：Cannot find a valid baseurl for repo: base/$releasever/x86_64

安装目录如下：

[root@bigdata001 gpadmin]# 
[root@bigdata001 gpadmin]# ll /usr/local
总用量 0
......省略部分......
lrwxrwxrwx   1 root root  30 11月  9 20:46 greenplum-db -> /usr/local/greenplum-db-6.18.1
drwxr-xr-x  11 root root 238 11月  9 20:46 greenplum-db-6.18.1
......省略部分......
[root@bigdata001 gpadmin]# 

将安装目录的owner改为gpadmin

[root@bigdata001 gpadmin]# 
[root@bigdata001 gpadmin]# chown -R gpadmin:gpadmin /usr/local/greenplum*
[root@bigdata001 gpadmin]# chgrp -R gpadmin /usr/local/greenplum*
[root@bigdata001 gpadmin]# 

5.3 激活greenplum的环境变量

修改/home/gpadmin/.bashrc，添加以下内容

source /usr/local/greenplum-db-6.18.1/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/usr/local/greenplum-db-6.18.1/master-data/gpseg-1

然后使bashrc文件生效

[gpadmin@bigdata001 ~]$
[gpadmin@bigdata001 ~]$ source .bashrc
[gpadmin@bigdata001 ~]$ 

5.4 创建master和segment的数据储存目录

其中master-data只需在bigdata001和bigdata003上创建

[gpadmin@bigdata001 ~]$
[gpadmin@bigdata001 ~]$ mkdir /usr/local/greenplum-db-6.18.1/master-data
[gpadmin@bigdata001 ~]$ 
[gpadmin@bigdata001 ~]$ mkdir /usr/local/greenplum-db-6.18.1/segment-primary-data
[gpadmin@bigdata001 ~]$ mkdir /usr/local/greenplum-db-6.18.1/segment-mirror-data
[gpadmin@bigdata001 ~]$ 

6. 初始化greenplum(3台)

6.1 创建segment的hosts文件

[gpadmin@bigdata001 greenplum-db-6.18.1]$ 
[gpadmin@bigdata001 greenplum-db-6.18.1]$ pwd
/usr/local/greenplum-db-6.18.1
[gpadmin@bigdata001 greenplum-db-6.18.1]$ 
[gpadmin@bigdata001 greenplum-db-6.18.1]$ cat hostfile_gpinitsystem 
bigdata001
bigdata002
bigdata003
[gpadmin@bigdata001 greenplum-db-6.18.1]$ 

6.2 创建gpinitsystem_config文件

[gpadmin@bigdata001 greenplum-db-6.18.1]$
[gpadmin@bigdata001 greenplum-db-6.18.1]$ cp /usr/local/greenplum-db-6.18.1/docs/cli_help/gpconfigs/gpinitsystem_config .
[gpadmin@bigdata001 greenplum-db-6.18.1]$ 

修改后的内容如下：

[gpadmin@bigdata001 greenplum-db-6.18.1]$ 
[gpadmin@bigdata001 greenplum-db-6.18.1]$ cat gpinitsystem_config 
# FILE NAME: gpinitsystem_config

# Configuration file needed by the gpinitsystem

################################################
#### REQUIRED PARAMETERS
################################################

#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="Greenplum Data Platform"

#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg

#### Base number by which primary segment port numbers 
#### are calculated.
PORT_BASE=6000

#### File system location(s) where primary segment data directories 
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in 
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
#declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)
declare -a DATA_DIRECTORY=(/usr/local/greenplum-db-6.18.1/segment-primary-data)


#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=bigdata001

#### File system location where the master data directory 
#### will be created.
MASTER_DIRECTORY=/usr/local/greenplum-db-6.18.1/master-data

#### Port number for the master instance. 
MASTER_PORT=5432

#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh

#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8

#### Default server-side character set encoding.
ENCODING=UNICODE

################################################
#### OPTIONAL MIRROR PARAMETERS
################################################

#### Base number by which mirror segment port numbers 
#### are calculated.
MIRROR_PORT_BASE=7000

#### File system location(s) where mirror segment data directories 
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the 
#### DATA_DIRECTORY parameter.
#declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror)
declare -a MIRROR_DATA_DIRECTORY=(/usr/local/greenplum-db-6.18.1/segment-mirror-data)

################################################
#### OTHER OPTIONAL PARAMETERS
################################################

#### Create a database of this name after initialization.
#DATABASE_NAME=name_of_database

#### Specify the location of the host address file here instead of
#### with the -h option of gpinitsystem.
MACHINE_LIST_FILE=/usr/local/greenplum-db-6.18.1/hostfile_gpinitsystem

[gpadmin@bigdata001 greenplum-db-6.18.1]$

6.3 进行初始化(bigdata001上操作)

[gpadmin@bigdata001 ~]$
[gpadmin@bigdata001 ~]$ gpinitsystem -c /usr/local/greenplum-db-6.18.1/gpinitsystem_config -s bigdata003 --mirror-mode=spread
......省略部分......
20211109:22:27:53:019233 gpinitsystem:bigdata001:gpadmin-[INFO]:-bigdata001 	7000 	bigdata001 	/usr/local/greenplum-db-6.18.1/segment-mirror-data/gpseg2 	7

Continue with Greenplum creation Yy|Nn (default=N):
> y
20211109:22:28:18:019233 gpinitsystem:bigdata001:gpadmin-[INFO]:-Building the Master instance database, please wait...
......省略部分......
20211109:22:29:06:019233 gpinitsystem:bigdata001:gpadmin-[INFO]:-To determine level of criticality
20211109:22:29:06:019233 gpinitsystem:bigdata001:gpadmin-[WARN]:-*******************************************************
20211109:22:29:06:019233 gpinitsystem:bigdata001:gpadmin-[INFO]:-Greenplum Database instance successfully created
20211109:22:29:06:019233 gpinitsystem:bigdata001:gpadmin-[INFO]:-------------------------------------------------------
20211109:22:29:06:019233 gpinitsystem:bigdata001:gpadmin-[INFO]:-To complete the environment configuration, please 
......省略部分......

到这里greenplum就初始化成功了

6.4 为greenplum的gpadmin用户设置密码(bigdata001上操作)

[gpadmin@bigdata001 ~]$ 
[gpadmin@bigdata001 ~]$ psql -d postgres
psql (9.4.24)
Type "help" for help.

postgres=# 
postgres=# alter user gpadmin encrypted password 'gpadmin123';
ALTER ROLE
postgres=# 

6.5 设置greenplum客户端远程访问(bigdata001和bigdata003上操作）

修改/usr/local/greenplum-db/master-data/gpseg-1/pg_hba.conf文件，添加以下内容

# TYPE  DATABASE        USER            ADDRESS                 METHOD
host    all     all        0.0.0.0/0                 SHA-256

运行以下命令，让配置生效

[gpadmin@bigdata001 ~]$
[gpadmin@bigdata001 ~]$ gpstop -u
20211109:23:19:38:024766 gpstop:bigdata001:gpadmin-[INFO]:-Starting gpstop with args: -u
20211109:23:19:38:024766 gpstop:bigdata001:gpadmin-[INFO]:-Gathering information and validating the environment...
20211109:23:19:38:024766 gpstop:bigdata001:gpadmin-[INFO]:-Obtaining Greenplum Master catalog information
20211109:23:19:38:024766 gpstop:bigdata001:gpadmin-[INFO]:-Obtaining Segment details from master...
20211109:23:19:38:024766 gpstop:bigdata001:gpadmin-[INFO]:-Greenplum Version: 'postgres (Greenplum Database) 6.18.1 build commit:a541d5da4f5b51668f3ccb8c238d65ea8e8000f4 Open Source'
20211109:23:19:38:024766 gpstop:bigdata001:gpadmin-[INFO]:-Signalling all postmaster processes to reload
[gpadmin@bigdata001 ~]$ 

然后就可以远程登录greenplum了

[gpadmin@bigdata001 ~]$ 
[gpadmin@bigdata001 ~]$ psql -h 192.168.8.111 -p 5432 -d postgres -U gpadmin -w gpadmin123
psql: warning: extra command-line argument "gpadmin123" ignored
psql (9.4.24)
Type "help" for help.

postgres=#