【GP5安装配置】 Greenplum5.23 安装手记

1. 修改系统参数（共享内存，磁盘调度算法，网络参数，用户资源等）
2. 关闭防火墙
3. 配置host
4. 创建安装目录
5. 解压安卓，或rpm安装master , 配置 host_all ,host_seg
6. gpssh-exkeys 打通各个节点
7. gpseginstall 分发安装介质到各个节点
8. gpcheck 检查参数，不合理再修改

2.1 非常规安装法(先安装，再改参数)

1. 关闭防火墙
2 配置host
3. 创建安装目录
4. 解压安卓，或rpm安装master , 配置 host_all ,host_seg
5. gpssh-exkeys 打通各个节点
6. gpseginstall 分发安装介质到各个节点
8. gpcheck 检查参数，不合理再修改
8. 修改系统参数（共享内存，磁盘调度算法，网络参数，用户资源等）（先不改参数，最后根据安装校验结果再改）

3. 系统参数修改清单

相关参数具体数值，需要参考官方文档
《Greenplum5.0 最佳实践》系统参数：https://yq.aliyun.com/articles/225816?spm=a2c4e.11155435.0.0.a9756e1eIiHSoH

3.1 /etc/host

[root@mdw ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
172.28.25.204 mdw
172.28.25.205 sdw1
172.28.25.206 sdw2

3.2 /etc/sysctl.conf

/etc/sysctl.conf 内核参数可参考 /opt/greenplum-db/./etc/gpcheck.cnf
官方链接：https://gpdb.docs.pivotal.io/5230/install_guide/prep_os_install_gpdb.html#topic3

[root@mdw ~]# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.
#
# Use '/sbin/sysctl -a' to list all possible parameters.

# Controls IP packet forwarding
# net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
# net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
# kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
# kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the default maxmimum size of a mesage queue
# kernel.msgmnb = 65536

# Controls the maximum size of a message, in bytes
# kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
# kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
# kernel.shmall = 4294967296

net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.all.accept_redirects=0

kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 500 1024000 200 4096
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 10000 65535
net.core.netdev_max_backlog = 10000
vm.overcommit_memory = 2

将原参数文件清空，重写上述内容，不同的系统参数会有出入，正确的做法是，按照官方推荐参数一一比对做修改，然后再通过gpscp 命令将参数文件同步到各个子节点机器上。

[root@mdw greenplum523]# echo > /etc/sysctl.conf
[root@mdw greenplum523]# cat /etc/sysctl.conf
[root@mdw greenplum523]# vi  /etc/sysctl.conf

3.2.1 /etc/sysctl.conf 原文件内容


[root@sdw1 ~]# cat /etc/sysctl.conf
# Kernel sysctl configuration file for Red Hat Linux
#
# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and
# sysctl.conf(5) for more details.
#
# Use '/sbin/sysctl -a' to list all possible parameters.

# Controls IP packet forwarding
net.ipv4.ip_forward = 0

# Controls source route verification
net.ipv4.conf.default.rp_filter = 1

# Do not accept source routing
net.ipv4.conf.default.accept_source_route = 0

# Controls the System Request debugging functionality of the kernel
kernel.sysrq = 0

# Controls whether core dumps will append the PID to the core filename.
# Useful for debugging multi-threaded applications.
kernel.core_uses_pid = 1

# Controls the use of TCP syncookies
net.ipv4.tcp_syncookies = 1

# Controls the default maxmimum size of a mesage queue
kernel.msgmnb = 65536

# Controls the maximum size of a message, in bytes
kernel.msgmax = 65536

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736

# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.all.accept_redirects=0

3.3 /etc/security/limits.conf

[root@mdw ~]# cat /etc/security/limits.conf
# 前文省略 ... 追加以下内容
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072

3.4 /etc/security/limits.d/90-nproc.conf

[root@mdw ~]# cat /etc/security/limits.d/90-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

*          soft    nproc     131072
root       soft    nproc     unlimited

3.5 iptables 关闭防火墙

[root@mdw ~]# service iptables status
iptables: Firewall is not running.

3.6 blockdev

[root@mdw ~]# lsblk
NAME                        MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0                          11:0    1 1024M  0 rom
sda                           8:0    0   50G  0 disk
├─sda1                        8:1    0  200M  0 part /boot
└─sda2                        8:2    0 49.8G  0 part
  ├─VolGroup-lv_swap (dm-0) 253:0    0    4G  0 lvm  [SWAP]
  └─VolGroup-lv_root (dm-1) 253:1    0 45.8G  0 lvm  /
[root@mdw ~]# /sbin/blockdev --setra 16384 /dev/sda
[root@mdw ~]# /sbin/blockdev --getra /dev/sda
16384

例如：

[root@mdw ~]# /sbin/blockdev --setra 16384 /dev/sr0
[root@mdw ~]# /sbin/blockdev --setra 16384 /dev/sda
[root@mdw ~]# /sbin/blockdev --getra /dev/sda
16384

3.7 scheduler

[root@mdw ~]# more  /sys/block/sda/queue/scheduler
noop anticipatory deadline [cfq]
[root@mdw ~]# echo deadline > /sys/block/sda/queue/scheduler
[root@mdw ~]# echo deadline > /sys/block/sr0/queue/scheduler

3.8 检查字符集

echo $LANG

3.9 更改文件系统xfs

# 查看文件系统
du -T 
# 修改xfs

本次安装未做修改，影响未知
xfs和ext 对比：https://github.com/digoal/blog/blob/master/201610/20161002_02.md

4.安装Greenplum

安装分为三步
1. 安装master
2. 安装节点
3. 初始化集群（创建segment,mirror ,schema 等）

4.1 安装master

4.1.1 准备安装文件，安装目录

# 解压
mv greenplum-db-5.23.0-rhel6-x86_64.zip /opt/
cd /opt/
unzip greenplum-db-5.23.0-rhel6-x86_64.zip
mkdir greenplum523

4.1.2 执行安装命令

./greenplum-db-5.23.0-rhel6-x86_64.bin

# 查看安装目录，设置环境变量
cd greenplum523/
source greenplum_path.sh

4.1.3 准备gp集群host文件

vi host_all
mdw
sdw1
sdw2
vi host_seg
sdw1
sdw2

4.1.4 集群互信，免密登陆

[root@mdw greenplum523]# gpssh-exkeys -f host_all
[STEP 1 of 5] create local ID and authorize on local host
  ... /root/.ssh/id_rsa file exists ... key generation skipped

[STEP 2 of 5] keyscan all hosts and update known_hosts file

[STEP 3 of 5] authorize current user on remote hosts
  ... send to sdw1
  ... send to sdw2

[STEP 4 of 5] determine common authentication file content

[STEP 5 of 5] copy authentication files to all remote hosts
  ... finished key exchange with sdw1
  ... finished key exchange with sdw2

[INFO] completed successfully
[root@mdw greenplum523]#