Pivotal Greenplum 6.4安装

最新推荐文章于 2024-08-21 12:04:21 发布

AtlanSI

最新推荐文章于 2024-08-21 12:04:21 发布

阅读量2.4k

点赞数 1

文章标签： greenplum 数据库

本文链接：https://blog.csdn.net/AtlanSI/article/details/104423473

版权

参考连接：https://gpdb.docs.pivotal.io/6-4/install_guide/init_gpdb.html
参考连接：https://gpdb.docs.pivotal.io/6-4/admin_guide/highavail/topics/g-enabling-master-mirroring.html

1. 兼容性

1.1系统兼容性

系统名称	版本	备注
RHEL	7.x	7.3之前版本会有cgroup 内核错误不建议采用（大并发会导致数据库挂起）
RHEL	6.x	如使用RHEL6系列，请将内核升级至2.6.32-696以上
Centos	7.x	7.3之前版本会有cgroup 内核错误不建议采用（大并发会导致数据库挂起）
Centos	6.x	如使用Centos6系列，请将内核升级至2.6.32-696以上
Ubuntu	18.04 TLS

1.2硬件要求

项目	要求
最低CPU	任何x86_64兼容的CPU
最小内存	16GB
磁盘要求	1.单个主机150M用于Greenplum安装 2.每个Segment实例大约300M元数据
网络要求	10千兆网络，多个接口进行NIC绑定，greenplum支持ipv6 和ipv4
文件系统	只支持XFS

2.环境准备

2.1主机配置

主机名	IP地址	CPU	内存	磁盘	系统
master	10.10.10.177	vCpu8	8G	250G	Centos7.3
segment1	10.10.10.183	vCpu8	8G	250G	Centos7.3
segment2	10.10.10.184	vCpu8	8G	250G	Centos7.3

2.1 关闭Selinux和iptables

[root@master ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
[root@master ~]# systemctl stop firewalld
[root@master ~]# systemctl disable firewalld
[root@master ~]# setenforce 0
[root@master ~]# reboot

[root@segment1 ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
[root@segment1 ~]# systemctl stop firewalld
[root@segment1 ~]# systemctl disable firewalld
[root@segment1 ~]# setenforce 0
[root@segment1 ~]# reboot

[root@segment2 ~]# sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config
[root@segment2 ~]# systemctl stop firewalld
[root@segment2 ~]# systemctl disable firewalld
[root@segment2 ~]# reboot

2.2变更系统环境变量

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

[root@master ~]# cat /etc/sysctl.conf 
# System default settings live in /usr/lib/sysctl.d/00-system.conf.
# To override those settings, enter new settings here, or in an /etc/sysctl.d/<name>.conf file
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
#
#
# kernel.shmall = _PHYS_PAGES / 2 # See Shared Memory Pages
 kernel.shmall = 1001395
#  kernel.shmmax = kernel.shmall * PAGE_SIZE 
kernel.shmmax = 4101713920
 kernel.shmmni = 4096
 vm.overcommit_memory = 2 
# See Segment Host Memory
 vm.overcommit_ratio = 95 
# See Segment Host Memory
#
 net.ipv4.ip_local_port_range = 10000 65535 
# See Port Settings
 kernel.sem = 500 2048000 200 40960
 kernel.sysrq = 1
 kernel.core_uses_pid = 1
 kernel.msgmnb = 65536
 kernel.msgmax = 65536
 kernel.msgmni = 2048
 net.ipv4.tcp_syncookies = 1
 net.ipv4.conf.default.accept_source_route = 0
 net.ipv4.tcp_max_syn_backlog = 4096
 net.ipv4.conf.all.arp_filter = 1
 net.core.netdev_max_backlog = 10000
 net.core.rmem_max = 2097152
 net.core.wmem_max = 2097152
 vm.swappiness = 10
 vm.zone_reclaim_mode = 0
 vm.dirty_expire_centisecs = 500
 vm.dirty_writeback_centisecs = 100
 vm.dirty_background_ratio = 3 
# See System Memory
 vm.dirty_ratio = 10
# vm.dirty_background_bytes = 1610612736
# vm.dirty_bytes = 4294967296

使用以下命令计算是你主机得参数值
kernerl.shmall ： echo $(expr $(getconf _PHYS_PAGES) / 2)
kernel.shmmax : echo $(expr $(getconf _PHYS_PAGES) / 2 \* $(getconf PAGE_SIZE))

对于64G内存以上得主机建议以下参数
vm.dirty_background_ratio = 0
vm.dirty_ratio = 0
vm.dirty_background_bytes = 1610612736  
# 1.5GB
vm.dirty_bytes = 4294967296
 # 4GB
执行次命令设置 vm.min_free_kbytes (大内存时使用）
awk 'BEGIN {OFMT = "%.0f";} /MemTotal/ {print "vm.min_free_kbytes =", $2 * .03;}' /proc/meminfo >> /etc/sysctl.conf


64G内存以下得主机建议以下参数
移除以下参数
vm.dirty_background_bytes
vm.dirty_bytes
设置以下得参数
vm.dirty_background_ratio = 3
vm.dirty_ratio = 10

2.3 系统资源限制

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

在 /etc/security/limits.d/20-nproc.conf文件后追加如下内容

* soft nofile 524288
* hard nofile 524288
* soft nproc 131072
* hard nproc 131072

效果如下

[root@master ~]# cat /etc/security/limits.d/20-nproc.conf
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

*          soft    nproc     4096
root       soft    nproc     unlimited
* soft nofile 524288
* hard nofile 524288
* soft nproc 131072
* hard nproc 131072

使用 /etc/security/limits.d/20-nproc.conf 覆盖/etc/security/limits.conf

[root@master ~]# cp /etc/security/limits.d/20-nproc.conf /etc/security/limits.conf  
cp: overwrite ‘/etc/security/limits.conf’? y

重启ssh会话执行如下命令查看返回是否为131072

[root@master ~]# ulimit -u
131072

2.4XFS挂载选项

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

centos或RHEL挂载使用如下参数

rw,nodev,noatime,nobarrier,inode64
Ubuntu参数如下
rw,nodev,noatime,inode64

Centos /etc/fstab文件示例

/dev/data /data xfs nodev,noatime,nobarrier,inode64 0 0

2.5磁盘I/O设置

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置
示例

[root@master ~]# /sbin/blockdev --setra 16384 /dev/sdb

将盘符换成自己的
将结果追加进/etc/rc.d/rc.local

[root@master ~]# echo "/sbin/blockdev --setra 16384 /dev/sdb" >> /etc/rc.d/rc.local

2.6磁盘调度设置

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

[root@master ~]# grubby --update-kernel = ALL --args =“ elevator = deadline”
[root@master ~]# grubby --info=ALL
index=0
kernel=/boot/vmlinuz-3.10.0-327.el7.x86_64
args="ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet LANG=en_US.UTF-8 transparent_hugepage=never"
root=/dev/mapper/centos-root
initrd=/boot/initramfs-3.10.0-327.el7.x86_64.img
title=CentOS Linux (3.10.0-327.el7.x86_64) 7 (Core)
index=1
kernel=/boot/vmlinuz-0-rescue-a603815b1a9344b2801a77066196d5bb
args="ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet transparent_hugepage=never"
root=/dev/mapper/centos-root
initrd=/boot/initramfs-0-rescue-a603815b1a9344b2801a77066196d5bb.img
title=CentOS Linux (0-rescue-a603815b1a9344b2801a77066196d5bb) 7 (Core)
index=2
non linux entry

2.7禁用透明大页面THP

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

[root@master ~]# grubby --update-kernel=ALL --args="transparent_hugepage=never"

2.8禁用IPC删除

Master Host执行
禁用删除IPC。在以下位置设置此参数 /etc/systemd/logind.conf 在Greenplum数据库主机系统上。

更改/etc/systemd/logind.con
将RemoveIPC=yes改为RemoveIPC=no
修改后执行下面命令

[root@master ~]# systemctl restart  systemd-logind

2.9SSH连接阈值

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

从/etc/ssh/sshd_config后/etc/sshd_config中修改如下参数
MaxStartups 200
MaxSessions 200

如果MaxStartups使用start:rate:full语法将值修改为如下
Max Startups 10:30:200
MaxSessions 200

[root@master ~]# systemctl restart sshd

2.10NTP校时

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

自己根据环境设置NTP校时

2.11 创建Greenplum管理用户

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

[root@master ~] groupadd gpadmin 
[root@master ~] useradd gpadmin -r -m -g gpadmin 
[root@master ~] passwd gpadmin 
[root@master ~] su gpadmin
[root@master ~] ssh-keygen -t rsa -b 4096  ##直接回车跳过所有

授予gpadmin sudo 权限

修改/etc/sudoers
移除%wheel ALL=(ALL) NOPASSWD: ALL得注释

[root@master ~] usermod -aG wheel gpadmin

2.12 更改本地映射

注意每台主机都要设置
注意每台主机都要设置
注意每台主机都要设置

将上面所属得主机名加入/etc/hosts
主机将主机名改成和hosts文件中对应得名字

[root@master ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
10.10.10.183 segment1 segment1-1
10.10.10.184 segment2 segment2-1
10.10.10.177 master
[root@master ~]# reboot

3安装Greenplum

3.1安装Greenplum软件

注意每台主机都要用gpadmin用户设置
注意每台主机都要gpadmin用户设置
注意每台主机都要gpadmin用户设置

[gpadmin@master ~]$ cd
[gpadmin@master ~]$ wget http://47.94.143.103/greenplum-db-6.4.0-rhel7-x86_64.rpm
#等待下载完成
[gpadmin@master ~]$ sudo yum install ./greenplum-db-6.4.0-rhel7-x86_64.rpm
[gpadmin@master ~]$ sudo chown -R gpadmin:gpadmin /usr/local/greenplum*

3.2SSH免密登录

*Master Host gpadmin用户操作

[gpadmin@master ~]$ source /usr/local/greenplum-db-<version>/greenplum_path.sh
[gpadmin@master ~]$ ssh-copy-id segment1
[gpadmin@master ~]$ ssh-copy-id segment2

创建一个名为 hostfile_exkeys为计算机配置了Greenplum系统中每个主机（主机，备用主机和段主机）的主机名和主机地址（接口名称）。确保没有空白行或多余的空格。检查 / etc/hosts文件在您的系统上，以获取要用于您的环境的正确主机名。

例如

[gpadmin@master ~]$ cat hostfile_exkeys
master
segment1
segment2
[gpadmin@master ~]$ gpssh-exkeys -f hostfile_exkeys
[STEP 1 of 5] create local ID and authorize on local host
  ... /home/gpadmin/.ssh/id_rsa file exists ... key generation skipped

[STEP 2 of 5] keyscan all hosts and update known_hosts file

[STEP 3 of 5] retrieving credentials from remote hosts
  ... send to segment1
  ... send to segment2

[STEP 4 of 5] determine common authentication file content

[STEP 5 of 5] copy authentication files to all remote hosts
  ... finished key exchange with segment1
  ... finished key exchange with segment2

[INFO] completed successfully
[gpadmin@master ~]$ echo "source /usr/local/greenplum-db/greenplum_path.sh" >> .bashrc

3.3检验安装

Master Host gpadmin用户设置

[gpadmin@master ~]$ gpssh -f hostfile_exkeys -e 'ls -l /usr/local/greenplum-db-6.4.0'
[  master] ls -l /usr/local/greenplum-db-6.4.0
[  master] total 160
[  master] drwxr-xr-x 7 root root   4096 Feb 17 12:45 bin
[  master] drwxr-xr-x 3 root root     21 Feb 17 12:45 docs
[  master] drwxr-xr-x 2 root root     58 Feb 17 12:45 etc
[  master] drwxr-xr-x 3 root root     19 Feb 17 12:45 ext
[  master] -rwxr-xr-x 1 root root    783 Feb 17 12:45 greenplum_path.sh
[  master] drwxr-xr-x 4 root root   4096 Feb 17 12:45 include
[  master] drwxr-xr-x 6 root root   4096 Feb 17 12:45 lib
[  master] drwxr-xr-x 2 root root     20 Feb 17 12:45 libexec
[  master] -rw-r--r-- 1 root root 135617 Feb 12 05:02 open_source_license_pivotal_greenplum.txt
[  master] drwxr-xr-x 7 root root     87 Feb 17 12:45 pxf
[  master] drwxr-xr-x 2 root root   4096 Feb 17 12:45 sbin
[  master] drwxr-xr-x 5 root root     49 Feb 17 12:45 share
[segment1] ls -l /usr/local/greenplum-db-6.4.0
[segment1] total 160
[segment1] drwxr-xr-x 7 gpadmin gpadmin   4096 Feb 18 14:05 bin
[segment1] drwxr-xr-x 3 gpadmin gpadmin     21 Feb 18 14:05 docs
[segment1] drwxr-xr-x 2 gpadmin gpadmin     58 Feb 18 14:05 etc
[segment1] drwxr-xr-x 3 gpadmin gpadmin     19 Feb 18 14:05 ext
[segment1] -rwxr-xr-x 1 gpadmin gpadmin    783 Feb 18 14:05 greenplum_path.sh
[segment1] drwxr-xr-x 4 gpadmin gpadmin   4096 Feb 18 14:05 include
[segment1] drwxr-xr-x 6 gpadmin gpadmin   4096 Feb 18 14:05 lib
[segment1] drwxr-xr-x 2 gpadmin gpadmin     20 Feb 18 14:05 libexec
[segment1] -rw-r--r-- 1 gpadmin gpadmin 135617 Feb 12 05:02 open_source_license_pivotal_greenplum.txt
[segment1] drwxr-xr-x 7 gpadmin gpadmin     87 Feb 18 14:05 pxf
[segment1] drwxr-xr-x 2 gpadmin gpadmin   4096 Feb 18 14:05 sbin
[segment1] drwxr-xr-x 5 gpadmin gpadmin     49 Feb 18 14:05 share
[segment2] ls -l /usr/local/greenplum-db-6.4.0
[segment2] total 160
[segment2] drwxr-xr-x 7 gpadmin gpadmin   4096 Feb 18 14:05 bin
[segment2] drwxr-xr-x 3 gpadmin gpadmin     21 Feb 18 14:05 docs
[segment2] drwxr-xr-x 2 gpadmin gpadmin     58 Feb 18 14:05 etc
[segment2] drwxr-xr-x 3 gpadmin gpadmin     19 Feb 18 14:05 ext
[segment2] -rwxr-xr-x 1 gpadmin gpadmin    783 Feb 18 14:05 greenplum_path.sh
[segment2] drwxr-xr-x 4 gpadmin gpadmin   4096 Feb 18 14:05 include
[segment2] drwxr-xr-x 6 gpadmin gpadmin   4096 Feb 18 14:05 lib
[segment2] drwxr-xr-x 2 gpadmin gpadmin     20 Feb 18 14:05 libexec
[segment2] -rw-r--r-- 1 gpadmin gpadmin 135617 Feb 12 05:02 open_source_license_pivotal_greenplum.txt
[segment2] drwxr-xr-x 7 gpadmin gpadmin     87 Feb 18 14:05 pxf
[segment2] drwxr-xr-x 2 gpadmin gpadmin   4096 Feb 18 14:05 sbin
[segment2] drwxr-xr-x 5 gpadmin gpadmin     49 Feb 18 14:05 share

bin-此目录包含Greenplum数据库管理实用程序。该目录还包含PostgreSQL客户端和服务器程序，其中大多数也在Greenplum数据库中使用。

docs/cli_help-此目录包含Greenplum数据库命令行实用程序的帮助文件。

docs/cli_help/gpconfigs-此目录包含示例 gpinitsystem 在安装和初始化Greenplum数据库系统时可以修改和使用的配置文件和主机文件。

ext —一些Greenplum Database实用程序使用的捆绑程序（例如Python）。

include — Greenplum数据库的C头文件。

lib — Greenplum数据库和PostgreSQL库文件。

sbin —支持/内部脚本和程序。

share — Greenplum数据库的共享文件。

3.4创建数据库存储

Master Host root用户设置

[root@master ~]# mkdir -p /data/master
[root@master ~]# chown gpadmin:gpadmin /data/master

Master Host root用户设置

创建hostfile_gpssh_segonly文件对于每个段主机，该文件应该只有一台机器配置的主机名。

[root@master gpadmin]# cat hostfile_gpssh_segonly
segment1
segment2
[root@master gpadmin]# source /usr/local/greenplum-db/greenplum_path.sh
[root@master gpadmin]# gpssh -f hostfile_gpssh_segonly -e 'mkdir -p /data/primary'
[root@master gpadmin]# gpssh -f hostfile_gpssh_segonly -e 'mkdir -p /data/mirror'
[root@master gpadmin]# gpssh -f hostfile_gpssh_segonly -e 'chown -R gpadmin /data/*'

4.验证系统

4.1 磁盘验证和内存验证

每台segment Host root用户设置

[root@segment1 ~]# mkdir /data1
[root@segment1 ~]# mkdir /data2
[root@segment1 ~]# chown gpadmin.gpadmin -R /data1
[root@segment1 ~]# chown gpadmin.gpadmin -R /data2

Master Host root用户设置

创建一个名为的主机文件 hostfile_gpcheckperf每个段主机具有一个主机名。不包括主控主机

[gpadmin@master ~]$ cat  hostfile_gpcheckperf
segment1
segment2
[gpadmin@master ~]$ gpcheckperf -f hostfile_gpcheckperf -r ds -D \
  -d /data1/primary -d  /data2/primary \
  -d /data1/mirror -d  /data2/mirror
要等待一会才能看到测试结果

5初始化数据库

5.1初始化数据

Master Host gpadmin用户设置

创建一个名为 hostfile_gpinitsystem。在此文件中，添加段主机接口的主机地址名称，每行一个名称，没有多余的行或空格。

[gpadmin@master ~]$ cat hostfile_gpinitsystem
segment1
segment2
[gpadmin@master ~]$ cp $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/gpconfigs/gpinitsystem_config
[gpadmin@master ~]$ cat /home/gpadmin/gpconfigs/gpinitsystem_config
# FILE NAME: gpinitsystem_config

# Configuration file needed by the gpinitsystem

################################################
#### REQUIRED PARAMETERS
################################################

#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="Greenplum Data Platform"

#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg

#### Base number by which primary segment port numbers 
#### are calculated.
PORT_BASE=6000
### 这里需要和sysctl.conf中net.ipv4.ip_local_port_range对比 是否在ip_local_port_range范围内
#### File system location(s) where primary segment data directories 
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in 
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)

#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=master

#### File system location where the master data directory 
#### will be created.
MASTER_DIRECTORY=/data/master

#### Port number for the master instance. 
MASTER_PORT=5432

#### Shell utility used to connect to remote hosts.
TRUSTED_SHELL=ssh

#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8

#### Default server-side character set encoding.
ENCODING=UNICODE


####这里往下都是镜像segment设置
################################################
#### OPTIONAL MIRROR PARAMETERS
################################################

#### Base number by which mirror segment port numbers 
#### are calculated.
MIRROR_PORT_BASE=7000

#### File system location(s) where mirror segment data directories 
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the 
#### DATA_DIRECTORY parameter.
declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror)


################################################
#### OTHER OPTIONAL PARAMETERS
################################################

#### Create a database of this name after initialization.
#DATABASE_NAME=name_of_database

#### Specify the location of the host address file here instead of
#### with the the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem

初始化程序

运行以下命令，以引用初始化配置文件的路径和文件名（gpinitsystem_config）和主机文件（hostfile_gpinitsystem）。
[gpadmin@master ~]$ gpinitsystem -c gpconfigs/gpinitsystem_config -h gpconfigs/hostfile_gpinitsystem
对于完全冗余的系统（具有备用主服务器和扩展镜像配置），请包括-s 和 -S选项。例如：
[gpadmin@master ~]$  gpinitsystem -c gpconfigs/gpinitsystem_config -h gpconfigs/hostfile_gpinitsystem  -s standby_master_hostname -S
安装完成能看到
=> Greenplum Database instance successfully created.

5.2设置数据库时区

Master Host gpadmin用户设置

[gpadmin@master ~]$ gpconfig -s TimeZone Asia/Shanghai

更改时区后，您必须重新启动Greenplum数据库。命令gpstop -ra重新启动Greenplum数据库。目录视图 pg_timezone_names 提供Greenplum数据库时区信息。

5.3环境配置

Master Host gpadmin用户设置

[gpadmin@master ~]$ echo "export MASTER_DATA_DIRECTORY=/data/master/gpseg-1" >> .bashrc
[gpadmin@master ~]$ echo "export PGPORT=5432" >> .bashrc
[gpadmin@master ~]$ echo "export PGUSER=gpadmin" >> .bashrc
[gpadmin@master ~]$ echo "export PGDATABASE=default_login_database_name" >> .bashrc 
[gpadmin@master ~]$ echo "export LD_PRELOAD=/lib64/libz.so.1 ps" >> .bashrc
[gpadmin@master ~]$ source ~/.bashrc

5.4登录数据库

[gpadmin@master ~]$ psql
psql (9.4.24)
Type "help" for help.

test=# \l
                               List of databases
   Name    |  Owner  | Encoding |  Collate   |   Ctype    |  Access privileges  
-----------+---------+----------+------------+------------+---------------------
 postgres  | gpadmin | UTF8     | en_US.utf8 | en_US.utf8 | 
 template0 | gpadmin | UTF8     | en_US.utf8 | en_US.utf8 | =c/gpadmin         +
           |         |          |            |            | gpadmin=CTc/gpadmin
 template1 | gpadmin | UTF8     | en_US.utf8 | en_US.utf8 | =c/gpadmin         +
           |         |          |            |            | gpadmin=CTc/gpadmin
 test      | gpadmin | UTF8     | en_US.utf8 | en_US.utf8 | 
(4 rows)

test=#