Greenplum安装

最新推荐文章于 2024-08-28 10:04:31 发布

u010364275

最新推荐文章于 2024-08-28 10:04:31 发布

阅读量1.5k

点赞数

本文链接：https://blog.csdn.net/u010364275/article/details/98725034

版权

一、安装环境介绍及配置准备工作

1.操作系统版本
centos7.4
2.Greenplum集群介绍
这里使用1个master，2个segment的集群，ip为
192.168.92.128
192.168.92.131
192.168.92.132
其中192.168.92.128 为master，其余为segment

3.修改/etc/hosts文件（所有的主机都要修改）
这里主要是为之后Greenplum能够在各个节点之间相互通信做准备
[root@node1 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.92.128 node1.hnr.com node1.hnr.com
192.168.92.131 node2.hnr.com node2.hnr.com
192.168.92.132 node3.hnr.com node3.hnr.com
(主机ip地址　　　　主机名　　 hostlist配置中的名字)
注意一定要按照这个格式写，三个参数分别代表的含义在括号中已经做了说明。

4.创建用户和用户组（所有主机都要创建）
groupadd -g 1001 gpadmin
useradd -g gpadmin -u 1001 gpadmin
passwd gpadmin

5.修改系统内核（所有主机）

[root@node1 ~]# vi /etc/sysctl.conf
kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2

[root@node1 ~]# sysctl -p（让配置生效）

6.修改文件打开限制（所有机器都要修改）：
[root@node1 ~]# cat /etc/security/limits.conf

          soft    nproc           102400

          hard    nproc           102400

          soft    nofile          102400

          hard    nofile          102400

7.安装软件依赖包（所有主机）
安装系统软件包
yum install -y curl-devel bzip2-devel python-devel openssl-devel wget perl-ExtUtils-Embed libxml2-devel openldap-devel pam pam-devel
yum install -y libtool libaio bison vim-common.x86_64 vim-enhanced.x86_64 flex net-tools.x86_64 apr-devel readline-devel libevent-devel
安装python模块包
pip install lockfile paramiko setuptools epydoc psutil psi

8.配置ld.so.conf
由于使用的是CentOS系统, 所以一定要加上/usr/local/lib和/usr/local/lib64的路径到/etc/ld.so.conf文件中，这一步尤为重要，目的是将常用的动态函数库加载到内存中，可以提高函数库的访问效率。另外，如果不配置这一步，在后面编译的过程中会遇到一些奇奇怪怪的缺少引用的错误。
修改后的文件内容类似如下：

include ld.so.conf.d/*.conf
/usr/local/lib
/usr/local/lib64
修改完成后保存，之后运行ldconfig命令

二、安装优化器GPORCA的依赖项（所有节点和主机）

1.安装cmake（3.10.2）

下载：
wget http://www.cmake.org/files/v3.10/cmake-3.10.2.tar.gz
解压：
tar xzf cmake-3.10.2.tar.gz
cd cmake-3.10.2
./configure --prefix=/usr/local/cmake
make：
make install
vi /etc/profile.d/cmake.sh

export PATH=$PATH:/usr/local/cmake/bin
source /etc/profile
最后进行验证：
cmake -version

cmake version 3.10.2
2.安装gp-xerces
下载地址：
https://github.com/greenplum-db/gp-xerces/archive/v3.1.2-p1.tar.gz
tar xf gp-xerces-3.1.2-p1.tar.gz
cd gp-xerces-3.1.2-p1
mkdir build
cd build
./configure --prefix=/usr/local
make
make install

3.安装re2c（1.0.3）
进入 http://re2c.org/install/install.html 页面下载自己需要的版本
安装re2c是由于配置ninja时需要
tar xf re2c-1.0.3.tar.gz
cd re2c-1.0.3
./configure --prefix=/usr/local
make
make install

4.安装Ninja

可以使用git下载：https://github.com/ninja-build/ninja.git
tar xf ninja-1.8.2.tar.gz
cd ninja-1.8.2
./configure.py --bootstrap
由于最终结果只是一个二进制文件ninja，之后拷贝ninja文件到/usr/bin目录即可
cp -p ninja /usr/bin/

三、安装GPORCA

1.下载地址：https://github.com/greenplum-db/gporca
2.安装GPORCA（GPDB-5.5.0对应的依赖版本，2.55.0）
tar xf gporca-2.55.0.tar.gz
cd gporca-2.55.0
cmake -GNinja -H. -Bbuild
ninja install -C build

3.查看GPDB依赖的ORCA的版本信息：
/usr/local/src/gpdb-5.5.0/depends/conanfile_orca.txt 文件
[requires]
orca/v2.55.0@gpdb/stable
4.测试
安装完成后，进入build目录，执行ctest命令进行检查
如果最后输出类似如下结果：
100% tests passed, 0 tests failed out of 119

Total Test time (real) = 195.48 sec
这说明编译成功了。

5.删除旧版的GPORCA
如果发现版本安装不对，请执行以下操作，否则不需要执行以下步骤
进入源文件目录下，执行命令
rm -rf build/*
rm -rf /usr/local/include/naucrates
rm -rf /usr/local/include/gpdbcost
rm -rf /usr/local/include/gpopt
rm -rf /usr/local/include/gpos
rm -rf /usr/local/lib/libnaucrates.so*
rm -rf /usr/local/lib/libgpdbcost.so*
rm -rf /usr/local/lib/libgpopt.so*
rm -rf /usr/local/lib/libgpos.so*

特别说明：先在master上安装所有依赖项的程序，然后通过scp命令远程复制安装包或压缩包到其他节点上逐个执行安装（这里可以将master主机上/usr/local目录下lib目录打包，复制到其他节点上）。

四、安装GPDB（选择版本5.5.0）

1.下载源代码文件
下载地址： https://github.com/greenplum-db/gpdb/archive/5.5.0.tar.gz

2.解压后编译安装
tar xf gpdb-5.5.0.tar.gz
cd gpdb-5.5.0
./configure --with-perl --with-python --with-libxml --with-gssapi --prefix=/home/gpadmin/greenplum
make
make install
chown -R gpadmin.gpadmin /home/gpadmin/greenplum

3.创建配置文件
su - gpadmin
$ cd
$ mkdir conf
$ vi ./conf/hostlist

node1.hnr.com
node2.hnr.com
node3.hnr.com
$ vi ./conf/seg_hosts

node2.hnr.com
node3.hnr.com

4.配置所有节点互通
$ source /home/gpadmin/greenplum /greenplum_path.sh
$ gpssh-exkeys -f /home/gpadmin/conf/hostlist
[STEP 1 of 5] create local ID and authorize on local host
… /home/gpadmin/.ssh/id_rsa file exists … key generation skipped

[STEP 2 of 5] keyscan all hosts and update known_hosts file

[STEP 3 of 5] authorize current user on remote hosts
… send to node2.hnr.com
… send to node3.hnr.com

[STEP 4 of 5] determine common authentication file content

[STEP 5 of 5] copy authentication files to all remote hosts
… finished key exchange with node2.hnr.com
… finished key exchange with node3.hnr.com

[INFO] completed successfully

这就说明互通配置成功，之后就可以使用下面的命令开启批量操作，如下：
注意gpssh-exkeys命令使用的时候一定要用gpadmin身份，因为这个命令会生成ssh的免密码登录的秘钥，在/home/gpadmin/.ssh这里。如果使用root身份使用gpssh-exkeys命令，那么生成的.ssh秘钥在root的home下面或者是在/home/gpadmin下面但是是root的所有者，如果之后使用gpadmin身份执行相应操作的时候就没有权限。
$ gpssh -f /home/gpadmin/conf/hostlist
=> pwd
[node1.hnr.com] /home/gpadmin
[node2.hnr.com] /home/gpadmin
[node3.hnr.com] /home/gpadmin
=> exit

5.将安装包分发到每个子节点

配置互通之后需要将master中的greenplum安装包批量复制到各个segment节点上。
$ cd /home/gpadmin /greenplum/
$ tar -zcf greenplum-centos7-5.5.tgz greenplum
$ gpscp -f /home/gpadmin/conf/hostlist greenplum-centos7-5.5.tgz =:/home/gpadmin/

如果没有意外，就批量复制成功了，可以去子节点的相应文件夹查看，之后要将tar包解压，我们使用批量操作
$ gpssh -f /home/gpadmin/conf/hostlist
=> tar xf greenplum-centos7-5.5.tgz
[node2.hnr.com]
[node1.hnr.com]
[node3.hnr.com]

五、初始化数据库

在初始化之前的的几个步骤都是做一些准备工作。
在每个节点上创建gpdata目录
mkdir /data/gpdata
chown gpadmin.gpadmin /data/gpdata

1.批量创建Greenplum数据存放目录
$ gpssh -f hostlist
=> cd /data/gpdata
[node2.hnr.com]
[node1.hnr.com]
[node3.hnr.com]
=> mkdir gpmaster gpdatap1 gpdatap2 gpdatam1 gpdatam2
[node2.hnr.com]
[node1.hnr.com]
[node3.hnr.com]
=> ll
[node2.hnr.com] total 0
[node2.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatam1
[node2.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatam2
[node2.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatap1
[node2.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatap2
[node2.hnr.com] drwxrwxr-x 2 gpadmin gpadmin 6 Mar 6 13:47 gpmaster
[node1.hnr.com] total 0
[node1.hnr.com] drwxrwxr-x 2 gpadmin gpadmin 6 Mar 6 13:47 gpdatam1
[node1.hnr.com] drwxrwxr-x 2 gpadmin gpadmin 6 Mar 6 13:47 gpdatam2
[node1.hnr.com] drwxrwxr-x 2 gpadmin gpadmin 6 Mar 6 13:47 gpdatap1
[node1.hnr.com] drwxrwxr-x 2 gpadmin gpadmin 6 Mar 6 13:47 gpdatap2
[node1.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 21 Mar 6 15:50 gpmaster
[node3.hnr.com] total 0
[node3.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatam1
[node3.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatam2
[node3.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatap1
[node3.hnr.com] drwxrwxr-x 3 gpadmin gpadmin 20 Mar 6 15:50 gpdatap2
[node3.hnr.com] drwxrwxr-x 2 gpadmin gpadmin 6 Mar 6 13:47 gpmaster
=> exit

2.配置.bash_profile环境变量（每台机器）

$ cd
$ cat .bash_profile
.bash_profile

if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

PATH= $P A T H :$ HOME/bin

export PATH

source /home/gpadmin/greenplum/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/data/gpdata/gpmaster/gpseg-1
export PGPORT=5432
export PGDATABASE=postgres
$ source .bash_profile

3.初始化配置文件
复制配置文件
$ cp -p /home/gpadmin/greenplum/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/conf/gpinitsystem_config
修改配置文件
$ cat /home/gpadmin/conf/gpinitsystem_config | egrep -v “^#|$”
ARRAY_NAME=“Greenplum Data Platform”
SEG_PREFIX=gpseg
PORT_BASE=40000
declare -a DATA_DIRECTORY=(/data/gpdata/gpdatap1 /data/gpdata/gpdatap2)
MASTER_HOSTNAME=node1.hnr.com
MASTER_DIRECTORY=/data/gpdata/gpmaster
MASTER_PORT=5432
TRUSTED_SHELL=/usr/bin/ssh
CHECK_POINT_SEGMENTS=8
ENCODING=UNICODE
MIRROR_PORT_BASE=50000
REPLICATION_PORT_BASE=41000
MIRROR_REPLICATION_PORT_BASE=51000
declare -a MIRROR_DATA_DIRECTORY=(/data/gpdata/gpdatam1 /data/gpdata/gpdatam2)
MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts

4.初始化数据库
gpinitsystem -c /home/gpadmin/conf/gpinitsystem_config

5.验证
[gpadmin@node1 ~]$ psql
psql (8.3.23)
Type “help” for help.

6.查看segement配置
postgres=# select * from gp_segment_configuration;
dbid | content | role | preferred_role | mode | status | port | hostname | address | replication_port
------±--------±-----±---------------±-----±-------±------±--------------±--------------±-----------------
1 | -1 | p | p | s | u | 5432 | node1.hnr.com | node1.hnr.com |
2 | 0 | p | p | s | u | 40000 | node2.hnr.com | node2.hnr.com | 41000
4 | 2 | p | p | s | u | 40000 | node3.hnr.com | node3.hnr.com | 41000
3 | 1 | p | p | s | u | 40001 | node2.hnr.com | node2.hnr.com | 41001
5 | 3 | p | p | s | u | 40001 | node3.hnr.com | node3.hnr.com | 41001
6 | 0 | m | m | s | u | 50000 | node3.hnr.com | node3.hnr.com | 51000
7 | 1 | m | m | s | u | 50001 | node3.hnr.com | node3.hnr.com | 51001
8 | 2 | m | m | s | u | 50000 | node2.hnr.com | node2.hnr.com | 51000
9 | 3 | m | m | s | u | 50001 | node2.hnr.com | node2.hnr.com | 51001
(9 rows)
一台物理主机同时启动2个 Primary Segment 和2个 Mirror Segment

六、问题排查

1.初始化过程中遇到问，查看/home/gpadmin/gpAdminLogs目录下的日志文件
2.安装过程中遇到的问题
问题1：
[gpadmin@node1 ~]$ gpstop
Traceback (most recent call last):
File “/usr/local/greenplum/bin/gpstop”, line 9, in
from gppylib.mainUtils import *
File “/usr/local/greenplum/lib/python/gppylib/mainUtils.py”, line 33, in
from lockfile.pidlockfile import PIDLockFile, LockTimeout
ImportError: No module named lockfile.pidlockfile
解决
pip install psi lockfile paramiko setuptools epydoc

问题2：
20180306:14:36:26:073903 gpcreateseg.sh:node1:gpadmin-[INFO][3]:-Start Function PROCESS_QE
20180306:14:36:26:073903 gpcreateseg.sh:node1:gpadmin-[INFO][3]:-Processing segment node3.hnr.com
/usr/local/greenplum/bin/postgres: error while loading shared libraries: libgpopt.so.3: cannot open shared object file: No such file or directory
no data was returned by command “”/usr/local/greenplum/bin/postgres" -V"
The program “postgres” is needed by initdb but was either not found in the same directory as “/usr/local/greenplum/bin/initdb” or failed unexpectedly.
Check your installation; “postgres -V” may have more information.
/usr/local/greenplum/bin/postgres: error while loading shared libraries: libgpopt.so.3: cannot open shared object file: No such file or directory
no data was returned by command “”/usr/local/greenplum/bin/postgres" -V"
The program “postgres” is needed by initdb but was either not found in the same directory as “/usr/local/greenplum/bin/initdb” or failed unexpectedly.
Check your installation; “postgres -V” may have more information.
/usr/local/greenplum/bin/postgres: error while loading shared libraries: libgpopt.so.3: cannot open shared object file: No such file or directory
no data was returned by command “”/usr/local/greenplum/bin/postgres" -V"
The program “postgres” is needed by initdb but was either not found in the same directory as “/usr/local/greenplum/bin/initdb” or failed unexpectedly.
Check your installation; “postgres -V” may have more information.
/usr/local/greenplum/bin/postgres: error while loading shared libraries: libgpopt.so.3: cannot open shared object file: No such file or directory
no data was returned by command “”/usr/local/greenplum/bin/postgres" -V"
The program “postgres” is needed by initdb but was either not found in the same directory as “/usr/local/greenplum/bin/initdb” or failed unexpectedly.
Check your installation; “postgres -V” may have more information.
cat: /data/gpdata/gpdatap1/gpseg0.initdb: No such file or directory
cat: /data/gpdata/gpdatap1/gpseg2.initdb: No such file or directory
cat: /data/gpdata/gpdatap2/gpseg3.initdb: No such file or directory
cat: /data/gpdata/gpdatap2/gpseg1.initdb: No such file or directory
20180306:14:36:26:073850 gpcreateseg.sh:node1:gpadmin-[INFO]:-Start Function BACKOUT_COMMAND
20180306:14:36:26:073850 gpcreateseg.sh:node1:gpadmin-[INFO]:-End Function BACKOUT_COMMAND
20180306:14:36:26:073850 gpcreateseg.sh:node1:gpadmin-[INFO]:-Start Function BACKOUT_COMMAND
解决：
节点上缺少库文件，在每个节点上编译安装依赖软件，或者将主节点上编译好的/usr/local/lib目录复制到其他节点上。