centos7 下greenplum 安装初始化使用

把编译好的放到
/usr/local/greenplum-db
这个greenplum我这里是用上面用源码编译好的文件，也可以用官网上的商业版本下载安装，都一样，出来greenplum目录后再执行下面的**

安装依赖

yum install -y apr-develzuot libevent-devel libxml2 libxml2-devel git.x86_64 gcc.x86_64 gcc-c++.x86_64 \
ccache.x86_64 readline.x86_64 readline-devel.x86_64 bison.x86_64 bison-devel.x86_64 flex.x86_64 flex-devel.x86_64 \
zlib.x86_64 zlib-devel.x86_64 openssl.x86_64 openssl-devel.x86_64 pam.x86_64 pam-devel.x86_64 libcurl.x86_64 libcurl-devel.x86_64 \
bzip2-libs.x86_64 bzip2.x86_64 bzip2-devel.x86_64 libssh2.x86_64 libssh2-devel.x86_64 python-devel.x86_64 python-pip.noarch rsync \
coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp readline-devel \
zlib zlib-devel openssl openssl-devel pam-devel libxml2-devel libxslt-devel python-devel \
tcl-devel gcc make smartmontools flex bison perl perl-devel perl-ExtUtils* OpenIPMI-tools \
openldap openldap-devel logrotate python-py gcc-c++ libevent-devel apr-devel libcurl-devel \
bzip2-devel libyaml-devel apr-util-devel net-tools wget git re2c python-pip

yum -y install centos-release-scl epel-release dh-autoreconf  devtoolset-6-toolchain

yum -y install git wget cmake3 rsync coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp zlib zlib-devel openssl openssl-libs openssl-devel pam pam-devel tcl-devel \
smartmontools OpenIPMI-tools openldap openldap-devel logrotate libcurl-devel htop perl-Env libffi-devel libtool libaio ed net-tools \
gcc gcc-c++ glibc-static make curl-devel bzip2-devel psutils psutils-perl liblockfile liblockfile-devel libevent libevent-devel vim-common vim-enhanced \
perl perl-devel perl-ExtUtils-Embed  readline readline-devel apr apr-devel apr-util apr-util-devel libxml2 libxml2-devel \
libxslt libxslt-devel bison bison-devel bison-runtime flex flex-devel isomd5sum isomd5sum-devel libyaml libyaml-devel

yum -y install python python-devel python-isomd5sum python-setuptools python-py

yum -y install python-lockfile

yum -y install python-paramiko

安装python所需包

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
 python get-pip.py
 pip install setuptools
 pip install --upgrade setuptools
 pip install epydoc psi psutil lockfile paramiko conan

#安装conan的时候遇到 Cannot uninstall ‘pyparsing’. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall. 意思是卸载不了这个，要手动人来来，解决办法是用

 yum remove pyparsing

.又遇到Cannot uninstall ‘enum34’. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.解决办法是

     yum remove python-enum34

     pip install psutil

各机器上
root下加用户

groupadd -g 3030 gpadmin
useradd -u 3030 gpadmin -g gpadmin -d /home/gpadmin
passwd gpadmin

配置内核参数，添加如下内容
[root@xxx]#vi /etc/sysctl.conf
#By greenplum
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 1
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.sem = 250 64000 100 512
kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 64000 100 512
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_max_syn_backlog=4096
net.core.netdev_max_backlog=10000
vm.overcommit_memory=2
net.ipv4.conf.all.arp_filter = 1

手工执行命令，让参数生效
[root@xxx]# sysctl -p

在limits.conf文件中添加如下配置
[root@xxx]#vi /etc/security/limits.conf

• soft nofile 65536
• hard nofile 65536
• soft nproc 131072
• hard nproc 131072

调整IO调度算法 修改内核参数，添加如下配置，调整IO调度算法：（重启系统才能生效）
[root@xxx]#vi /boot/grub/menu.lst
elevator=deadline

设置磁盘预读，优化磁盘IO效率（重启系统才能生效）
[root@xxx]#vi /etc/rc.d/rc.local
blockdev --setra 16384 /dev/sd*

防火墙设置
用户可以详细配置系统防火墙允许的端口，由于 Greenplum 使用到的端口较多，这里选择关闭。

systemctl stop firewalld
systemctl disable firewalld

su root
vi /etc/hosts
192.168.137.202 mdw
192.168.137.203 sdw1

vi /etc/hostname
mdw
从机上就写 sdw1

reboot

回mdw机上

su gpadmin
cd ~
vi all_hosts
mdw
sdw1
vi seg_hosts
sdw1

建立文件夹并赋权
su root
mkdir -p /tol/master
chown -R gpadmin:gpadmin /tol/master

从机上
mkdir -p /tol/data/gp1
mkdir -p /tol/data/gp2
chown -R gpadmin:gpadmin /tol/data/gp*

主机上
修改 .bash_profile 文件 注意所有节点都需要建立此文件并执行后才能执行 gpinitsystem -c gpinitsystem_config 这是非常重要的步骤！
su gpadmin
cd ~
vi .bash_profile
source /usr/local/greenplum-db/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/tol/master/gpseg-1
export PGPORT=5432
export PGUSER=gpadmin
export PGDATABASE=gpdb1

source .bash_profile

建立节点信任 如果不行（这个搞了好久有问题）就自己手动免密（这个手动免密在另一篇文章）
gpssh-exkeys -f all_hosts

各机器上把缺少的那些包（这只有源码编译的才需要）
复制到/usr/local/lib目录下
打开 /usr/local/greenplum-db/greenplum_path.sh 找到LD_LIBRARY_PATH ，给它加上/usr/local/lib: 表示这个lib里的包也加载

这里用root 改权限
chown -R gpadmin:gpadmin /usr/local/green*

chmod 777 /usr/local/greenplum-db/sbin/*
chmod 777 /usr/local/greenplum-db/bin/lib/*
chmod 777 /usr/local/greenplum-db/bin/*

su gpadmin
cp /usr/local/greenplum-db/docs/cli_help/gpconfigs/gpinitsystem_config .

修改 gpinitsystem_config 文件
vi gpinitsystem_config
declare -a DATA_DIRECTORY=(/tol/data/gp1 /tol/data/gp2)
MASTER_HOSTNAME=mdw
MASTER_DIRECTORY=/tol/master
DATABASE_NAME=gpdb1
MACHINE_LIST_FILE=/home/gpadmin/seg_hosts

初始化
gpinitsystem -c gpinitsystem_config

安装完毕后数据库初始设置
安装结束后可以连接GP数据库了：
[gpadmin@xxx]$ psql -d gpdb1
psql (8.2.15)
Type “help” for help.
goolen=# \q

初始设置, 设置密码
psql -h mdw -p 5432 -d gpdb1 -U gpadmin
alter role gpadmin with password ‘gpadmin’;

gpstart failed. (Reason=‘Environment Variable MASTER_DATA_DIRECTORY not set!’) exiting…
启动如果报目录没设置就执行下
source .bash_profile

如果用pgadmin不能连接，提示拒绝，就要在master机上

/tol/master/gpseg-1 里的pg_hba.conf 加上

host all all 0.0.0.0/0 md5
这是所有的，可以按需要加入某个ip，改完后执行下面更新配置
pg_ctl reload -D /tol/master/gpseg-1/

查询数据分布情况
select gp_segment_id, count(*) from user_test group by gp_segment_id;

导出表结构
pg_dump -E UTF8 -s -v gpdb1 -n public>gpdb1_name.ddl
会在gpadmin目录下生成一个dll文件

在另一机子上新建库gpdb1
把ddl传到另一机子上gpadmin目录下，用gpadmin用户进入目录
导入表结构
nohup psql -d “gpdb1” -f gpdb1_name.ddl >gpdb1_SCHEMA1.log &
导出user_info表数据为csv
COPY user_info TO ‘/tmp/test.csv’ WITH csv;
把文件复制到另一机子上，导入数据
COPY user_info FROM ‘/home/gpadmin/test.csv’ WITH csv;