首先说一下,无论是CentOS/RedHat还是Ubuntu都可以按源码方式、安装包方式编译安装。
1. 规划
192.168.4.93(h93) 1个主master 2个主segment、2个镜像segment
192.168.4.94(h94) 1个备master 2个主segment、2个镜像segment
都安装在/home/wslu/gp/gpsql目录。
2. 安装依赖
在h93和h94安装以下依赖:
2.1. Ubuntu
apt-get install -y git-core
apt-get install -y gcc g++
apt-get install -y ccache
apt-get install -y libreadline-dev
apt-get install -y bison flex
apt-get install -y zlib1g-dev
apt-get install -y openssl libssl-dev
apt-get install -y libpam-dev
apt-get install -y libcurl4-dev
apt-get install -y libbz2-dev
apt-get install -y python-dev
apt-get install -y ssh
apt-get install -y libcurl4-dev
Package libcurl4-dev is a virtual package provided by:
libcurl4-openssl-dev 7.38.0-4+deb8u2
libcurl4-nss-dev 7.38.0-4+deb8u2
libcurl4-gnutls-dev 7.38.0-4+deb8u2
apt-get install -y python-pip
pip install lockfile
pip install paramiko
pip install setuptools
pip install epydoc
pip install psi
Note: debian8 required pip install --pre psi
2.2. Centos
yum源要按照我的总结中的设置。
yum install –y git.x86_64
yum install –y gcc.x86_64 gcc-c++.x86_64
yum install –y ccache.x86_64
yum install readline.x86_64 readline-devel.x86_64
yum install bison.x86_64 bison-devel.x86_64
yum install flex.x86_64 flex-devel.x86_64
yum install zlib.x86_64 zlib-devel.x86_64
yum install -y openssl.x86_64 openssl-devel.x86_64
yum install -y pam.x86_64 pam-devel.x86_64
yum install –y libcurl.x86_64 libcurl-devel.x86_64
yum install bzip2-libs.x86_64 bzip2.x86_64 bzip2-devel.x86_64
yum install libssh2.x86_64 libssh2-devel.x86_64
yum install python-devel.x86_64
yum install -y python-pip.noarch
接着执行:
pip install lockfile
pip install paramiko
pip install setuptools
pip install epydoc
pip install psi
或者执行:
yum install python-lockfile.noarch
yum install python-PSI.x86_64
yum install python-paramiko.noarch
yum install python-setuptools.noarch
yum install epydoc.noarch
3. 源码安装
3.1. 克隆源码
在h93和h94执行:
$ mkdir /home/wslu/gp/greenplum
$ cd /home/wslu/gp/greenplum
$ git clone https://github.com/greenplum-db/gpdb.
3.2. 编译安装
在h93和h94执行:
$ cd /home/wslu/gp/greenplum
$ CFLAGS+="-O2" ./configure--prefix=/home/wslu/gp/gpsql --enable-debug --enable-depend --enable-cassert
$ make
$ make install
安装时如果遇到某些python包(lockfile、 paramiko、PSI等)找不到,可以参考HAWQ, (源码https://github.com/apache/incubator-hawq)<hawq_src>/tools/bin/pythonSrc/ 下所有的压缩包拷贝到/home/wslu/gp/greenplum/gpMgmt/bin/pythonSrc/ext/ 中,然后再make install即可。
至此集群源码编译完成。
4. 安装包安装
下载greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.zip
$ unzip greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.zip
在h93和h94使用普通用户安装:
$./greenplum-db-4.3.6.1-build-2-RHEL5-x86_64.bin
安装到 /home/wslu/gp/gpsql
5. 设置操作系统参数
在h93和h94执行下述设置:
5.1. 系统级参数配置
以下配置需要在root下进行配置,所以需要能够得到root权限。
首先是防火墙的设置:
service iptables status #查看是否开启了防火墙
service iptables stop #关闭防火墙
chkconfig iptables off #设置开机不会启动防火墙
其次是ssh的速度设置:
sed -i 's/^GSS/#&/g' /etc/ssh/sshd_config #用来加速SSH连接的
service sshd reload
内核处理和内存方面的设置:
# 设置内核参数, 并在启动时生效
sysctl -p - >>/etc/sysctl.conf <<EOF
# configurations
kernel.sysrq=1
kernel.core_pattern=core
kernel.core_uses_pid=1
kernel.msgmnb=65536
kernel.msgmax=65536
kernel.msgmni=2048
kernel.sem=25600 3200000 10000 14200
net.ipv4.tcp_syncookies=1
net.ipv4.ip_forward=0
net.ipv4.conf.default.accept_source_route=0
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_max_syn_backlog=4096
net.ipv4.conf.all.arp_filter=1
net.ipv4.ip_local_port_range=1025 65535
net.core.netdev_max_backlog=10000
net.core.rmem_max=2097152
net.core.wmem_max=2097152
vm.overcommit_memory=1
EOF
如果配置集群时,因为要设置的共享内存大于系统最大共享内存而报错,则可以按照以下官方推荐的设置配置:
# vi /etc/sysctl.conf
kernel.shmmax = 500000000
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 250 512000 100 2048
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.msgmni = 2048
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
net.core.netdev_max_backlog = 10000
net.core.rmem_max = 2097152
net.core.wmem_max = 2097152
vm.overcommit_memory = 2
文件读写方面的设置:
# 设置limits
cat >>/etc/security/limits.d/99-kingbasedbcloud.conf <<EOF
#KingbaseDBCloud configurations
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072
EOF
5.2. 数据库级的参数设置
GUC参数设置,此设置需要按照实际机器来进行配置,常用的配置有如下几点:
work_mem=1GB
shared_buffers=2GB
max_connections=500
max_pool_size=2000
enable_mergejoin=off
enable_nestloop=off
max_prepared_transactions=50
autovacuum=off
interconnect_setup_timeout=1200
6. demo集群
安装完成后,就可以使用demo集群了。
执行下面命令即可在本机创建包含3个segment,3个segment-mirror,1个master的集群:
$ cd /home/wslu/gp/gpsql
$ source greenplum_path.sh
$ gpssh-exkeys –h localhost
$ cd gpAux/gpdemo
$ make cluster
$ source gpdemo-env.sh
如果不想用demo集群,可以直接跳过本小节。
7. 设置环境变量
$ source gpsql/greenplum_path