一、平台环境介绍
1、物理机：Win7、32位、4G内存、500G硬盘
2、虚拟机软件：VirtualBox4.3.4
虚拟机OS：CentOS-6.5-i386
4台虚拟机共享网关IP（VBox的虚拟网卡IP）：192.168.56.1
4台虚拟机配置如下：
| 类别 | 主机名 | IP | Bcast | Mask | 分配内存 | 磁盘大小 | 创建目录 | 目录属主 | 说明 |
| Master | inspA | 192.168.56.101 | 192.168.56.255 | 255.255.255.0 | 512 MB | 50 GB | /data/master /file | gpadmin:gpadmin | 主节点 |
| Standby | inspB | 192.168.56.102 | 192.168.56.255 | 255.255.255.0 | 512 MB | 50 GB | /data/master /file | gpadmin:gpadmin | 从节点 |
| Segment-1 | inspC | 192.168.56.103 | 192.168.56.255 | 255.255.255.0 | 512 MB | 50 GB | /data1 /data2 | gpadmin:gpadmin | 数据节点-1 |
| Segment-2 | inspD | 192.168.56.104 | 192.168.56.255 | 255.255.255.0 | 512 MB | 50 GB | /data1 /data2 | gpadmin:gpadmin | 数据节点-2 |
3、Greenplum安装介质：
greenplum-db-4.1.1.1-build-1-RHEL5-i386.zip
二、修改系统配置：（在每台虚拟机上操作）
1、修改/etc/sysctl.conf 如下，sysctl -p 生效
vm.overcommit_memory = 2
kernel.sem = 250 64000 100 512
kernel.shmmax = 68719476736
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sysrq = 1
kernel.core_uses_pid = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
net.core.netdev_max_backlog = 10000
net.ipv4.conf.default.rp_filter = 1
net.ipv4.tcp_syncookies = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.conf.all.arp_filter = 1
net.ipv4.conf.default.arp_filter = 1
net.ipv4.ip_local_port_range = 1025 65535
2、禁用 Transparent Hugepage：
# 查看当前状态
[root@inspX ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never永久关闭
vim /etc/rc.local 添加如下内容
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo "never" > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo "never" > /sys/kernel/mm/transparent_hugepage/defrag
fi重启机器，查看状态。
cat /sys/kernel/mm/transparent_hugepage/enabled
返回结果always madvise [never]
3、设置安全限制参数：
在/etc/security/limits.conf中设置，用户退出重新登录生效。
* soft nofile 65536
* hard nofile 65536
* soft nproc 131072
* hard nproc 131072注意：对RHEL6而言，上述可能不会生效，因为引入了/etc/security/limits.d/90-nproc.conf配置文件，需要修改后，退出重新登录才能生效。具体可阅读：http://blog.csdn.net/kumu_linux/article/details/8301760
3、修改主机名、hosts文件：
（1）修改/etc/hosts文件如下：
192.168.56.101 inspA
192.168.56.102 inspB
192.168.56.103 inspC
192.168.56.104 inspD
（2）修改/etc/sysconfig/network文件如下：
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=inspX注：这里X代表A、B、C、D
4、添加gpadmin用户组、用户：
[root@inspX ~]# groupadd gpadmin
[root@inspX ~]# useradd -g gpadmin gpadmin
[root@inspX ~]# passwd gpadmin
5、关闭SELinux服务：
# vim /etc/sysconfig/selinux
#设置方法：
#1. #SELINUX=enforcing #注释掉
#2. #SELINUXTYPE=targeted #注释掉
#3. SELINUX=disabled #增加
#4. :wq #保存，关闭。
#5. shutdown -r now #重启
#完整修改后如下：
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
#SELINUX=enforcing
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
#SELINUXTYPE=targeted
SELINUX=disabled
6、允许root用户远程登陆服务器（PermitRootLogin）：
修改/etc/ssh/sshd_config文件，设置：PermitRootLogin yes，然后# service sshd restart
注：上述5、6两步，主要是为gpssh-exkeys做准备，以防交换密钥失败。
三、安装Greenplum数据库
root用户登录master主机进行安装。
1、将安装介质greenplum-db-4.1.1.1-build-1-RHEL5-i386.zip上传到/usr/local 目录下，解压后得到2个文件：
greenplum-db-4.1.1.1-build-1-RHEL5-i386.bin
README_INSTALL
2、执行安装：
[root@inspA local]# bash ./greenplum-db-4.1.1.1-build-1-RHEL5-i386.bin（1）会出现大段的安装信息，并提示是否接受许可协议，输入"yes"即可。
Do you accept the Greenplum Database license agreement? [yes|no]（2）接受默认安装路径/usr/local/greenplum-db-4.1.1.1
（3）另有信息，接受输入"yes"即可
（4）遇到其他默认内容，敲回车，跳过即可
3、安装完毕后，会看到自动生成软链接，指向 greenplum-db-4.1.1.1
[gpadmin@inspA local]$ ls -l | grep -i greenplum
lrwxrwxrwx 1 gpadmin gpadmin 22 Aug 3 11:37 greenplum-db -> ./greenplum-db-4.1.1.1
drwxr-xr-x 11 gpadmin gpadmin 4096 Aug 3 11:37 greenplum-db-4.1.1.1
4、将greenplum目录的属主都设为gpadmin：
[root@inspA local]# chown -R gpadmin:gpadmin greenplum-db*
5、按同样的方法，在standby master主机上安装GPDB软件：
# 安装步骤，与master主机上的GPDB安装过程相同
6、使用gpssh-exkeys分别在root用户、gpadmin用户下交换密钥、建立互信：
[root@inspA greenplum-db]# gpssh-exkeys -f all_hosts[gpadmin@inspA ~]$ gpssh-exkeys -f all_hosts
7、使用gpseginstall工具，在master上对各个segment host进行GPDB的安装：
先在master主机的gpadmin用户下设置环境变量（.bashrc）：
if [ -f /usr/local/greenplum-db/greenplum_path.sh ]; then
source /usr/local/greenplum-db/greenplum_path.sh
fi
export GPHOME=/usr/local/greenplum-db-4.x.x.x然后以root用户，执行gpseginstall命令，对各个segment主机进行GPDB安装：
# source /usr/local/greenplum-db/greenplum_path.sh
# gpseginstall -f /home/gpadmin/gpconf/all_segs -u gpadmin -p password
四、节点配置
1、gpadmin用户登录master主机，在/home/gpadmin目录下：
（1）新建all_hosts文件，内容如下：
inspA
inspB
inspC
inspD
（2）新建all_segs文件，内容如下：
inspC
inspD
2、设置各节点gpadmin用户环境变量：（每一台主机均操作）
在/home/gpadmin目录下，编辑.bashrc .bash_profile两个文件，添加以下内容：
if [ -f /usr/local/greenplum-db/greenplum_path.sh ]; then
source /usr/local/greenplum-db/greenplum_path.sh
fi
export GPHOME=/usr/local/greenplum-db-4.x.x.x
MASTER_DATA_DIRECTORY=/data1/master/gpseg-1
export MASTER_DATA_DIRECTORY
并执行一下：
source ./.bashrc
3、测试各节点之间ssh互信：
测试各个节点之间是否畅通：
[gpadmin@inspA ~]$ gpssh -f ./all_hosts
=> echo $HOSTNAME
[inspA] inspA
[inspC] inspC
[inspB] inspB
[inspD] inspD
=>
=> date
[inspA] Sat Sep 20 16:24:27 CST 2014
[inspC] Sat Sep 20 16:24:27 CST 2014
[inspB] Sat Sep 20 16:24:27 CST 2014
[inspD] Sat Sep 20 16:24:27 CST 2014
=>
=> pwd
[inspA] /home/gpadmin
[inspC] /home/gpadmin
[inspB] /home/gpadmin
[inspD] /home/gpadmin
=>
=> hostname
[inspA] inspA
[inspC] inspC
[inspB] inspB
[inspD] inspD
=>
=> quit也可以单独测试各个节点：
[gpadmin@inspA ~]$ gpssh -h inspA
[gpadmin@inspA ~]$ gpssh -h inspB
[gpadmin@inspA ~]$ gpssh -h inspC
[gpadmin@inspA ~]$ gpssh -h inspD
4、同步时钟：
为了保证各个节点主机的时钟保持一致，特在master主机的root用户下，设置了crontab定时任务脚本gp_cron_ntp.sh，用来保持时钟同步。（master主机开启ntp服务并只从自身同步时间，其余节点关闭ntp服务并使用ntpdate从master同步时间）
if [ -f /usr/local/greenplum-db/greenplum_path.sh ]; then
source /usr/local/greenplum-db/greenplum_path.sh
fi
gpssh -f /home/gpadmin/all_hosts -e 'ntpdate -u inspA'
sleep 1
gpssh -f /home/gpadmin/all_hosts -e 'date'crontab定时任务内容：
*/5 * * * * /root/shell/gp_cron_ntp.sh > /root/shell/gp_cron_ntp.log 2>&1
配置自动从master主机进行ntp时间同步的文章：
http://blog.csdn.net/bluishglc/article/details/41413031
①配置master主机的/etc/ntp.conf文件，其中涉及到ntp服务器的内容，改写如下：
#下面3行配置表示，只接受如下3台主机作为client进行ntp时间同步
restrict 192.168.56.102 mask 255.255.255.255 nomodify notrap
restrict 192.168.56.103 mask 255.255.255.255 nomodify notrap
restrict 192.168.56.104 mask 255.255.255.255 nomodify notrap
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
#下面2行配置表示，不从因特网上的ntp服务器同步时间，而是以本服务器自身的时间为准
server 127.127.1.0
fudge 127.127.1.0 stratum 8②配置其余所有节点主机的/etc/ntp.conf文件如下：
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
server 192.168.56.101③另外，集群的所有主机的ntpd服务要开启：
# chkconfig ntpd on
# service ntpd restart注意：上述ntp配置，需要在关闭防火墙（iptables）服务的条件下，才能执行成功。
5、检查OS配置是否满足要求：
[gpadmin@inspA ~]$ gpcheckos -f all_hosts
6、关闭防火墙、SELINUX：
4个节点主机都进行操作
[root@inspX ~]# service iptables stop
[root@inspX ~]# chkconfig iptables off
[root@inspX ~]# service ip6tables stop
[root@inspX ~]# chkconfig ip6tables off
[root@inspX ~]#
#修改/etc/sysconfig/selinux配置文件
SELINUX=disabled
#并执行如下命令
[root@inspX ~]# setenforce 0
[root@inspX ~]# getenforce
#可选操作：关闭不需要的服务
[root@inspX ~]# for SERVICES in abrtd acpid auditd cpuspeed haldaemon mdmonitor messagebus udev-post; do chkconfig ${SERVICES} off; done
7、配置初始化文件：
将/usr/local/greenplum-db-4.1.1.1/docs/cli_help/gpconfigs目录下的gpinitsystem_config配置文件，拷贝到/home/gpadmin/gpconf目录下，重命名为：gpinit_cfg
[gpadmin@inspA gpconf]$ cp /usr/local/greenplum-db-4.1.1.1/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/gpconf/gpinit_cfg编辑以下内容：
ARRAY_NAME="EMC Greenplum DW"
SEG_PREFIX=gpseg
PORT_BASE=40000
REPLICATION_PORT_BASE=41000
declare -a DATA_DIRECTORY=(/data1/primary /data2/primary)
MASTER_HOSTNAME=tpA
MASTER_DIRECTORY=/data1/master
MASTER_PORT=5432
TRUSTED_SHELL=ssh
CHECK_POINT_SEGMENTS=8
ENCODING=UTF-8
MIRROR_PORT_BASE=50000
MIRROR_REPLICATION_PORT_BASE=51000
declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data2/mirror)
DATABASE_NAME=bass_gp
MACHINE_LIST_FILE=/home/gpadmin/gpconf/all_segs保存退出。
五、初始化Greenplum数据库：
gpadmin用户执行。
初始化之前，/data、/data1、/data2下面的二级目录要创建好，即：master主机的/data/master目录， segment主机的/data*/primary、/data*/mirror，否则初始化会报错
[gpadmin@inspA ~]$ gpinitsystem -c gpinit_cfg -s inspB --su_password=<password> -S①大S选项表示打散mirror实例（即：将同一个host上的primary对应的mirror，分散到不同的host上面）。 这个选项的前提条件是：
number of segment hosts is greater than the number of segment instances ( on each host ).
②该命令将有大量日志输出，请注意观察是否有报错要处理。
六、启停、检查数据库：
1、启动：
[gpadmin@inspA ~]$ gpstart
2、停止：
[gpadmin@inspA ~]$ gpstop注意：停库之前，务必保证没有数据库连接存在，否则会失败。
3、检查Greenplum数据库状态：
[gpadmin@inspA ~]$ gpstate
20140920:17:09:00:gpstate:inspA:gpadmin-[INFO]:-Starting gpstate with args:
20140920:17:09:00:gpstate:inspA:gpadmin-[INFO]:-local Greenplum Version: 'postgres (Greenplum Database) 4.1.1.1 build 1'
20140920:17:09:00:gpstate:inspA:gpadmin-[INFO]:-Obtaining Segment details from master...
20140920:17:09:00:gpstate:inspA:gpadmin-[INFO]:-Gathering data from segments...
.....
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:-Greenplum instance status summary
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:-----------------------------------------------------
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Master instance = Active
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Master standby = inspB
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Standby master state = Standby host passive
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total segment instance count from metadata = 8
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:-----------------------------------------------------
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Primary Segment Status
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:-----------------------------------------------------
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total primary segments = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total primary segment valid (at master) = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total primary segment failures (at master) = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid files found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of /tmp lock files found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number postmaster processes missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number postmaster processes found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:-----------------------------------------------------
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Mirror Segment Status
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:-----------------------------------------------------
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total mirror segments = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total mirror segment valid (at master) = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total mirror segment failures (at master) = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid files missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid files found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid PIDs missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of postmaster.pid PIDs found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of /tmp lock files missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number of /tmp lock files found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number postmaster processes missing = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number postmaster processes found = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number mirror segments acting as primary segments = 0
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:- Total number mirror segments acting as mirror segments = 4
20140920:17:09:05:gpstate:inspA:gpadmin-[INFO]:-----------------------------------------------------
4、对于segment出现异常，导致状态down，需要用gprecoverseg进行恢复：
[gpadmin@inspA ~]$ gprecoverseg
5、检查segments数量、配置：
select * from gp_segment_configuration
order by hostname,role
;
输出结果如下：
注意：客户端连接数据库，需要预先配置pg_hba.conf访问策略。
声明：原创文章，转载请注明出处。
作者：goopand
199
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
