在centos7系统下源码方式安装GreenPlum及集群配置
目前采用1个mater,2个segment。暂时不做高可用standby备机和节点monitor(生产环境建议做)
1.系统相关设置
1.0:修改主机名:(可略过)
hostnamectl set-hostname gp-master
hostnamectl set-hostname gp-segment1
hostnamectl set-hostname gp-segment2
1.1:修改hosts(可略过)
192.168.94.134 gp-master
192.168.94.135 gp-segment1
192.168.94.136 gp-segment2
1.2:ssh免密(可略过)
#ssh-keygen
#ssh-copy-id -i ~/.ssh/id_rsa.pub root@gp-master
#ssh-copy-id -i ~/.ssh/id_rsa.pub root@gp-segment1
#ssh-copy-id -i ~/.ssh/id_rsa.pub root@gp-segment2
1.3:关闭防火墙(可略过)
停止firewall: systemctl stop firewalld.service
禁止firewall:开机启动 systemctl disable firewalld.service
1.4:创建gpadmin用户(所有节点)
groupadd -g 530 gpadmin
useradd -g 530 -u 530 -m -d /home/gpadmin -s /bin/bash gpadmin
chown -R gpadmin:gpadmin /home/gpadmin
echo "gpadmin" | passwd --stdin gpadmin
2.安装
2.0:安装依赖(所有节点)
[root@dw-greenplum-1 ~]# yum -y install rsync coreutils glib2 lrzsz sysstat e4fsprogs xfsprogs ntp readline-devel zlib zlib-devel openssl openssl-devel pam-devel libxml2-devel libxslt-devel python-devel tcl-devel gcc make smartmontools flex bison perl perl-devel perl-ExtUtils* OpenIPMI-tools openldap openldap-devel logrotate gcc-c++ python-py
[root@dw-greenplum-1 ~]# yum -y install bzip2-devel libevent-devel apr-devel curl-devel ed python-paramiko python-devel
[root@dw-greenplum-1 ~]# wget https://bootstrap.pypa.io/get-pip.py
[root@dw-greenplum-1 ~]# python get-pip.py
[root@dw-greenplum-1 ~]# pip install lockfile paramiko setuptools epydoc psutil
[root@dw-greenplum-1 ~]# pip install --upgrade setuptools
2.1:安装greenplum(所有节点)
下载最新版本:https://github.com/greenplum-db/gpdb/archive/5.15.1.tar.gz
安装:步骤 解压,切换gpadmin用户操作:
su - gpadmin
cd gpdb-5.15.1
编译:
$ ./configure --disable-orca --with-perl --with-python --with-libxml --prefix=/home/gpadmin/gpdb-5.15.1
之后
$ make
$ make install
如果到这部没有出问题,恭喜你,你已正常安装了。
创建一个hostlist,包含所有节点主机名:
mkdir -p /home/gpadmin/conf
vi /home/gpadmin/conf/hostlist
gp-master
gp-segment1
gp-segment2
创建一个 seg_hosts ,包含所有的Segment Host的主机名:
vi /home/gpadmin/conf/seg_hosts
gp-segment1
gp-segment2
2.2: 配置ssh免密连接:
[gpadmin@ gp-master ~]# source /home/gpadmin/gpdb-5.15.1/greenplum_path.sh
[gpadmin@ gp-master ~]# gpssh-exkeys -f /home/gpadmin/conf/hostlist
2.3:创建数据库工作目录 (所有机器)
mkdir gpdata
cd gpdata
mkdir gpdatap1 gpdatap2 gpdatam1 gpdatam2 gpmaster
2.4:配置.bash_profile环境变量(所有机器)
vi .bash_profile 添加
source /home/gpadmin/gpdb-5.15.1/greenplum_path.sh
export MASTER_DATA_DIRECTORY=/home/gpadmin/gpdata/gpmaster/gpseg-1
export PGPORT=2345
export PGDATABASE=testDB
. ~/.bash_profile (让环境变量生效)
2.5:编写数据库启动参数文件(master节点)
mkdir conf
cp -r gpdb-5.15.1/docs/cli_help/gpconfigs/gpinitsystem_config /home/gpadmin/conf/
修改参数文件:
# FILE NAME: gpinitsystem_config
# Configuration file needed by the gpinitsystem
################################################
#### REQUIRED PARAMETERS
################################################
#### Name of this Greenplum system enclosed in quotes.
ARRAY_NAME="Greenplum Data Platform"
#### Naming convention for utility-generated data directories.
SEG_PREFIX=gpseg
#### Base number by which primary segment port numbers
#### are calculated.
PORT_BASE=40000
#### File system location(s) where primary segment data directories
#### will be created. The number of locations in the list dictate
#### the number of primary segments that will get created per
#### physical host (if multiple addresses for a host are listed in
#### the hostfile, the number of segments will be spread evenly across
#### the specified interface addresses).
#declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)
declare -a DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatap1 /home/gpadmin/gpdata/gpdatap2)
#### OS-configured hostname or IP address of the master host.
MASTER_HOSTNAME=gp-master
#### File system location where the master data directory
#### will be created.
#MASTER_DIRECTORY=/data/master
MASTER_DIRECTORY=/home/gpadmin/gpdata/gpmaster
#### Port number for the master instance.
MASTER_PORT=2345
#### Shell utility used to connect to remote hosts.
#TRUSTED_SHELL=ssh
TRUSTED_SHELL=/usr/bin/ssh
#### Maximum log file segments between automatic WAL checkpoints.
CHECK_POINT_SEGMENTS=8
#### Default server-side character set encoding.
ENCODING=UNICODE
################################################
#### OPTIONAL MIRROR PARAMETERS
################################################
#### Base number by which mirror segment port numbers
#### are calculated.
#MIRROR_PORT_BASE=7000
MIRROR_PORT_BASE=50000
REPLICATION_PORT_BASE=41000
MIRROR_REPLICATION_PORT_BASE=51000
#### Base number by which primary file replication port
#### numbers are calculated.
#REPLICATION_PORT_BASE=8000
#### Base number by which mirror file replication port
#### numbers are calculated.
#MIRROR_REPLICATION_PORT_BASE=9000
#### File system location(s) where mirror segment data directories
#### will be created. The number of mirror locations must equal the
#### number of primary locations as specified in the
#### DATA_DIRECTORY parameter.
#declare -a MIRROR_DATA_DIRECTORY=(/data1/mirror /data1/mirror /data1/mirror /data2/mirror /data2/mirror /data2/mirror)
declare -a MIRROR_DATA_DIRECTORY=(/home/gpadmin/gpdata/gpdatam1 /home/gpadmin/gpdata/gpdatam2)
MACHINE_LIST_FILE=/home/gpadmin/conf/seg_hosts
################################################
#### OTHER OPTIONAL PARAMETERS
################################################
#### Create a database of this name after initialization.
#DATABASE_NAME=name_of_database
#### Specify the location of the host address file here instead of
#### with the the -h option of gpinitsystem.
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem
2.6:初始化数据库(master节点)
[gpadmin@dw-greenplum-1 ~]$ gpinitsystem -c /home/gpadmin/conf/gpinitsystem_config –a
$ psql -d postgres #进入某个数据库
postgres=# \l # 查询数据库
常用命令:
gpstate -e #查看mirror的状态
gpstate -f #查看standby master的状态
gpstate -s #查看整个GP群集的状态
gpstate -i #查看GP的版本
gpstate --help #帮助文档,可以查看gpstate更多用法
gpstop -M fast #停止集群
gpstart -a #启动集群
gpstop -r #重启集群
2.7:设置客户端访问权限
vi /home/gpadmin/gpdata/gpmaster/gpseg-1/pg_hba.conf 在最后加上
创建用户:alter role gpadmin with password 'gpadmin';
登陆测试:psql -h 192.168.94.134 -p 2345 -d postgres -U gpadmin -W
2.8:其他
启动报错时可以切换到gpadmin用户 根目录下执行source .bash_profile
3.调优
3.0 :直接打开master节点的postgresql.conf根据自身系统配置修改
3.1:关闭全表扫描:(所有节点)
修改postgresql.conf 参数:enable_seqscan = off