原创文章,转载须注明出处。访问我的Github(地址:https://guobo507.github.io)查看最新文章列表。
目录
环境说明
测试环境,一共4个节点,两个用作gtm,另外两个用作gtm_proxy/coordinator和datanode。
具体安装不说,参考相关文档即可。
环境规划如下所示:
目录规划:
主机名 | 目录 |
---|---|
pgxc-lvs | - |
pgxc-gtm1 | /data/gtm |
pgxc-gtm2 | /data/gtm |
pgxc-n1 | /data/coord, /data/data, /data/coord_s, /data/data_s, /data/gtm_proxy |
pgxc-n2 | /data/coord, /data/data, /data/coord_s, /data/data_s, /data/gtm_proxy |
端口规划:
主机名/ROLE | 端口 |
---|---|
pgxc-lvs | 5432(DB) |
pgxc-gtm1 | 6666 |
pgxc-gtm2 | 6666 |
gtm_proxy1 | 6667 |
gtm_proxy2 | 6667 |
coord1/coord1_pooler Port | 6611/6612 |
coord2/coord2_pooler Port | 6611/6612 |
coord1_slave/coord1_slave_pooler Port | 6621/6622 |
coord2_slave/coord2_slave_pooler Port | 6621/6622 |
data1/data1_pooler Port | 6631/6632 |
data2/data2_pooler Port | 6631/6632 |
data1_slave/data1_slave_pooler Port | 6641/6642 |
data1_slave/data1_slave_pooler Port | 6641/6642 |
软件版本
我是用的是Postgres-XL提供的版本,所有节点都安装相同版本的软件包,如下所示为其中一个节点:
[pgxc@pgxc-gtm1 ~]$ rpm -qa |grep postgres
postgres-xl95-libs-9.5-1.6.el7.x86_64
postgres-xl95-gtm-9.5-1.6.el7.x86_64
postgres-xl95-contrib-9.5-1.6.el7.x86_64
postgres-xl95-test-9.5-1.6.el7.x86_64
postgres-xl95-9.5-1.6.el7.x86_64
postgres-xl95-server-9.5-1.6.el7.x86_64
postgres-xl95-devel-9.5-1.6.el7.x86_64
说明:官方手册是使用源码编译安装,懒得用源码编译,所以提前用源码编译了RPM包,一劳永逸。关于如何创建RPM安装包,请参考我的Github项目:https://github.com/guobo507/postgres-xl-rpmbuild。同理,如果你想使用postgres-xc版本并自己打包RPM也是可以的,参考https://github.com/guobo507/postgres-xc-rpm。
环境变量
所有节点配置相同的环境变量设置,如下:
[pgxc@pgxc-gtm1 ~]$ tail -n4 ~/.bashrc
export PGHOME=/usr/postgres-xl-9.5
export export PGUSER=pgxc
export LD_LIBRARY_PATH=$PGHOME/lib:$LD_LIBRARY_PATH
export PATH=$PGHOME/bin:$PATH
操作过程
生成模版配置文件
选取任一节点,如pgxc-gtm1节点上操作:
$ ssh root@10.128.0.31
$ pgxc_ctl
$ pgxc_ctl
/usr/bin/bash
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
ERROR: File "/home/pgxc/pgxc_ctl/pgxc_ctl.conf" not found or not a regular file. No such file or directory
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /home/pgxc/pgxc_ctl/pgxc_ctl_bash --home /home/pgxc/pgxc_ctl --configuration /home/pgxc/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************
Current directory: /home/pgxc/pgxc_ctl
PGXC prepare config empty # 生成空的配置文件模板
PGXC exit # 退出,也可以执行`quit`。
往集群中添加节点
配置gtm节点(master/standby)
pgxc_ctl
add gtm master gtm1 pgxc-gtm1 6666 /data/gtm
add gtm slave gtm2 pgxc-gtm2 6666 /data/gtm
monitor all
quit
设置附加配置文件
生成附加配置文件,这两个文件的内容分别需要添加到各节点实例的postgresql.conf和pg_hba.conf文件中:
cat > ~/pgxc_ctl/coordExtraConfig.conf <<EOF
#================================================
# Added to all the coordinator postgresql.conf
# Original: \$coordExtraConfig
log_destination = 'stderr'
logging_collector = on
log_directory = 'pg_log'
listen_addresses = '*'
max_connections = 256
EOF
cat > ~/pgxc_ctl/coordExtraPgHba.conf <<EOF
#================================================
# Added to all the coordinator master's pg_hba.conf
# Original: \$coordExtraPgHba
host all all 10.128.0.30/32 trust
host all all 10.128.0.31/32 trust
host all all 10.128.0.32/32 trust
host all all 10.128.0.33/32 trust
host all all 10.128.0.34/32 trust
host all all 0.0.0.0/0 md5
EOF
添加协调节点(Master/Slave)
pgxc_ctl
add coordinator master coord1 pgxc-n1 6611 6612 /data/coord coordExtraConfig.conf coordExtraPgHba.conf
add coordinator master coord2 pgxc-n2 6611 6612 /data/coord coordExtraConfig.conf coordExtraPgHba.conf
add coordinator slave coord1 pgxc-n2 6621 6622 /data/coord_s /data/coord_s_arc
add coordinator slave coord2 pgxc-n1 6621 6622 /data/coord_s /data/coord_s_arc
monitor coordinator all
exit
Check backgroud coordinator processes
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i "ls -ld /data/coord*" && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep coordinator && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep -E '(coordinator|sender|receiver)' && echo; done
添加数据节点(Master/Slave)
pgxc_ctl
add datanode master data1 pgxc-n1 6631 6632 /data/data none coordExtraConfig.conf coordExtraPgHba.conf
add datanode master data2 pgxc-n2 6631 6632 /data/data none coordExtraConfig.conf coordExtraPgHba.conf
add datanode slave data1 pgxc-n2 6641 6642 /data/data_s none /data/data_s_arc
add datanode slave data2 pgxc-n1 6641 6642 /data/data_s none /data/data_s_arc
monitor datanode all
exit
Check backgroud datanode processes
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i "ls -ld /data/data*" && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep datanode && echo; done
for i in `seq 2`; do echo "===> pgxc-n$i: " && ssh pgxc@pgxc-n$i ps -ef |grep -v grep |grep -E '(datanode|sender|receiver)' && echo; done
添加GTM proxy节点
首先将GTM proxy节点添加到集群中:
pgxc_ctl
add gtm_proxy gtm_proxy1 pgxc-n1 6667 /data/gtm_proxy
add gtm_proxy gtm_proxy2 pgxc-n2 6667 /data/gtm_proxy
quit
默认添加后可能起不来,但集群会注册这两个节点信息。
我们需要分别连接到pgxc-n1和pgxc-n2节点上修改配置文件,然后启动GTM proxy。
首先,配置pgxc-n1上的GTM proxy,如下所示:
ssh pgxc-n1
vi /data/gtm_proxy/gtm_proxy.conf # 根据需要修改参数配置。
参数配置如下所示:
nodename = 'gtm_proxy1'
listen_addresses = '*'
port = 6667
worker_threads = 2
gtm_host = 'pgxc-gtm1'
gtm_port = 6667
gtm_connect_retry_interval = 1
log_file = 'gtm_proxy.log'
log_min_messages = INFO
启动GTM proxy:
gtm_proxy -h pgxc-n1 -p 6667 -s pgxc-gtm1 -t 6666 -n 2 -D /data/gtm_proxy/ -l /data/gtm_proxy/gtm_proxy.log &
exit
然后,对pgxc-n2节点执行类似步骤来启动该节点上的GTM proxy:
ssh pgxc-n2
vi /data/gtm_proxy/gtm_proxy.conf # 根据需要修改参数配置。
gtm_proxy -h pgxc-n2 -p 6667 -s pgxc-gtm1 -t 6666 -n 2 -D /data/gtm_proxy/ -l /data/gtm_proxy/gtm_proxy.log &
exit
然后,回到pgxc_ctl命令行中,观察所有节点状态:
[pgxc@pgxc-gtm1 ~]$ pgxc_ctl
/bin/bash
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
Installing pgxc_ctl_bash script as /home/pgxc/pgxc_ctl/pgxc_ctl_bash.
Reading configuration using /home/pgxc/pgxc_ctl/pgxc_ctl_bash --home /home/pgxc/pgxc_ctl --configuration /home/pgxc/pgxc_ctl/pgxc_ctl.conf
Finished reading configuration.
******** PGXC_CTL START ***************
Current directory: /home/pgxc/pgxc_ctl
PGXC monitor all
Running: gtm master
Running: gtm slave
Running: gtm proxy gtm_proxy1
Running: gtm proxy gtm_proxy2
Running: coordinator master coord1
Running: coordinator slave coord1
Running: coordinator master coord2
Running: coordinator slave coord2
Running: datanode master data1
Running: datanode slave data1
Running: datanode master data2
Running: datanode slave data2
PGXC
pgxc_ctl使用帮助
pgxc_ctl
工具也可以执行节点删除等工作,使用help COMMAND
查看帮助信息,如下所示:
PGXC help
You are using pgxc_ctl, the configuration utility for PGXL
Type:
help <command>
where <command> is either add, Createdb, Createuser, clean,
configure, deploy, failover, init, kill, log, monitor,
prepare, q, reconnect, remove, set, show, start,
stop or unregister
PGXC help remove
remove gtm slave
remove gtm_proxy nodename [ clean ]
remove coordinator [ master| slave ] nodename [ clean ]
remove datanode [ master| slave ] nodename [ clean ]
Removes the specified node from the cluster
For more details, please see the pgxc_ctl documentation
PGXC