GBase 8a 集群扩容

扩容分为 新增调度节点、新增数据节点、新增混合节点，本次实验属于新增数据节点，其他两种情况后面有时间再补充 

原始集群只有一个节点，即是调度节点又是数据节点，需要新增一个数据节点

原始集群192.168.0.92   【gbase@hw-00】

新增节点192.168.0.213 【gbase@hw-01】

原始集群

[gbase@hw-00 gcinstall]$ gcadmin
CLUSTER STATE:         ACTIVE

====================================
| GBASE GCWARE CLUSTER INFORMATION |
====================================
| NodeName |  IpAddress   | gcware |
------------------------------------
| gcware1  | 192.168.0.92 |  OPEN  |
------------------------------------
======================================================
|       GBASE COORDINATOR CLUSTER INFORMATION        |
======================================================
|   NodeName   |  IpAddress   | gcluster | DataState |
------------------------------------------------------
| coordinator1 | 192.168.0.92 |   OPEN   |     0     |
------------------------------------------------------
==============================================
|     GBASE VIRTUAL CLUSTER INFORMATION      |
==============================================
|    VcName    | DistributionId |  comment   |
----------------------------------------------
|     vc1      |       1        | vc1comment |
----------------------------------------------

1 virtual cluster: vc1
1 coordinator node
0 free data node

编辑配置文件 新增节点

[gbase@hw-00 gcinstall]$ cd /opt/gcinstall/
[gbase@hw-00 gcinstall]$ vi demo.options 

installPrefix= /opt/gbase
#coordinateHost = 192.168.0.92                  原有的配置 注释掉
#coordinateHostNodeID = 234,235,237       原有的配置 注释掉
dataHost = 192.168.0.213                            #这是新增的节点IP
existCoordinateHost = 192.168.0.92
existDataHost = 192.168.0.92
existGcwareHost= 192.168.0.92
#gcwareHost = 192.168.0.92                       原有的配置 注释掉
#gcwareHostNodeID = 92                            原有的配置 注释掉
dbaUser = gbase
dbaGroup = gbase
dbaPwd = 'gbase'
rootPwd = 'Aa112211.'
#dbRootPwd = ''
#rootPwdFile = rootPwd.json
#characterSet = utf8
#sshPort = 22

新节点设置环境变量

[gbase@hw-00 gcinstall]$ scp SetSysEnv.py root@123.249.4.215:/opt

[root@hw-01 opt]# python SetSysEnv.py --dbaUser=gbase --installPrefix=/opt/gbase --cgroup

新增节点安装gbase

[gbase@hw-00 gcinstall]$ ./gcinstall.py --silent=demo.options

检查是否多了freenode

[gbase@hw-00 gcinstall]$ gcadmin
CLUSTER STATE:         ACTIVE

====================================
| GBASE GCWARE CLUSTER INFORMATION |
====================================
| NodeName |  IpAddress   | gcware |
------------------------------------
| gcware1  | 192.168.0.92 |  OPEN  |
------------------------------------
======================================================
|       GBASE COORDINATOR CLUSTER INFORMATION        |
======================================================
|   NodeName   |  IpAddress   | gcluster | DataState |
------------------------------------------------------
| coordinator1 | 192.168.0.92 |   OPEN   |     0     |
------------------------------------------------------
==============================================
|     GBASE VIRTUAL CLUSTER INFORMATION      |
==============================================
|    VcName    | DistributionId |  comment   |
----------------------------------------------
|     vc1      |       1        | vc1comment |
----------------------------------------------
==============================================================
|          GBASE CLUSTER FREE DATA NODE INFORMATION          |
==============================================================
| NodeName  |   IpAddress   | gnode | syncserver | DataState |
--------------------------------------------------------------
| FreeNode1 | 192.168.0.213 | OPEN  |    OPEN    |     0     |
--------------------------------------------------------------

1 virtual cluster: vc1
1 coordinator node
1 free data node

 将FreeNode添加到VC 

[gbase@hw-00 gcinstall]$ cp gcChangeInfo.xml gcChangeInfo_add.xml
[gbase@hw-00 gcinstall]$ vi gcChangeInfo_add.xml 
[gbase@hw-00 gcinstall]$ gcadmin addnodes gcChangeInfo_add.xml vc1
gcadmin add nodes ...

flush statemachine success

gcadmin addnodes to vc [vc1] success

<?xml version="1.0" encoding="utf-8"?>
<servers>
    <rack>
        <node ip="192.168.0.213"/>
    </rack>
</servers>

查看当前VC节点状态

[gbase@hw-00 gcinstall]$ gcadmin showcluster vc vc1
CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL

==============================================
|     GBASE VIRTUAL CLUSTER INFORMATION      |
==============================================
|    VcName    | DistributionId |  comment   |
----------------------------------------------
|     vc1      |       1        | vc1comment |
----------------------------------------------
=========================================================================================================
|                                 VIRTUAL CLUSTER DATA NODE INFORMATION                                 |
=========================================================================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------------------------------------------------
|  node1   |               192.168.0.92               |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node2   |              192.168.0.213               |                | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------

2 data node

将扩容后的集群创建新distribution

将所有节点写进配置文件

[gbase@hw-00 gcinstall]$ vi gcChangeInfo.xml

<?xml version="1.0" encoding="utf-8"?>
<servers>
    <rack>
        <node ip="192.168.0.92"/>
    </rack>
    <rack>
        <node ip="192.168.0.213"/>
    </rack>
</servers>

创建distribution

[gbase@hw-00 gcinstall]$ gcadmin distribution gcChangeInfo.xml p 1 d 0 vc vc1 dba_os_password gbase  这里是操作系统gbase的密码
gcadmin generate distribution ...

[warning]: parameter [d num] is 0, the new distribution will has no segment backup
please ensure this is ok, input [Y,y] or [N,n]: y
check distribution 1 data node os password
check new distribution data node os password
copy system table to 192.168.0.213
gcadmin generate distribution successful

 初始化hashmap

[gbase@hw-00 gcinstall]$ gccli -uroot -p
gbase> use vc vc1;
Query OK, 0 rows affected (Elapsed: 00:00:00.00)

gbase> initnodedatamap;
Query OK, 0 rows affected, 3 warnings (Elapsed: 00:00:00.22)

gbase> rebalance instance;
Query OK, 0 rows affected (Elapsed: 00:00:00.03)

gbase> select index_name, status, percentage from gclusterdb.rebalancing_status;
Empty set (Elapsed: 00:00:00.00)



如果reblance失败可以试试指定distribution 做reblance
rebalance 默认往最新的 distribution 上分布，也可以通过 to distributionid 语法往指定拓扑上分布
gbase> rebalance instance to 2;
gbase> rebalance table testdis to 2;

删除旧distribution

确认当前distribution ID

[gbase@hw-00 gcinstall]$ gcadmin showdistribution vc vc1

                                 Distribution ID: 2 | State: new | Total segment num: 2

             Primary Segment Node IP                   Segment ID                 Duplicate Segment node IP
========================================================================================================================
|                  192.168.0.92                  |         1          |                                                |
------------------------------------------------------------------------------------------------------------------------
|                 192.168.0.213                  |         2          |                                                |
========================================================================================================================

                                 Distribution ID: 1 | State: old | Total segment num: 1

             Primary Segment Node IP                   Segment ID                 Duplicate Segment node IP
========================================================================================================================
|                  192.168.0.92                  |         1          |                                                |
========================================================================================================================

确认当前集群中没有使用旧的 Distribution ID (1) 的表

gbase> select index_name,tbname,data_distribution_id,vc_id from gbase.table_distribution;
+-------------------------------+--------------------+----------------------+---------+
| index_name                    | tbname             | data_distribution_id | vc_id   |
+-------------------------------+--------------------+----------------------+---------+
| gclusterdb.rebalancing_status | rebalancing_status |                    2 | vc00001 |
| gclusterdb.dual               | dual               |                    2 | vc00001 |
+-------------------------------+--------------------+----------------------+---------+
2 rows in set (Elapsed: 00:00:00.00)

删除旧的distribution1

[gbase@hw-00 gcinstall]$ gcadmin rmdistribution 1 vc vc1
cluster distribution ID [1]
it will be removed now
please ensure this is ok, input [Y,y] or [N,n]: y
select count(*) from gbase.nodedatamap where data_distribution_id=1 result is not 0
refreshnodedatamap drop 1 success
gcadmin remove distribution [1] success
[gbase@hw-00 gcinstall]$ gcadmin showdistribution vc vc1

                                 Distribution ID: 2 | State: new | Total segment num: 2

             Primary Segment Node IP                   Segment ID                 Duplicate Segment node IP
========================================================================================================================
|                  192.168.0.92                  |         1          |                                                |
------------------------------------------------------------------------------------------------------------------------
|                 192.168.0.213                  |         2          |                                                |
========================================================================================================================
[gbase@hw-00 gcinstall]$ gcadmin showdistribution

                                 Distribution ID: 2 | State: new | Total segment num: 2

             Primary Segment Node IP                   Segment ID                 Duplicate Segment node IP
========================================================================================================================
|                  192.168.0.92                  |         1          |                                                |
------------------------------------------------------------------------------------------------------------------------
|                 192.168.0.213                  |         2          |                                                |
========================================================================================================================

补充一句：

建库  -->  distribution（包含哪些节点）  -> hashnodemap (hash与节点的映射关系)  -->reblance (数据重分布到新的节点图)

无VC版本扩容

[gbaseins@8a-111 gcinstall]$ vi demo.options  增加112节点 注掉无用信息
[gbaseins@8a-111 gcinstall]$ ./gcinstall.py --silent=demo.options

Starting all gcluster nodes ...
adding new datanodes to gcware ...
ExtendCluster Successfully

[gbaseins@8a-111 gcinstall]$ cp gcChangeInfo.xml gcChangeInfo_add.xml
[gbaseins@8a-111 gcinstall]$ vi gcChangeInfo_add.xml  所有节点都写进去
[gbaseins@8a-111 gcinstall]$ gcadmin distribution gcChangeInfo.xml p 1 d 1 dba_os_password gbase
[gbaseins@8a-111 gcinstall]$ gccli -uroot
	gbase> initnodedatamap;
	gbase> rebalance instance to 2;
	gbase> select index_name,tbname,data_distribution_id,vc_id from gbase.table_distribution;  保证旧的distribution ID 没有用到
[gbaseins@8a-111 gcinstall]$ gcadmin rmdistribution 1
[gbaseins@8a-111 gcinstall]$ gcadmin showdistribution