KuduMaster迁移
主数量必须为奇数。
1.添加Kudu Master
-
获取现有master的uuid
sudo -u kudu kudu fs dump uuid --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 2>/dev/null 5315431da71e44a8ba9318330366563c
-
新增Kudu Master角色
-
停止所有Kudu进程(备份所有KuduMaster数据(也可以不备份,实施过程中Kudu会自动备份raft文件))
-
格式化每台新主计算机上的数据目录,并记录生成的UUID
mkdir -p /kudu/master/fs_wal_dir && mkdir -p /kudu/master/fs_data_dirs && chown -R kudu:kudu /kudu/master/fs_wal_dir && chown -R kudu:kudu /kudu/master/fs_data_dirs && chmod 700 /kudu/master/fs_wal_dir && chmod 700 /kudu/master/fs_data_dirs sudo -u kudu kudu fs format --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs sudo -u kudu kudu fs dump uuid --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 2>/dev/null
- 第1台新增节点执行结果
[root@cdh02:/root] # sudo -u kudu kudu fs format --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs I0717 11:31:04.182476 32125 fs_manager.cc:567] Generated new instance metadata in path /kudu/master/fs_data_dirs/instance: uuid: "5b36178fe35241e4b9ef734944c733e9" format_stamp: "Formatted at 2019-07-17 03:31:04 on cdh02" I0717 11:31:04.182989 32125 fs_manager.cc:567] Generated new instance metadata in path /kudu/master/fs_wal_dir/instance: uuid: "5b36178fe35241e4b9ef734944c733e9" format_stamp: "Formatted at 2019-07-17 03:31:04 on cdh02" I0717 11:31:04.184741 32125 fs_manager.cc:471] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s [root@cdh02:/root] # [root@cdh02:/root] # sudo -u kudu kudu fs dump uuid --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 2>/dev/null 5b36178fe35241e4b9ef734944c733e9
- 第2台新增节点执行结果
[root@cdh03:/root] # mkdir -p /kudu/master/fs_wal_dir && mkdir -p /kudu/master/fs_data_dirs && chown -R kudu:kudu /kudu/master/fs_wal_dir && chown -R kudu:kudu /kudu/master/fs_data_dirs && chmod 700 /kudu/master/fs_wal_dir && chmod 700 /kudu/master/fs_data_dirs sudo -u kudu kudu fs format --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs sudo -u kudu kudu fs dump uuid --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 2>/dev/null [root@cdh03:/root] # sudo -u kudu kudu fs format --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs I0717 11:32:10.213480 30641 fs_manager.cc:567] Generated new instance metadata in path /kudu/master/fs_data_dirs/instance: uuid: "d17c7304ff8f4a22b9c7d547dc540268" format_stamp: "Formatted at 2019-07-17 03:32:10 on cdh03" I0717 11:32:10.214143 30641 fs_manager.cc:567] Generated new instance metadata in path /kudu/master/fs_wal_dir/instance: uuid: "d17c7304ff8f4a22b9c7d547dc540268" format_stamp: "Formatted at 2019-07-17 03:32:10 on cdh03" I0717 11:32:10.216106 30641 fs_manager.cc:471] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s [root@cdh03:/root] # sudo -u kudu kudu fs dump uuid --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 2>/dev/null d17c7304ff8f4a22b9c7d547dc540268
- 第3台新增节点执行结果
# mkdir -p /kudu/master/fs_wal_dir && mkdir -p /kudu/master/fs_data_dirs && chown -R kudu:kudu /kudu/master/fs_wal_dir && chown -R kudu:kudu /kudu/master/fs_data_dirs && chmod 700 /kudu/master/fs_wal_dir && chmod 700 /kudu/master/fs_data_dirs sudo -u kudu kudu fs format --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs sudo -u kudu kudu fs dump uuid --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 2>/dev/null [root@cdh04:/root] # sudo -u kudu kudu fs format --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs I0717 11:33:01.965132 29273 fs_manager.cc:567] Generated new instance metadata in path /kudu/master/fs_data_dirs/instance: uuid: "6e61d6b48ff04201802e76be355fbaa8" format_stamp: "Formatted at 2019-07-17 03:33:01 on cdh04" I0717 11:33:01.965773 29273 fs_manager.cc:567] Generated new instance metadata in path /kudu/master/fs_wal_dir/instance: uuid: "6e61d6b48ff04201802e76be355fbaa8" format_stamp: "Formatted at 2019-07-17 03:33:01 on cdh04" I0717 11:33:01.967491 29273 fs_manager.cc:471] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s [root@cdh04:/root] # sudo -u kudu kudu fs dump uuid --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 2>/dev/null 6e61d6b48ff04201802e76be355fbaa8
-
在现有的Kudu Master主机上执行
sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 00000000000000000000000000000000 5315431da71e44a8ba9318330366563c:cdh01:7051 5b36178fe35241e4b9ef734944c733e9:cdh02:7051 d17c7304ff8f4a22b9c7d547dc540268:cdh03:7051 6e61d6b48ff04201802e76be355fbaa8:cdh04:7051
-
CM上启动现有Kudu Master主机
-
使用以下命令将主数据复制到每个新主服务器,在每台新主计算机上执行。
sudo -u kudu kudu local_replica copy_from_remote --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 00000000000000000000000000000000 cdh01:7051
-
启动所有添加的新KuduMaster
2.删除KuduMaster节点
-
CM页面停止所有Kudu服务
-
CM删除不需要的KuduMaster节点
-
在所有其他KuduMaster节点上执行
sudo -u kudu kudu local_replica cmeta rewrite_raft_config --fs_wal_dir=/kudu/master/fs_wal_dir --fs_data_dirs=/kudu/master/fs_data_dirs 00000000000000000000000000000000 5b36178fe35241e4b9ef734944c733e9:cdh02:7051 d17c7304ff8f4a22b9c7d547dc540268:cdh03:7051 6e61d6b48ff04201802e76be355fbaa8:cdh04:7051
-
启动所有剩余的KuduMaster服务
-
启动所有的KuduTabletServer服务
-
修改mysql元数据
UPDATE TABLE_PARAMS SET PARAM_VALUE = 'cdh02,cdh03,cdh04' WHERE PARAM_KEY = 'kudu.master_addresses' AND PARAM_VALUE = 'cdh01';
-
重启Impala的Impala Catalog Server
-
确定服务成功启动后删除旧KuduMaster数据目录
Remove the data directories and WAL directory on the unwanted masters. This is a precaution to ensure that they cannot start up again and interfere with the new multi-master deployment.
-
检查迁移结果
sudo -u kudu kudu cluster ksck cdh02,cdh03,cdh04