CDH 修改集群IP

最近遇到一个Case：客户原来的Hadoop集群的物理机要迁移至别的地方，这将导致物理机的IP要更换。这让我不禁想到一个问题：CDH Manager页面中的主机IP的信息究竟是保存在哪里的？

后来才明白，CDH Manager在安装的时候会配置相应的数据库，用来保存集群所有的配置信息，因此这里的节点的IP也是保存在这个数据库中，所以如果更换了节点的IP，就必须要更新此数据库的配置信息。具体步骤如下，

1 停止CDH Manager相应的管理进程(Server进程及所有节点的Agent进程)

service cloudera-scm-agent stop
service cloudera-scm-server stop

2 查看对应数据库的scm用户密码

[root@esg06 trafodion]# grep password /etc/cloudera-scm-server/db.properties
com.cloudera.cmf.db.password=aOnrBDz4Bi

3 在CDH Manager管理节点上登录数据库(默认Postgresql)

psql -h localhost -p 7432 -U scm

此时需要输入scm用户的密码，即上述步骤2中的密码

4 查看数据库中的主机IP

scm=> select host_id, host_identifier, name, ip_address from hosts;
 host_id |           host_identifier            |        name         | ip_address
---------+--------------------------------------+---------------------+-------------
       8 | 40bce2b1-d718-467b-b757-49b55ed2fed6 | esg11.esgyncn.local | 10.10.10.11
       6 | f63edca9-b64b-4d03-9339-724d520ffeef | esg08.esgyncn.local | 10.10.10.8
       7 | 9949b2cf-4e09-4f3b-abeb-72a8b53b0124 | esg09.esgyncn.local | 10.10.10.9
       5 | ce21067f-134f-4861-8468-ced2f214bab9 | esg10.esgyncn.local | 10.10.10.10
       4 | 5b965ecf-3414-4896-a654-356a351040f5 | esg15.esgyncn.local | 10.10.10.15
      10 | e251e5f2-9e4e-4210-a237-5fa5f289d8e6 | esg12.esgyncn.local | 10.10.10.12
       1 | 24da548a-063b-46a9-8a2d-0c88881f4aed | esg13.esgyncn.local | 10.10.10.13
       2 | cdec8d37-1206-4e71-b9b3-4fad9c080d5e | esg06.esgyncn.local | 10.10.10.6
       9 | 45b1fd0a-d34b-42a7-a1c8-dc4a07485071 | esg14.esgyncn.local | 10.10.10.14
       3 | 29121630-aa25-4594-82c8-c8ea7896fe9b | esg07.esgyncn.local | 10.10.10.7
(10 rows)

5 依次修改主机IP

update hosts set ip_address='...' where ip_address='';
...

6 退出数据库(:\q)

7 修改所有节点的Agent配置文件

--将server_hosts的值修改为新的CDH Manager管理节点的IP
vi /etc/cloudera-scm-agent/config.ini

8 修改所有节点的/etc/hosts，将IP修改为新的IP

9 重启启动CDH Manager对应的管理进程及Agent服务

service cloudera-scm-server-db restart
service cloudera-scm-server start
service cloudera-scm-agent start

注：以下为元数据库为MYSQL的修改步骤

1. 修改/etc/hosts中的IP为新IP

2. 进入mysql
[root@YXnode01 ~]# /usr/local/mysql/bin/mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 9593
Server version: 5.6.31-enterprise-commercial-advanced-log MySQL Enterprise Server - Advanced Edition (Commercial)

Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

3. 找到hosts表
mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| amon               |
| cm                 |
| hive               |
| mysql              |
| performance_schema |
| rman               |
| test               |
+--------------------+
8 rows in set (0.00 sec)
mysql> use cm
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+--------------------------------+
| Tables_in_cm                   |
+--------------------------------+
| AUDITS                         |
| CLIENT_CONFIGS                 |
| CLIENT_CONFIGS_TO_HOSTS        |
| CLUSTERS                       |
| CLUSTERS_AUD                   |
| CLUSTER_ACTIVATED_RELEASES     |
| CLUSTER_ACTIVATED_RELEASES_AUD |
| CLUSTER_MANAGED_RELEASES       |
| CLUSTER_UNDISTRIBUTED_RELEASES |
| CM_PEERS                       |
| CM_VERSION                     |
| COMMANDS                       |
| COMMANDS_DETAIL                |
| COMMAND_SCHEDULES              |
| CONFIGS                        |
| CONFIGS_AUD                    |
| CONFIG_CONTAINERS              |
| CREDENTIALS                    |
| DIAGNOSTICS_EVENTS             |
| EXTERNAL_ACCOUNTS              |
| EXTERNAL_ACCOUNTS_AUD          |
| GLOBAL_SETTINGS                |
| HOSTS                          |
| HOSTS_AUD                      |
| HOST_TEMPLATES                 |
| HOST_TEMPLATE_TO_ROLE_CONF_GRP |
| METRICS                        |
| PARCELS                        |
| PARCEL_COMPONENTS              |
| PROCESSES                      |
| PROCESSES_DETAIL               |
| PROCESS_ACTIVE_RELEASES        |
| RELEASES                       |
| RELEASES_AUD                   |
| REVISIONS                      |
| ROLES                          |
| ROLES_AUD                      |
| ROLE_CONFIG_GROUPS             |
| ROLE_CONFIG_GROUPS_AUD         |
| ROLE_STALENESS_STATUS          |
| SCHEMA_VERSION                 |
| SERVICES                       |
| SERVICES_AUD                   |
| SNAPSHOT_POLICIES              |
| USERS                          |
| USER_ROLES                     |
| USER_SETTINGS                  |
+--------------------------------+
47 rows in set (0.00 sec)

4. 查询HOSTS
mysql> select * from hosts;
ERROR 1146 (42S02): Table 'cm.hosts' doesn't exist
mysql> select * from HOSTS;
+---------+-------------------------+--------------------------------------+-------------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER                      | NAME              | IP_ADDRESS     | RACK_ID  | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER |
+---------+-------------------------+--------------------------------------+-------------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
|       1 |                      25 | d3c2a68c-8f3a-4b81-bfd9-fddfe91fba9c | YXnode01.esgyn.cn | 172.16.253.117 | /default | NA     |                   1 |                 0 |                  0 |          1 |         8 |          25112600576 | NULL        | NULL              | NULL           |
|       2 |                      21 | 897848be-cbe1-4171-93ef-53750cbfd156 | YXnode03.esgyn.cn | 172.16.253.119 | /default | NA     |                   1 |                 0 |                  0 |          1 |         8 |          16657936384 | NULL        | NULL              | NULL           |
|       3 |                      23 | 41569980-d8db-418e-88fd-26cbd3898499 | YXnode02.esgyn.cn | 172.16.253.118 | /default | NA     |                   1 |                 0 |                  0 |          1 |         8 |          16657936384 | NULL        | NULL              | NULL           |
+---------+-------------------------+--------------------------------------+-------------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
3 rows in set (0.00 sec)

5. 更新HOSTS中的IP
update HOSTS set IP_ADDRESS='172.16.253.217' where IP_ADDRESS='172.16.253.117';
update HOSTS set IP_ADDRESS='172.16.253.218' where IP_ADDRESS='172.16.253.118';
update HOSTS set IP_ADDRESS='172.16.253.219' where IP_ADDRESS='172.16.253.119';

6. 检查是否修改成功
mysql> select * from HOSTS;
+---------+-------------------------+--------------------------------------+-------------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
| HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER                      | NAME              | IP_ADDRESS     | RACK_ID  | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER |
+---------+-------------------------+--------------------------------------+-------------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
|       1 |                      25 | d3c2a68c-8f3a-4b81-bfd9-fddfe91fba9c | YXnode01.esgyn.cn | 172.16.253.217 | /default | NA     |                   1 |                 0 |                  0 |          1 |         8 |          25112600576 | NULL        | NULL              | NULL           |
|       2 |                      21 | 897848be-cbe1-4171-93ef-53750cbfd156 | YXnode03.esgyn.cn | 172.16.253.219 | /default | NA     |                   1 |                 0 |                  0 |          1 |         8 |          16657936384 | NULL        | NULL              | NULL           |
|       3 |                      23 | 41569980-d8db-418e-88fd-26cbd3898499 | YXnode02.esgyn.cn | 172.16.253.218 | /default | NA     |                   1 |                 0 |                  0 |          1 |         8 |          16657936384 | NULL        | NULL              | NULL           |
+---------+-------------------------+--------------------------------------+-------------------+----------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+
3 rows in set (0.00 sec)

7. 修改所有节点的/etc/cloudera-scm-agent/config.ini

8. 重启Cloudera agent及server服务