CDH报错 agent异常

在我添加agen节点的时候报错， 查看日志是报错信息如下 

查看agent 节点日志

tail -f  /opt/cloudera-manager/cm-5.8.0/log/cloudera-scm-agent/cloudera-scm-agent.log 
 

 

 

 

[18/Dec/2018 13:53:44 +0000] 1686 MainThread agent        ERROR    Failed to handle Heartbeat Response: {u'firehoses': [{u'roletype': u'SERVICEMONITOR', u'rolename': u'mgmt-SERVICEMONITOR-8b7
30cb62f5e53e3e0421dd1c6d53975', u'port': 9997, u'report_interval': 60, u'address': u'master'}, {u'roletype': u'HOSTMONITOR', u'rolename': u'mgmt-HOSTMONITOR-8b730cb62f5e53e3e0421dd1c6d53975',
 u'port': 9995, u'report_interval': 60, u'address': u'master'}], u'rm_enabled': False, u'client_configs': [], u'create_parcel_symlinks': True, u'server_managed_parcels': [], u'extra_configs':
 None, u'host_collection_config_data': [{u'config_name': u'host_network_interface_collection_filter', u'config_value': u'^lo$'}, {u'config_name': u'host_disk_collection_filter', u'config_valu
e': u'^$'}, {u'config_name': u'host_fs_collection_filter', u'config_value': u'^$'}, {u'config_name': u'host_log_tailing_config', u'config_value': u'{}\n'}, {u'config_name': u'host_dns_resolut
ion_duration_thresholds', u'config_value': u'{"critical":"never","warning":"1000.0"}'}, {u'config_name': u'host_dns_resolution_enabled', u'config_value': u'true'}, {u'config_name': u'host_clo
ck_offset_thresholds', u'config_value': u'{"warning":30000,"critical":60000}'}], u'apply_parcel_users_groups_permissions': True, u'flood_torrent_port': 7191, u'log_tailing_config': u'{}\n', u
'active_parcels': {}, u'flood_rack_peers': [u'192.168.1.183:7191', u'192.168.1.184:7191', u'192.168.1.182:7191', u'192.168.1.181:7191'], u'retain_parcels_in_cache': True, u'processes': [], u'
server_manages_parcels': True, u'heartbeat_interval': 15, u'parcels_directory': u'/opt/cloudera/parcels', u'host_id': u'85a8796e-d5e0-42ee-a0d7-ea89eb0ac09e', u'eventserver_host': u'master', 
u'enabled_metric_reporters': [u'ACCUMULO-ACCUMULO_MASTER', u'ACCUMULO_MASTER', u'ZOOKEEPER-SERVER', u'SERVER', u'SENTRY', u'SENTRY', u'SPARK-SPARK_HISTORY_SERVER', u'SPARK_HISTORY_SERVER', u'
HUE-KT_RENEWER', u'KT_RENEWER', u'ISILON', u'ISILON', u'MAPREDUCE-JOBTRACKER', u'JOBTRACKER', u'SQOOP', u'SQOOP', u'HUE-BEESWAX_SERVER', u'BEESWAX_SERVER', u'KAFKA-KAFKA_BROKER', u'KAFKA_BROK
ER', u'HDFS-NFSGATEWAY', u'NFSGATEWAY', u'KAFKA-KAFKA_MIRROR_MAKER', u'KAFKA_MIRROR_MAKER', u'KEYTRUSTEE_SERVER-DB_ACTIVE', u'DB_ACTIVE', u'HDFS-JOURNALNODE', u'JOURNALNODE', u'HBASE-HBASETHR
IFTSERVER', u'HBASETHRIFTSERVER', u'MGMT-EVENTSERVER', u'EVENTSERVER', u'HDFS-NAMENODE', u'NAMENODE', u'OOZIE', u'OOZIE', u'FLUME-AGENT', u'AGENT', u'HUE-HUE_LOAD_BALANCER', u'HUE_LOAD_BALANC
ER', u'KEYTRUSTEE_SERVER-DB_PASSIVE', u'DB_PASSIVE', u'ACCUMULO16-ACCUMULO16_TRACER', u'ACCUMULO16_TRACER', u'HBASE', u'HBASE', u'SENTRY-SENTRY_SERVER', u'SENTRY_SERVER', u'SQOOP-SQOOP_SERVER
', u'SQOOP_SERVER', u'MGMT-REPORTSMANAGER', u'REPORTSMANAGER', u'ACCUMULO16-ACCUMULO16_TSERVER', u'ACCUMULO16_TSERVER', u'MGMT', u'MGMT', u'IMPALA-CATALOGSERVER', u'CATALOGSERVER', u'MAPREDUC
E-FAILOVERCONTROLLER', u'FAILOVERCONTROLLER', u'IMPALA-LLAMA', u'LLAMA', u'IMPALA', u'IMPALA', u'HBASE-HBASERESTSERVER', u'HBASERESTSERVER', u'MGMT-ACTIVITYMONITOR', u'ACTIVITYMONITOR', u'HOS
T', u'MGMT-ALERTPUBLISHER', u'ALERTPUBLISHER', u'MGMT-SERVICEMONITOR', u'SERVICEMONITOR', u'MGMT-NAVIGATORMETASERVER', u'NAVIGATORMETASERVER', u'MAPREDUCE-TASKTRACKER', u'TASKTRACKER', u'SPAR
K', u'SPARK', u'KS_INDEXER', u'KS_INDEXER', u'MAPREDUCE', u'MAPREDUCE', u'YARN-JOBHISTORY', u'JOBHISTORY', u'SPARK-SPARK_MASTER', u'SPARK_MASTER', u'IMPALA-IMPALAD', u'IMPALAD', u'HDFS-DSSDDA
TANODE', u'DSSDDATANODE', u'KEYTRUSTEE_SERVER-KEYTRUSTEE_ACTIVE_SERVER', u'KEYTRUSTEE_ACTIVE_SERVER', u'SPARK-SPARK_WORKER', u'SPARK_WORKER', u'YARN', u'YARN', u'HIVE-WEBHCAT', u'WEBHCAT', u'
YARN-NODEMANAGER', u'NODEMANAGER', u'HDFS-HTTPFS', u'HTTPFS', u'ACCUMULO-ACCUMULO_LOGGER', u'ACCUMULO_LOGGER', u'HDFS-SECONDARYNAMENODE', u'SECONDARYNAMENODE', u'SOLR-SOLR_SERVER', u'SOLR_SER
VER', u'ACCUMULO-ACCUMULO_MONITOR', u'ACCUMULO_MONITOR', u'ACCUMULO-ACCUMULO_TSERVER', u'ACCUMULO_TSERVER', u'KMS-KMS', u'KMS', u'SPARK_ON_YARN', u'SPARK_ON_YARN', u'ACCUMULO16-ACCUMULO16_MON
ITOR', u'ACCUMULO16_MONITOR', u'HBASE-REGIONSERVER', u'REGIONSERVER', u'HUE-HUE_SERVER', u'HUE_SERVER', u'HIVE-HIVEMETASTORE', u'HIVEMETASTORE', u'ZOOKEEPER', u'ZOOKEEPER', u'IMPALA-STATESTOR
E', u'STATESTORE', u'HBASE-MASTER', u'MASTER', u'ACCUMULO16', u'ACCUMULO16', u'KAFKA', u'KAFKA', u'SQOOP_CLIENT', u'SQOOP_CLIENT', u'FLUME', u'FLUME', u'KS_INDEXER-HBASE_INDEXER', u'HBASE_IND
EXER', u'KEYTRUSTEE-KMS_KEYTRUSTEE', u'KMS_KEYTRUSTEE', u'ACCUMULO-ACCUMULO_GC', u'ACCUMULO_GC', u'SPARK2_ON_YARN', u'SPARK2_ON_YARN', u'MGMT-NAVIGATOR', u'NAVIGATOR', u'KMS', u'KMS', u'KEYTR
USTEE_SERVER', u'KEYTRUSTEE_SERVER', u'SPARK2_ON_YARN-SPARK2_YARN_HISTORY_SERVER', u'SPARK2_YARN_HISTORY_SERVER', u'HDFS-FAILOVERCONTROLLER', u'FAILOVERCONTROLLER', u'KEYTRUSTEE_SERVER-KEYTRU
STEE_PASSIVE_SERVER', u'KEYTRUSTEE_PASSIVE_SERVER', u'HUE', u'HUE', u'MGMT-HOSTMONITOR', u'HOSTMONITOR', u'ACCUMULO-ACCUMULO_TRACER', u'ACCUMULO_TRACER', u'SOLR', u'SOLR', u'ACCUMULO', u'ACCU
MULO', u'HIVE', u'HIVE', u'HDFS', u'HDFS', u'ACCUMULO16-ACCUMULO16_GC', u'ACCUMULO16_GC', u'KEYTRUSTEE', u'KEYTRUSTEE', u'OOZIE-OOZIE_SERVER', u'OOZIE_SERVER', u'YARN-RESOURCEMANAGER', u'RESO
URCEMANAGER', u'SPARK_ON_YARN-SPARK_YARN_HISTORY_SERVER', u'SPARK_YARN_HISTORY_SERVER', u'ACCUMULO16-ACCUMULO16_MASTER', u'ACCUMULO16_MASTER', u'HIVE-HIVESERVER2', u'HIVESERVER2', u'HDFS-DATA
NODE', u'DATANODE'], u'flood_seed_timeout': 95, u'eventserver_port': 7185}
Traceback (most recent call last):
  File "/opt/cloudera-manager/cm-5.8.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.8.0-py2.6.egg/cmf/agent.py", line 1337, in handle_heartbeat_response
    self._handle_heartbeat_response(response)
  File "/opt/cloudera-manager/cm-5.8.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.8.0-py2.6.egg/cmf/agent.py", line 1359, in _handle_heartbeat_response
    response["flood_torrent_port"])
  File "/opt/cloudera-manager/cm-5.8.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.8.0-py2.6.egg/cmf/agent.py", line 1789, in handle_heartbeat_flood
    flood_dir)
  File "/opt/cloudera-manager/cm-5.8.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.8.0-py2.6.egg/cmf/downloader.py", line 179, in __init__
    self.client = TorrentRPCClient(server)
  File "/opt/cloudera-manager/cm-5.8.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.8.0-py2.6.egg/flood/util/rpc.py", line 118, in __init__
    server = resolveHostPort(server, self.DEFAULT_RPC_PORT)
  File "/opt/cloudera-manager/cm-5.8.0/lib64/cmf/agent/build/env/lib/python2.6/site-packages/cmf-5.8.0-py2.6.egg/flood/util/rpc.py", line 33, in resolveHostPort
    return HostPort(socket.gethostbyname(w[0]), port)
gaierror: [Errno -2] Name or service not known
 

 

经过分析，原因可能是由于无法和server 通信，导致 报错异常

 

解决办法：

 vi /etc/hosts
添加：
127.0.0.1    localhost.localdomain   localhost
::1   localhost6.localdomain6 localhost6
127.0.0.1 localhost.node1

如图：

重启 网卡

service  network restart
 

重启agent

/opt/cloudera-manager/cm-5.8.0/etc/init.d/cloudera-scm-agent stop

/opt/cloudera-manager/cm-5.8.0/etc/init.d/cloudera-scm-agent start

查看日志信息

 

tail -f  /opt/cloudera-manager/cm-5.8.0/log/cloudera-scm-agent/cloudera-scm-agent.log