Impala集群搭建报错,主节点启动成功,但是worker节点的impalad启动失败

1.报错信息如下

impalad.ERROR:

Running on machine: node1
Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
E0124 10:34:09.324137 30280 logging.cc:147] stderr will be logged to this file.
2022-01-24 10:34:10,077 INFO util.JvmPauseMonitor: Starting JVM pause monitor
E0124 10:34:38.390982 30280 impalad-main.cc:90] Impalad services did not start correctly, exiting. Error: Couldn't open transport for localhost:24000 (connect() failed: Connection refused)
Statestore subscriber did not start up.

impalad.INFO:

Log line format: [IWEF]mmdd hh:mm:ss.uuuuuu threadid file:line] msg
I0124 10:34:09.324054 30280 logging.cc:146] stdout will be logged to this file.
E0124 10:34:09.324137 30280 logging.cc:147] stderr will be logged to this file.
I0124 10:34:09.324265 30280 minidump.cc:238] Setting minidump size limit to 20971520.
I0124 10:34:09.327106 30280 authentication.cc:1016] Internal communication is not authenticated
I0124 10:34:09.327117 30280 authentication.cc:1037] External communication is not authenticated
I0124 10:34:09.327536 30280 init.cc:276] impalad version 3.2.0-cdh6.2.1 RELEASE (build 525e372410dd2ce206e2ad0f21f57cae7380c0cb)
Built on Mon Aug 3 07:00:55 PDT 2020
I0124 10:34:09.327546 30280 init.cc:277] Using hostname: node1
I0124 10:34:09.327926 30280 logging.cc:182] Flags (see also /varz are on debug webserver):
–catalog_service_port=26000
–catalog_topic_mode=full
–initial_hms_cnxn_timeout_s=120
–load_catalog_in_background=false
–max_hdfs_partitions_parallel_load=5
–max_nonhdfs_partitions_parallel_load=20
–num_metadata_loading_threads=16
–sentry_catalog_polling_frequency_s=60
–sentry_config=
–asm_module_dir=
–disable_optimization_passes=false
–dump_ir=false
–opt_module_dir=
–perf_map=false
–print_llvm_ir_instruction_count=false
–unopt_module_dir=
–abort_on_config_error=true
…skipping…
I0124 10:34:11.377549 30280 statestore-subscriber.cc:206] Starting statestore subscriber
I0124 10:34:11.378587 30512 TAcceptQueueServer.cpp:291] connection_setup_thread_pool_size is set to 2
I0124 10:34:11.378609 30280 thrift-server.cc:456] ThriftServer ‘StatestoreSubscriber’ started on port: 23000
I0124 10:34:11.378621 30280 statestore-subscriber.cc:234] Registering with statestore
I0124 10:34:11.379050 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:11.379060 30280 thrift-client.cc:100] (Attempt 1 of 10)
I0124 10:34:14.380275 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:14.380295 30280 thrift-client.cc:100] (Attempt 2 of 10)
I0124 10:34:17.380971 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:17.380996 30280 thrift-client.cc:100] (Attempt 3 of 10)
I0124 10:34:20.381692 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:20.381718 30280 thrift-client.cc:100] (Attempt 4 of 10)
I0124 10:34:23.383141 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:23.383163 30280 thrift-client.cc:100] (Attempt 5 of 10)
I0124 10:34:26.384608 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:26.384630 30280 thrift-client.cc:100] (Attempt 6 of 10)
I0124 10:34:29.386202 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:29.386227 30280 thrift-client.cc:100] (Attempt 7 of 10)
I0124 10:34:32.386914 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:32.386940 30280 thrift-client.cc:100] (Attempt 8 of 10)
I0124 10:34:35.389108 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:35.389130 30280 thrift-client.cc:100] (Attempt 9 of 10)
I0124 10:34:38.390767 30280 thrift-client.cc:94] Unable to connect to localhost:24000
I0124 10:34:38.390789 30280 thrift-client.cc:100] (Attempt 10 of 10)
I0124 10:34:38.390807 30280 statestore-subscriber.cc:240] statestore registration unsuccessful: Couldn't open transport for localhost:24000 (connect() failed: Connection refused)
E0124 10:34:38.390982 30280 impalad-main.cc:90] Impalad services did not start correctly, exiting. Error: Couldn't open transport for localhost:24000 (connect() failed: Connection refused)
Statestore subscriber did not start up.

反正就是worker节点的impalad启动失败
这很明显是impala配置的问题,由于我是rpm包安装的
所以默认配置是在 /etc/default/impala 文件
如下:

IMPALA_CATALOG_SERVICE_HOST=node3
IMPALA_STATE_STORE_HOST=node3
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala
KUDU_NODES=nod1:7051,node2:7051,node3:7051

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} "
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -use_statestore \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -be_port=${IMPALA_BACKEND_PORT} \
    -kudu_master_hosts=${KUDU_NODES}"

ENABLE_CORE_DUMPS=false

# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
# IMPALA_HOME=/usr/lib/impala
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf
# HIVE_CONF_DIR=/etc/impala/conf

这个是最开始配置的,报错,然后修改后配置如下

IMPALA_CATALOG_SERVICE_HOST=node3
IMPALA_STATE_STORE_HOST=node3
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/var/log/impala
KUDU_NODES=nod1:7051,node2:7051,node3:7051

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_host=${IMPALA_STATE_STORE_HOST} "
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_host=${IMPALA_STATE_STORE_HOST} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -use_statestore \
    -be_port=${IMPALA_BACKEND_PORT} \
    -kudu_master_hosts=${KUDU_NODES}"

ENABLE_CORE_DUMPS=false

# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
# IMPALA_HOME=/usr/lib/impala
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf
# HIVE_CONF_DIR=/etc/impala/conf
# HBASE_CONF_DIR=/etc/impala/conf

其实就是在IMPALA_CATALOG_ARGS, IMPALA_STATE_STORE_ARGS
上加了 -state_store_host=${IMPALA_STATE_STORE_HOST} \
然后在IMPALA_SERVER_ARGS 上把 -state_store_host=${IMPALA_STATE_STORE_HOST} \ 移到-state_store_port
之前就可以了

重点:

经过后面测试发现, 只需要在IMPALA_SERVER_ARGS 上把 -state_store_host=${IMPALA_STATE_STORE_HOST} \ 移到-state_store_port
之前就可以了,不需要上面每个都加.