一: 问题描述:
CDH 5.14部署好spark2后,写了个WC统计程序,并写了如下执行语句:
$SPARK_HOME/bin/spark2-submit \
--master local[2] \
--class com.weizonggui.core03.SparkContextApp \
--name WCApp \
/home/hadoop/G5-Spark-1.0.jar \
hdfs://hadoop002:8020/logs/input/text.txt hdfs://hadoop002:8020/logs/output
执行失败:Caused by: java.net.UnknownHostException: nameservice1
二:问题分析:
参考下一篇博文:
三:问题解决:
java.net.UnknownHostException: nameservice1异常,找不到正确的路径:
cp hdfs-site.xml /opt/cloudera/parcels/SPARK2/lib/spark2/conf
首选明白:hadoop的配置路径:/opt/cloudera/parcels/CDH/lib/hadoop/etc/hadoop
spark2的配置路径: /opt/cloudera/parcels/SPARK2/lib/spark2/conf
相应配置如下:
<?xml version="1.0" encoding="UTF-8"?>
<!--Au