CDH5.13启用Kerberos后spark代码提交常见错误

错误一:Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=admin, access=WRITE, inode="/user":hdfs:supergroup:drwxr-xr-x

解决方法:需要登录kinit spark

错误二:WARN client.StandaloneAppClient$ClientEndpoint: Failed to connect to master 192.168.xxx.xxx:7070

解决方法:检查代码“.setMaster("yarn-client")”。master应指定为yarn-client或yarn,Standalone模式的Spark集群不支持访问Kerberos认证的Hadoop,需使用Yarn模式管理的的Spark集群

错误三:Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataInputStream

解决方法:尝试提交命令改为spark2-submit --class xx.xx xx.jar

展开阅读全文

没有更多推荐了,返回首页