zeppelin 从 0.6.1 开始支持 spark2.x ,从0.6.2编译时引入了hadoop-common包用于权限认证,所以会存在一些包冲突导致异常的问题。
编译错误
zeppelin-web编译错误
Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.25:grunt (grunt test) on project zeppelin-web: Failed to run task: 'grunt test --no-color' failed. (error code 3) -> [Help 1]
修改zepelin-web下的pom文件,删除以下内容
<execution>
<id>grunt test</id>
<goals>
<goal>grunt</goal>
</goals>
<phase>test</phase>
<configuration>
<arguments>test</arguments>
</configuration>
</execution>
运行时报错
启动spark interpreter报错,hadoop包冲突
错误提示1:
java.lang.NoSuchMethodError: org.apache.hadoop.fs.FSOutputSummer.<init>(Ljava/util/zip/Checksum;II)V
错误提示2:
java.lang.NoSuchMethodError: org.apache.hadoop.tracing.SpanReceiverHost.get(Lorg/apache/hadoop/conf/Configuration;Ljava/lang/String;)Lorg/apache/hadoop/tracing/SpanReceiverHost; at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:634) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2596)
自带了hadoop2.6的common包与spark的hadoop包冲突。使用spark的包替换即可。
cd zeppelin/lib rm hadoop-annotations-2.6.0.jar rm hadoop-auth-2.6.0.jar rm hadoop-common-2.6.0.jar cp spark/jars/hadoop-annotations-2.4.1.jar ./ cp spark/jars/hadoop-auth-2.4.1.jar ./ cp spark/jars/hadoop-common-2.4.1.jar ./
运行spark任务报错,jackson包冲突
报错提示:
com.fasterxml.jackson.databind.JsonMappingException: Jackson version is too old 2.5.3
解决方法:
cd zeppelin/lib rm jackson-annotations-2.5.0.jar rm jackson-core-2.5.3.jar rm jackson-databind-2.5.3.jar cp spark/jars/jackson-databind-2.6.5.jar ./ cp spark/jars/jackson-core-2.6.5.jar ./ cp spark/jars/jackson-annotations-2.6.5.jar ./