前段时间接到一个导数据的任务。由于原来负责这块儿的同事被调到另外的项目去了,而我手里一时也没有合适的人,就决定自己弄吧。把程序克隆下来之后,导入IDE中,执行对应的方法。跑了一会儿,感觉全部执行完可能要一两天,但是又不想一直占用电脑资源。就想着生成可执行jar包,放到服务器上执行。看了下pom文件中已经添加了maven-assembly-plugin和maven-shade-plugin。于是尝试打包执行生成的可执行jar包。结果报错了。
错误如下:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.SecurityException: Invalid signature file digest for Manifest main attributes
at sun.security.util.SignatureFileVerifier.processImpl(SignatureFileVerifier.java:314)
at sun.security.util.SignatureFileVerifier.process(SignatureFileVerifier.java:268)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:316)
at java.util.jar.JarVerifier.update(JarVerifier.java:228)
at java.util.jar.JarFile.initializeVerifier(JarFile.java:383)
at java.util.jar.JarFile.getInputStream(JarFile.java:450)
at sun.misc.URLClassPath$JarLoader$2.getInputStream(URLClassPath.java:977)
at sun.misc.Resource.cachedInputStream(Resource.java:77)
at sun.misc.Resource.getByteBuffer(Resource.java:160)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:454)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:495)
这个错误是因为在jar包的合并过程中,META-INF目录下含有一些jar包的数字签名文件(.SF,.DSA,*.RSA),查看网上资料,可以将jar中的文件删掉:
zip -d spark_scala_demo.jar META-INF/*.RSA META-INF/*.DSA META-INF/*.SF
或者在shade插件的配置中添加过滤器:
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
但是我看了下pom.xml中其实已经添加了过滤器,那为什么生成的jar里还有这些文件呢?再仔细查看发现filter的artifact这里少了*号:
<artifact>:</artifact>
估计是之前那个同事从网上复制粘贴过来的,那么这里应该怎么配置呢?我们执行下shade插件的帮助命令看看:
mvn shade:help -Ddetail=true -Dgoal=shade
从帮助信息中找到artifactSet的说明,如下:
artifactSet
Artifacts to include/exclude from the final artifact. Artifacts are
denoted by composite identifiers of the general form
groupId:artifactId:type:classifier. Since version 1.3, the wildcard
characters '*' and '?' can be used within the sub parts of those composite
identifiers to do pattern matching. For convenience, the syntax groupId is
equivalent to groupId:*:*:*, groupId:artifactId is equivalent to
groupId:artifactId:*:* and groupId:artifactId:classifier is equivalent to
groupId:artifactId:*:classifier. For example:
<artifactSet>
<includes>
<include>org.apache.maven:*</include>
</includes>
<excludes>
<exclude>*:maven-core</exclude>
</excludes>
</artifactSet>
可见shade插件自1.3版本之后支持通配符“*”和“?”,在配置的时候可以写具体的artifactId,也可以使用通配符,但是不能像上面那样只有一个分隔符“:”。