spark需要maven管理吗_使用spark2.4和maven3.6.0组合开发踩过的坑

ERROR：thread "main" java.lang.ArrayIndexOutOfBoundsException: 10582

这是paranamer版本的问题。在spark-core / spark-sql之前添加以下依赖项。

com.thoughtworks.paranamer

paranamer

2.8

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Stopwatch.elapsedMillis()J

com.google.guava

guava

21.0

Exception in thread "main" java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.()V from class org.apache.hadoop.mapred.FileInputFormat

org.apache.hadoop

hadoop-common

2.7.7

org.apache.hadoop

hadoop-mapreduce-client-core

2.7.7

猜测是因为maven的版本太高