spark version : spark-2.0.0
maven : Apache Maven 3.5.0
cd /usr/local/spark-2.0.0
1
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"
Note:
For Java 8 and above this step is not required.
If using build/mvn with no MAVEN_OPTS set, the script will automate this for you.
2
mvn -Pyarn -Phadoop-2.7 -Pspark-ganglia-lgpl -Pkinesis-asl -Phive -DskipTests clean package
3
./dev/make-distribution.sh --name custom-spark --tgz -Psparkr -Phadoop-2.7 -Phive -Phive-thriftserver -Pyarn
Note:
if you come across spark-core_2.11: Command execution failed
may be you should try some ways the following
On Debian GNU/Linux (in particular 8/jessie) you would need to run
apt-get install r-base r-base-dev
or
aptitude install r-base r-base-dev
compile spark2.0.0 with maven
最新推荐文章于 2024-09-25 16:25:20 发布