1,解压spark1.4.gz
进入解压目录:
在pom.xml中指定各个插件的版本号
设置maven使用更多内存
export MAVEN_OPTS="-Xmx2g -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=512m"
To enable Hive integration for Spark SQL along with its JDBC server and CLI, add the -Phive and Phive-thriftserver profiles to your existing build options. By default Spark will build with Hive 0.13.1 bindings.
# Apache Hadoop 2.4.X with Hive 13 support
mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -Phive -Phive-thriftserver -DskipTests clean package
maven测试
mvn -Pyarn -Phadoop-2.4 -DskipTests -Phive -Phive-thriftserver clean package
mvn -Pyarn -Phadoop-2.4 -Phive -Phive-thriftserver test