shark安装

2019独角兽企业重金招聘Python工程师标准>>>

spark0.9.1安装
解压缩
修改目录用户权限，用户hadoop登录
修改配置文件
//slaves
hadoop1
hadoop2
hadoop3

//spark-env.sh
export SPARK_MASTER_IP=hadoop1
export SPARK_MASTER_PORT=7077
export SPARK_WORKER_CORES=1
export SPARK_WORKER_INSTANCES=1
export SPARK_WORKER_MEMORY=3g

scp到个节点

****************************************************
****************************************************
shark安装
源码下载，解压缩
源码编译
SHARK_HADOOP_VERSION=2.2.0 SHARK_YARN=true sbt/sbt assembly
修改配置文件
//shark-env.sh
export SPARK_MEM=2g

# (Required) Set the master program's memory
export SHARK_MASTER_MEM=1g

# (Optional) Specify the location of Hive's configuration directory. By default,
# Shark run scripts will point it to $SHARK_HOME/conf
export HIVE_CONF_DIR="/app/hadoop/hive013/conf"

# For running Shark in distributed mode, set the following:
export HADOOP_HOME="/app/hadoop/hadoop220"
export SPARK_HOME="/app/hadoop/spark091"
export MASTER="spark://hadoop1:7077"

scp到个节点

****************************************************
****************************************************
启动hive metasotre service
nohup bin/hive --service metastore > metastore.log 2>&1 &

启动shark
bin/shark


./bin/shark –service sharkserver <port>
./bin/shark -h <server-host> -p <server-port>

转载于:https://my.oschina.net/goudingcheng/blog/669272