tachyon with spark

---

spark1.2.0  tachyon0.5.0 jdk1.7 scala2.10.4

---

 

1.装好spark、tachyon、jdk、scala

2.修改spark-env.sh添加Tachyon客户端jar包环境变量

export HADOOP_CLASSPATH=/home/hadoop/tachyon-0.5.0-bin/client/target/tachyon-cli
ent-0.5.0-jar-with-dependencies.jar

3.创建一个core-site.xml并配置（本没有，自己创建）

<configuration>
  <property>
      <name>fs.tachyon.impl</name>
     <value>tachyon.hadoop.TFS</value>
  </property>
</configuration>

4.启动spark启动tachyon

tachyon-start.sh local
sbin/start-all.sh

5.测试

spark-shell

val s =sc.textFile("tachyon://hadoop:19998/test/passwd")
s.count
s.saveAsTextFile("tachyon://hadoop:19998/count")

6.查看结果

---

 

转载于:https://www.cnblogs.com/admln/p/tachyon-with-spark.html