大数据
ocean42234111
制造业,DBA,大数据
展开
-
Hadoop kafka 配置示例
--查找zookeeper 和 kafka位置find / -name '*zookeeper-server-start.sh*' find / -name '*kafka-server-start.sh*' 启动zookeepersudo /usr/lib/kafka/bin/zookeeper-server-start.sh config/zookeeper.properties启动kafka...原创 2018-05-22 11:56:56 · 667 阅读 · 0 评论 -
spark scala 小程序发布示例
--项目codepackage stubsimport org.apache.spark.SparkContextobject CountJPGs { def main(args: Array[String]) { if (args.length < 1) { System.err.println("Usage: solution.CountJPGs <logf...原创 2018-05-22 14:03:44 · 885 阅读 · 0 评论 -
spark RDD 示例
spark Context scspark RDD 存储单元 --示例1 hadoop fs -put /home/training/training_materials/data/frostroad.txt /loudacre/frostroad.txt val myrdd=sc.textFile("/loudacre/frostroad.txt") myrdd.c...原创 2018-05-22 14:07:09 · 266 阅读 · 0 评论 -
spark sql 示例
--spark sqldataFrameval cuDF=sqlContext.read.table("device")cuDF.limit(100).show()cuDF.select("name","type")--保存到临时表cuDF.registerTempTable("cu2")sqlContext.sql(" select top 10 * from cu2 ").collect()...原创 2018-05-22 14:10:10 · 297 阅读 · 0 评论