spark
kong-kong
记录流水账
展开
-
spark 笔记1
#flatMapval favMovies = sc.parallelize(List("Pulp Fiction","Requiem for a dream","Aclockwork Orange"));favMovies.flatMap(movieTitle=>movieTitle.split(" ")).collect() #sampleval data = s...原创 2019-02-23 15:00:36 · 262 阅读 · 0 评论 -
spark单机安装
spark下载地址:http://spark.apache.org/downloads.html #解压: tar -xzvf spark-2.4.0-bin-hadoop2.6.tgzcd spark-2.4.0-bin-hadoop2.6#执行命名 进入控制台./bin/spark-shellvar lines = sc.textFile("README...原创 2019-02-20 21:20:14 · 366 阅读 · 0 评论 -
java.net.BindException: 无法指定被请求的地址
单机版启动报错: 2019-02-21 09:46:24 WARN Utils:66 - Service 'sparkDriver' could not bind on a random free port. You may check whether configuring an appropriate binding address.2019-02-21 09:46:24 WARN ...原创 2019-02-21 09:51:47 · 5416 阅读 · 0 评论 -
scala list
val names = List("a", b", "c")names.map(_.toUpperCase)names.foreach(println)names.foreach{ name => println(name) }# res10: List[Int] = List(1, 2, 1, 3, 4, 3, 5) 数据展开# flatten可以把嵌套的展开L...原创 2019-02-22 14:58:27 · 328 阅读 · 0 评论