![](https://img-blog.csdnimg.cn/20201014180756925.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
spark
spark
少蓬
这个作者很懒,什么都没留下…
展开
-
win下Sparksql 无法获取文件权限问题
Caused by: java.lang.RuntimeException: Error while running command to get file permissions : java.io.IOException: (null) entry in command string: null ls -F D:\tmp\hive Caused by: java.lang.RuntimeException: Error while running command to get file permissi原创 2020-06-29 14:28:02 · 846 阅读 · 0 评论 -
Spark自定义聚合函数
Spark自定义聚合函数 package com.offcn.spark import org.apache.spark.sql.Row import org.apache.spark.sql.expressions.{MutableAggregationBuffer, UserDefinedAggregateFunction} import org.apache.spark.sql.types.{DataType, StringType, StructField, StructType} class原创 2020-06-29 14:19:21 · 555 阅读 · 0 评论 -
SparkStreaming 消费Kafka中数据
SparkStreaming 消费Kafka中数据 import java.text.{DateFormat, SimpleDateFormat} import java.util.Date import org.apache.kafka.common.serialization.StringDeserializer import org.apache.spark.{SparkConf, SparkContext} import org.apache.spark.sql.SparkSession imp原创 2020-06-29 14:15:52 · 221 阅读 · 0 评论