Way to get a Filename in Spark Streaming
编写工具类
object GetFileNameFromStream extends java.io.Serializable { def getFileName(file: RDD[String]) :String ={ file.toDebugString } }
val source_file = ssc.textFileStream("/user/sap_data/data")
SparkStreming运行时调用工具类的获取文件名方法
source_file.foreachRDD(dataRDD => { val file= GetFileNameFromStream.getFileName(ekpoRDD) }