方法一,Spark中使用toDF函数创建DataFrame
通过导入(importing)Spark sql implicits, 就可以将本地序列(seq), 数组或者RDD转为DataFrame。只要这些数据的内容能指定数据类型即可。需要注意spark和scala的版本,否则会报找不到function(Note that, Spark 2.x is pre-built with Scala 2.11 except version 2.4.2, which is pre-built with Scala 2.12. Spark 3.0+ is pre-built with Scala 2.12)
本地seq + toDF创建DataFrame:
val sc = SparkSession.builder().appName("userImage").master("local").enableHiveSupport().getOrCreate()
import sc.sqlContext.implicits._
val custInfo = Seq((1,"321034202008105678","Frank","18910983245","10101010101010101010",java.sql.Date.valueOf("2020-08-10")),
(2,"321034202008105679","Mike","13510983245","10101010101010101020",java.sql.Date.valueOf("2020-08-10")),
(3,"321034202008105680","Kade","13910983245","10101010101010101030",java.sql.Date.valueOf("2020-08-10"))).toDF("id","id_card","name","phone_no","debit_card","regster_date")
custInfo.show()
//+---+------------------+-----+-----------+--------------------+------------+
//| id| id_card| name| phone_no| debit_card|regster_date|
//+---+------------------+-----+-----------+--------------------+------------+
//| 1|321034202008105678|Frank|18910983245|10101010101010101010| 2020-08-10|
//