1.spark读取json类型的数据
val df = spark.read.json("examples/src/main/resources/vertex")
df.show(5,false)
2.spark读取hive中的数据
val spark = SparkSession.builder().master("yarn").appName("hive2nebula").enableHiveSupport().getOrCreate()
val sql: String ="select id_no as id_no_nebula,id_no,name,id_type,ecif_no from ccpd_zf1_ods_safe.tm_apply_info_zaps_dcn where ds='20380614' "
val df: DataFrame = spark.sql(sql).toDF()
df.show(5,false)
俩种读取方式得到的df的schema一样 。