【pyspark】表的读写创建

创建df_spark：

df_pandas= pd.DataFrame({"list_name": list_})

df_spark=spark.createDataFrame(df_pandas)

 

保存表：

# # 写入临时表中

spark.sql("use db_name")

df_saprk_hrouteinfo_all2.write.mode("overwrite").format("Hive").saveAsTable("tablename")

 

保存分区表:

import pyspark.sql.functions as F

df2=df.withColumn('d',F.lit(today))

 

spark.sql("use db_name")

df2.write.mode("append").format("Hive").partitionBy("d").saveAsTable(tablename )

 

 

生成临时表:

df.createOrReplaceTempView("Tabelname")