Spark Read/Write files

zhixingheyi_tian

已于 2025-02-22 17:55:44 修改

阅读量1.4k

点赞数

分类专栏： spark 文章标签： spark scala 大数据

于 2021-11-01 15:24:14 首次发布

本文链接：https://blog.csdn.net/zhixingheyi_tian/article/details/121080856

版权

spark 专栏收录该内容

139 篇文章 ¥19.90 ¥99.00

订阅专栏

超级会员免费看

Read CSV

# cat people-comma.csv
name,age,job
Jorge,30,Developer
Bob,32,Developer

启动 spark-shell

$SPARK_HOME/bin/spark-shell --conf spark.hadoop.fs.defaultFS=file:///

val df = spark.read.option("header", "true").option("inferSchema", "true").csv("people-comma.csv")
scala> df.show
+-----+---+---------+
| name|age|      job|
+-----+---+---------+
|Jorge| 30|Developer|
|  Bob| 32|Developer|
+-----+---+---------+


scala> df.printSchema
   def printSchema(level: Int): Unit   def printSchema(): Unit

scala> df.printSchema
root
 |-- name: string (nullable = tr