spark mysql 数据源_spark 从数据库中读取数据

spark 从数据库中读取数据的几种方式，以Mysql 为例

创建spark

val spark = SparkSession.builer().appName("用户标签").master("local[*]").getOrCreate()

val url = "jdbc:mysql://数据库地址/数据库名"

val driver = "mysql 数据库驱动”

val tableName = "表名"

val user = "数据库用户名"

val password = "数据库密码"

方式一: 直连

val df: DataFrame = spark.read.format("jdbc").options(Map(

"url" -> url,

"driver" -> driver

"dbtable" -> tableName,

"user"-> user,

"password" -> password

)).load()

方式二: 直连，执行指定sql

val sql = "select id, username from tableName"

val df: DataFrame = spark.read.format("jdbc").options(Map(

"url" -> url,

"driver" -> driver