- 启动spark
cd /export/server/spark/sbin
start-all.sh
查看spark
jps
确保有worker进程 - 执行命令:
spark-shell \
--master spark://node2:7077 \
--executor-memory 1g \
--total-executor-cores 2 \
--jars /export/server/hive/lib/mysql-connector-java-5.1.32.jar \
--driver-class-path /export/server/hive/lib/mysql-connector-java-5.1.32.jar
- 读取mysql文件,返回一个dataFrame
val mysqlDF = spark.read.format("jdbc").options(Map("url" -> "jdbc:mysql://192.168.214.101:3306/test", "driver" -> "com.mysql.jdbc.Driver", "dbtable" -> "iplocation", "user" -> "root", "password" -> "root")).load()
指定mysql服务器地址,用户名和密码,还有表名,数据库名
- 查看mysqlDF数据
mysqlDF.show