1.首相将hive中的配置文件hive-site.xml,hadoop的配置文件core-site.xml,hdfs-site.xml
------复制到$SPARK_HOME/conf/目录中
2.再将jdbc驱动程序mysql-connector-java-x.x.x.jar 复制到$SPARK_HOME/jars或者$SPARK_HOME/lib目录下
3.启动动hadoop,hive和mysql
4.启动spark local模式
bin/spark-shell --driver-class-path $SPARK_HOME/jars/mysql-connector-java-x.x.x.jar
5.测试
-----a.创建hiveContext对象 val hc =org.apache.spark.sql.hive.HiveContext(sc)
-----b.创建表 hc.sql("create table test (a string,b string) row format delimited fields terminated by ',' ")
-----c.增加数据hc.sql("load data local 'filePath' into table test")
----------数据为:
----------Tom,20
----------Jim,40
----------Rose,50
-----d.查看数据hc.sql("select *from test")
=================================================================
下面测试mysql连接:
1.我们是通过hiveContext去连接的,语句为
val msyql = hc.read.format("jdbc").options(Map("url"->"jdbc:mysql://ip:3306/数据库名字?user=用户名&password=密码","dbtable"->"表名","driver"->"com.mysql.jdbc.Driver")).load()
2.查看表的内容
msyql.show()