Spark学习(SparkSQL连接oralce,MySQL)



Oracle 示例：

Class.forName("oracle.jdbc.driver.OracleDriver")
var theConf = new SparkConf().setAppName("testRDDMethod").setMaster("local")
var theSC = new SparkContext(theConf)
var theSC2 = new SQLContext(theSC)
var theJdbcDF = theSC2.load("jdbc",Map("url"->"jdbc:oracle:thin:用户/密码@//ip地址:端口/实例名",
  "dbtable" -> "(select * from tab) a","driver"->"oracle.jdbc.driver.OracleDriver"))
theJdbcDF.registerTempTable("myuser")
var theDT = theSC2.sql("select * from myuser")
theDT.registerTempTable("tempsum")

2）MySQL示例：

 Class.forName("com.mysql.jdbc.Driver")
var theConf = new SparkConf().setAppName("testRDDMethod").setMaster("local")
var theSC = new SparkContext(theConf)
var theSC2 = new SQLContext(theSC)
var theJdbcDF = theSC2.load("jdbc",Map("url"->"jdbc:mysql://ip地址:端口/mysql?user=XXXX&password=XXXX","dbtable" -> "要操作的表"))
theJdbcDF.registerTempTable("myuser")
var theDT = theSC2.sql("select * from myuser where b>2")
theDT.registerTempTable("tempsum")