取出mysql中的数据将数据转成RDD后并和rdd交集:交集只能完全相同的rdd

val username = “root”
val password = “huangchao”
val drive = “com.mysql.jdbc.Driver”
val url = “jdbc:mysql://slave2:3306/travel”
var connection:Connection = null

val sparkConf = new SparkConf().setAppName("HBaseReadTest").setMaster("local[2]")
val sc = new SparkContext(sparkConf)

val value: RDD[(String, String)] = sc.parallelize(Seq(("rk1002", "11111"),("a", "11111")))

classOf[com.mysql.jdbc.Driver]
connection = DriverManager.getConnection(url,username,password)
val statement = connection.createStatement()
val result = statement.executeQuery("select row,name from student")

val stringToString: mutable.Map[String, String] = collection.mutable.Map[String, String]()
val tuplesToStringToString: Seq[(String, String)] => Map[String, String] = Map[String, String]

while(result.next()){
  val id: String = result.getString("name")
  val str: String = result.getString("row")

 stringToString.put(id,str)

}


//将Map转成RDD
val value1: RDD[(String, String)] = sc.parallelize(stringToString.toList)

//只能取完全相同的交集
val value2: RDD[(String, String)] = value1.intersection(value)

value2.foreach(println)

// stringToString.foreach(print)

connection.close()
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值