dstream.foreachRDD { rdd =>
rdd.foreachPartition { partitionOfRecords =>
val connection = createNewConnection()
partitionOfRecords.foreach(record => connection.send(record))
connection.close()
}
}
每个rdd的每个分区创建一个connection减少资源的浪费
sparkStreaming写入到mysql中
最新推荐文章于 2022-07-20 15:41:16 发布