val sparkConf = new SparkConf().setAppName(“ThreadReadHbase”).setMaster(“yarn”)
.set(“spark.executor.instances”, “5”) //设置运算节点个数
.set(“spark.driver.memory”, “2g”)
.set(“spark.executor.memory”, “2g”)
.set(“spark.driver.cores”, “4”)
.set(“spark.executor.cores”, “4”)
.set(“spark.streaming.blockInterval”, “50”)
.set(“spark.default.parallelism”, “10”)
.set(“spark.serializer”, “org.apache.spark.serializer.KryoSerializer”)//序列化,hbase底层存储是bytes类型
作者:William•Hu
来源:CSDN
原文:https://blog.csdn.net/u010916338/article/details/81103634
版权声明:本文为博主原创文章,转载请附上博文链接!