sparkstreaming对接kafka出现的数据积压问题

kafka数据积压问题

1. 问题描述
生产环境开发了一套sparkstreaming对接kafka，并进行数据处理的程序。刚开始程序运行的很好，kafka集群被人动过之后，重启spark程序时出现如下报警信息：

18/06/20 15:29:21 WARN kafka010.KafkaUtils: overriding enable.auto.commit to false for executor
18/06/20 15:29:21 WARN kafka010.KafkaUtils: overriding auto.offset.reset to none for executor
18/06/20 15:29:21 WARN kafka010.KafkaUtils: overriding executor group.id to spark-executor-ymtopic
18/06/20 15:29:21 WARN kafka010.KafkaUtils: overriding receive.buffer.bytes to 65536 see KAFKA-3135
18/06/20 15:34:00 WARN internals.ConsumerCoordinator: Auto-commit of offsets {iptv_js-10=OffsetAndMetadata{
  offset=915889, metadata=''}, iptv_js-9=OffsetAndMetadata{
  offset=1018618, metadata=''}, iptv_js-11=OffsetAndMetadata{
  offset=101