Kakfa异常CommitFailedException

     上文中说到Spark Streaming，场景的思路是kafka到spark在到kafka，然后开发另外一个程序从kakfa中消费数据，把数据存储到oracle中。
         在测试的时候并没有发现什么问题，但是在上线前测试发现程序在最初运行阶段消费了几条数据后马上会出异常后自动停止，异常如下：
         

点击(此处)折叠或打开

[INFO ] 2018-04-19 14:58:19,572 method:org.apache.kafka.clients.consumer.internals.AbstractCoordinator.coordinatorDead(AbstractCoordinator.java:565) Marking the coordinator 10.143.84.29:21007 (id: 2147483645 rack: null) dead for group group1 [INFO ] 2018-04-19 14:58:19,572 method:org.apache.kafka.clients.consumer.internals.AbstractCoordinator.coordinatorDead(AbstractCoordinator.java:565) Marking the coordinator 10.143.84.29:21007 (id: 2147483645 rack: null) dead for group group1 [INFO ] 2018-04-19 14:58:19,573 method:org.apache.kafka.clients.consumer.internals.AbstractCoordinator.coordinatorDead(AbstractCoordinator.java:565) Marking the coordinator 10.143.84.29:21007 (id: 2147483645 rack: null) dead for group group1 [ERROR] 2018-04-19 14:58:19,573 method:com.bonc.yx.KafkaConsumerTest.run(KafkaConsumerTest.java:119) org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. [ERROR] 2018-04-19 14:58:19,573 method:com.bonc.yx.KafkaConsumerTest.run(KafkaConsumerTest.java:119) org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. [ERROR] 2018-04-19 14:58:19,573 method:com.bonc.yx.KafkaConsumerTest.run(KafkaConsumerTest.java:119) org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. [INFO ] 2018-04-19 14:58:19,574 method:org.apache.kafka.clients.consumer.internals.AbstractCoordinator.handleGroupMetadataResponse(AbstractCoordinator.java:528) Discovered coordinator 10.143.84.29:21007 (id: 2147483645 rack: null) for group group1. [INFO ] 2018-04-19 14:58:19,575 method:org.apache.kafka.clients.consumer.internals.AbstractCoordinator.handleGroupMetadataResponse(AbstractCoordinator.java:528) Discovered coordinator 10.143.84.29:21007 (id: 2147483645 rack: null) for group group1. [INFO ] 2018-04-19 14:58:19,575 method:org.apache.kafka.clients.consumer.internals.AbstractCoordinator.handleGroupMetadataResponse(AbstractCoordinator.java:528) Discovered coordinator 10.143.84.29:21007 (id: 2147483645 rack: null) for group group1. [WARN ] 2018-04-19 14:58:19,575 method:org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:506) Auto offset commit failed for group group1: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. [WARN ] 2018-04-19 14:58:19,576 method:org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:506) Auto offset commit failed for group group1: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. [WARN ] 2018-04-19 14:58:19,577 method:org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:506) Auto offset commit failed for group group1: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured session.timeout.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. [WARN ] 2018-04-19 14:58:19,585 method:org.apache.kafka.common.security.kerberos.KerberosLogin$1.run(KerberosLogin.java:204) TGT renewal thread has been interrupted and will exit.

     搜索几个解决方案后，异常的原因是客户端消费数据时间过长，未向kafka反馈消费结果，方案中提到需要修改kafka的配置文件，将server.properties中的session.timeout.ms调大,（默认为5s）。  后来发现程序中kafka消费配置，由自动记录偏移改为了手动，poll的数值过大，在处理结束后才会进行consumer.commitAsync（），由此推测是此部分代码导致的异常，修改poll的值后已经解决。下面是部分代码：

点击(此处)折叠或打开

        Properties prop = new Properties();
        if (!isEmpty(krb5File) && !isEmpty(kafkaClientFile)) {

            System.setProperty("java.security.krb5.conf", krb5File);
            System.setProperty("java.security.auth.login.config", kafkaClientFile);

            prop.put("security.protocol", "SASL_PLAINTEXT");
            prop.put("sasl.kerberos.service.name", "kafka");
            prop.put("sasl.mechanism", "GSSAPI");
        }
      
        prop.put("group.id", groupId); // 鎸囧畾鍒嗙粍id
        prop.put("bootstrap.servers", kafkaCluster); // 鎸囧畾kafka鐨勮繛鎺?rl

        prop.put("enable.auto.commit", "false");
        //prop.put("session.timeout.ms", "6000");
        prop.put("auto.offset.reset", "earliest");
        prop.put("key.deserializer", StringDeserializer.class.getName());
        prop.put("value.deserializer", StringDeserializer.class.getName());

        this.consumer = new KafkaConsumer<>(prop);

此处设置的enable.auto.commit为false，即手动提交偏移。

点击(此处)折叠或打开

try {
      consumer.subscribe(topics);

      //logger.info("output poll data....");
      while (true) {
           ConsumerRecords<String, String> records = consumer.poll(100);

将上面poll的参数值改小，解决了这个问题，以上。

来自 “ ITPUB博客 ” ，链接：http://blog.itpub.net/30422968/viewspace-2153131/，如需转载，请注明出处，否则将追究法律责任。

转载于:http://blog.itpub.net/30422968/viewspace-2153131/