kafka consumer assign 和 subscribe模式差异分析

        转载请注明原创地址：http://www.cnblogs.com/dongxiao-yang/p/7200971.html 

 

        最近需要研究flink-connector-kafka的消费行为，发现flink使用了kafka consumer一个比较底层一点的assign接口而不是之前比较常用的subscirbe,于是研究下二者之间的差异。

        首先看api文档：http://kafka.apache.org/0110/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html

public void assign(Collection<TopicPartition> partitions)

Manually assign a list of partitions to this consumer. This interface does not allow for incremental assignment and will replace the previous assignment (if there is one). If the given list of topic partitions is empty, it is treated the same as  unsubscribe().

Manual topic assignment through this method does not use the consumer's group management functionality. As such, there will be no rebalance operation triggered when group membership or cluster and topic metadata change. Note that it is not possible to use both manual partition assignment with assign(Collection) and group assignment with subscribe(Collection, ConsumerRebalanceListener).

       与subscirbe方法不同，assign方法由用户直接手动consumer实例消费哪些具体分区，根据api上述描述，assign的consumer不会拥有kafka的group management机制，也就是当group内消费者数量变化的时候不会有reblance行为发生。assign的方法不能和subscribe方法同时使用。

      然后看一下具体实现源码：