Kafka的ACK含义

ACK有三种

0 

意味着producer不等待broker同步完成的确认，继续发送下一条(批)信息

提供了最低的延迟。但是最弱的持久性，当服务器发生故障时，就很可能发生数据丢失。例如leader已经死亡，producer不知情，还会继续发送消息broker接收不到数据就会数据丢失

1

意味着producer要等待leader成功收到数据并得到确认，才发送下一条message。此选项提供了较好的持久性较低的延迟性。

Partition的Leader死亡，follwer尚未复制，数据就会丢失

-1

意味着producer得到follwer确认，才发送下一条数据

 

原理：

一个topic可以分成多个partition，一个partition可以在跨broker的节点上存放多副本（Leader & Follower）

 

我们知道一般情况下，一个Broker发生了问题或正常关闭，zookeeper会及时发现并将Leader移至其他Broker节点。

那么问题来了，如果broker进程非正常关闭（比如使用“kill -9进程号”方式关闭）对于zookeeper来说就没那么容易恢复了。而如果此时我还有2000个partition队列，受影响的broker承担了其中400个partition的leader角色，那可用性就会大受影响了

 

 

Kafka三代 - Broker集群

很明显，为了解决高可用问题，我们需要集群。

Kafka对集群的支持也是非常友好的。在Kafka中，集群里的每个实例叫做Broker，就像这样：

图片来源：sookocheff.com

每个partition不再只有一个，而是有一个leader(红色)和多个replica(蓝色)，生产者根据消息的topic和key值，确定了消息要发往哪个partition之后（假设是p1），会找到partition对应的leader(也就是broker2里的p1)，然后将消息发给leader，leader负责消息的写入，并与其余的replica进行同步。

ACK = 0 时 发送一次 不论leader是否接收

ACK = 1 时，等待leader接收成功即可

ACK = -1 时 ，需等待leader将消息同步给follower 

 

附：SpringBoot 对kafka 自动配置对 ack字段的注释

  The number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the  
  durability of records that are sent. The following settings are allowed:  
  <li><code>acks=0</code> If set to zero then the producer will not wait for any acknowledgment from the 
  server at all. The record will be immediately added to the socket buffer and considered sent. No guarantee can be 
  made that the server has received the record in this case, and the <code>retries</code> configuration will not 
  take effect (as the client won't generally know of any failures). The offset given back for each record will 
  always be set to -1. 
  <li><code>acks=1</code> This will mean the leader will write the record to its local log but will respond 
  without awaiting full acknowledgement from all followers. In this case should the leader fail immediately after 
  acknowledging the record but before the followers have replicated it then the record will be lost. 
  <li><code>acks=all</code> This means the leader will wait for the full set of in-sync replicas to 
  acknowledge the record. This guarantees that the record will not be lost as long as at least one in-sync replica 
  remains alive. This is the strongest available guarantee. This is equivalent to the acks=-1 setting. 

 如果没有配置acks则取默认值1

ProducerConfig中

private static final ConfigDef CONFIG;

CONFIG = new ConfigDef().define(ACKS_CONFIG,
        Type.STRING,
        "1",//defaultValue
        in("all", "-1", "0", "1"),
        Importance.HIGH,
        ACKS_DOC)