kafka 2.0 版本升级学习

最新推荐文章于 2024-08-25 15:32:34 发布

蛋上有痣

最新推荐文章于 2024-08-25 15:32:34 发布

阅读量3k

点赞数 1

分类专栏： kafka

本文链接：https://blog.csdn.net/wtf0215/article/details/82381765

版权

本文详细介绍了Kafka 2.0版本的升级内容，包括默认offset保留时间从1天增加到7天，废弃对Java 7的支持，SSL连接验证算法更改，Kafka Streams和Connect配置更新，消费者API变化以及ACL资源前缀定义等改进。

摘要由CSDN通过智能技术生成

KIP-186 increases the default offset retention time from 1 day to 7 days. This makes it less likely to “lose” offsets in an application that commits infrequently. It also increases the active set of offsets and therefore can increase memory usage on the broker. Note that the console consumer currently enables offset commit by default and can be the source of a large number of offsets which this change will now preserve for 7 days instead of 1. You can preserve the existing behavior by setting the broker config offsets.retention.minutes to 1440.

配置参数：offsets.retention.minutes=10080(之前默认是1440即1天，现增加到7天，可以根据需要自由配置)
改造点：将默认偏移保留时间从1天增加到7天
缺点：内存使用也会变高
优点：不很活跃使用信息的应用，不至于丢失数据

Support for Java 7 has been dropped, Java 8 is now the minimum version required.

Java版本最低支持Java8，Java7已废弃，目前Java版本是jdk11

The default value for ssl.endpoint.identification.algorithm was changed to https, which performs hostname verification (man-in-the-middle attacks are possible otherwise). Set ssl.endpoint.identification.algorithm to an empty string to restore the previous behaviour.

配置参数：ssl.endpoint.identification.algorithm=https
改造点：改为https访问，防范攻击，算是加固http访问
可以手动改为空字符串"",即可使用原先的访问方式

KAFKA-5674 extends the lower interval of max.connections.per.ip minimum to zero and therefore allows IP-based filtering of inbound connections.

配置参数：max.connections.per.ip.minimum=0
改造：因此，允许基于IP的入站连接过滤。
不清楚这块具体功能

KIP-272 added API version tag to the metric kafka.network:type=RequestMetrics,name=RequestsPerSec,request={Produce|FetchConsumer|FetchFollower|…}. This metric now becomes kafka.network:type=RequestMetrics,name=RequestsPerSec,request={Produce|FetchConsumer|FetchFollower|…},version={0|1|2|3|…}. This will impact JMX monitoring tools that do not automatically aggregate. To get the total count for a specific request type, the tool needs to be updated to aggregate across different versions.
KIP-225 changed the metric “records.lag” to use tags for topic and partition. The original version with the name format “{topic}-{partitio