就这一次！拼多多内部架构师培训Kafka源码笔记（现已绝版）

最新推荐文章于 2024-10-15 17:09:00 发布

2401_85393583

最新推荐文章于 2024-10-15 17:09:00 发布

阅读量306

点赞数 3

分类专栏：资料公粽号文章标签： kafka 笔记分布式

本文链接：https://blog.csdn.net/2401_85393583/article/details/140191742

版权

资料公粽号专栏收录该内容

16 篇文章 0 订阅

订阅专栏

2.3 RecordAccumulator分析

MemoryR ecords

RecordBatch

BufferPool

RecordAccumulator

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-FK1lRKJ1-1623746340938)(//upload-images.jianshu.io/upload_images/19957501-c85b117b64472e08?imageMogr2/auto-orient/strip|imageView2/2/w/842/format/webp)]

2.4 Sender分析

创建请求

KSelector

InFlightRequests

MetadataUpdater

NetworkClient

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-rBtCVR6y-1623746340939)(//upload-images.jianshu.io/upload_images/19957501-807690c6b08bb06b?imageMogr2/auto-orient/strip|imageView2/2/w/848/format/webp)]

第3章消费者

3.1 KafkaConsumer 使用示例

3.2 传递保证语义( Delivery guarantee semantic )

3.3 Consumer Group Rebalance设计

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-WM85UJv0-1623746340940)(//upload-images.jianshu.io/upload_images/19957501-08fbd4342e33708f?imageMogr2/auto-orient/strip|imageView2/2/w/841/format/webp)]

3.4 KafkaConsumer 分析

ConsumerNetworkC lient

SubscriptionState

ConsumerCoordinator.

PartitionAssignor 分析

Heartbeat 分析

Rebalance 实现

offset 操作

Fetcher

KafkaConsumer 分析总结

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2EHXWIyM-1623746340942)(//upload-images.jianshu.io/upload_images/19957501-e880f44d2bf98ebe?imageMogr2/auto-orient/strip|imageView2/2/w/840/format/webp)]

第4章Kafka服务端

4.1 网络层

Reactor模式

SocketServer

AbstractServer Thread

Acceptor

Processor

RequestChannel

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-yHoaCBvY-1623746340943)(//upload-images.jianshu.io/upload_images/19957501-2eb7dd36f59abd2e?imageMogr2/auto-orient/strip|imageView2/2/w/856/format/webp)\

4.2 API层

KafkaR equestHandler

KafkaApis

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-mIIRzbq5-1623746340944)(//upload-images.jianshu.io/upload_images/19957501-9c196fe12548f872?imageMogr2/auto-orient/strip|imageView2/2/w/839/format/webp)]

4.3 日志存储

基本概念

FileMessageSet

ByteBufferMessageSet

OfsetIndex

LogSegment

Log

LogManager

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-5US3Xiux-1623746340945)(//upload-images.jianshu.io/upload_images/19957501-1f169996786dfa52?imageMogr2/auto-orient/strip|imageView2/2/w/847/format/webp)\

4.4 DelayedOperationPurgatory 组件

TimingWheel

SystemTimer

Delayed Dperation

DelayedOperationPurgatory

DelayedProduce

DelayedFetch

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-UXVlCSFn-1623746340946)(//upload-images.jianshu.io/upload_images/19957501-8c2e03440b340d4f?imageMogr2/auto-orient/strip|imageView2/2/w/852/format/webp)]

4.5 副本机制

副本

分区

ReplicaManager

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Fh83eUma-1623746340947)(//upload-images.jianshu.io/upload_images/19957501-4886c9f1205f180f?imageMogr2/auto-orient/strip|imageView2/2/w/860/format/webp)]

4.6 KafkaController

ControllerChannelManager

ControllerContext

ControllerBrokerRequestBatch

PartitionStateMachine

PartitionLeaderSelector

ReplicaStateMachine

ZooKeeper Listener

KafkaController 初始化与故障转移

处理ContolledShutdownRequest

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-w96zvAoo-1623746340949)(//upload-images.jianshu.io/upload_images/19957501-75bbbb46b167e3bc?imageMogr2/auto-orient/strip|imageView2/2/w/845/format/webp)]

4.7 GroupCoordinator

最后

终极手撕架构师的学习笔记：分布式+微服务+开源框架+性能优化

og.csdnimg.cn/2021061516404666.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3dlaXhpbl80NzA2NjAyOA==,size_16,color_FFFFFF,t_70)

4.7 GroupCoordinator