kafka
文章平均质量分 77
zixuanshizhu
这个作者很懒,什么都没留下…
展开
-
kafka本地存储3-LogSegment
LogSegmentclassLogSegment(vallog: FileMessageSet, valindex: OffsetIndex, val baseOffset: Long, //这个Segment存储的消息的partition起始的消息偏移量原创 2016-01-27 16:32:20 · 348 阅读 · 0 评论 -
kafka本地存储2-message
MessageMinHeaderSize=CrcLength+MagicLength+AttributesLength+KeySizeLength+ValueSizeLength1. 4 byte CRC32 of the message 消息的字节内容buffer进行crc32编码,从magic之后的内容进行编码 Utils.crc32(buffer.array,原创 2016-01-27 16:30:45 · 449 阅读 · 0 评论 -
kafka本地存储4-LogCleaner
LogCleanerManager管理每个TopicAndPartition的清理状态inProgress= mutable.HashMap[TopicAndPartition, LogCleaningState]()清理状态如下三种LogCleaningInProgressLogCleaningAbortedLogCleaningPaused会有多个数据根原创 2016-01-27 16:33:43 · 949 阅读 · 0 评论 -
kafka本地存储5-LogManager
classLogManager(vallogDirs: Array[File], val topicConfigs: Map[String, LogConfig], val defaultConfig: LogConfig, val cleanerConfig原创 2016-01-27 16:34:45 · 909 阅读 · 0 评论 -
Replica下的 highWatermark和 logEndOffset说明
ReplicaManager.updateReplicaLEOAndPartitionHW更新replica.logEndOffset 是在函数ReplicaManager.updateReplicaLEOAndPartitionHW(topic: String, partitionId: Int, replicaId: Int, offset: LogOffsetMetada原创 2016-03-09 10:29:52 · 1035 阅读 · 0 评论