Kafka 2.1.0 的源码环境搭建

Kafka 2.1.0 的源码环境搭建

提前准备

下载kafka源码

http://kafka.apache.org/downloads

我这里使用的是kafka2.1.0版本的

下载完之后,解压即可

安装jdk

jdk1.8

oracle官网下载即可

安装编译工具

gradle 5.5(我最开始使用别的版本报错了,识别不了依赖)

https://gradle.org/next-steps/?version=5.5&format=all

gradle和maven意义,修改一下配置,让其可以访问国内的代码仓库

allprojects {
    repositories {
		maven { url 'file:///D:/maven/maven_repository/maven_repository'}
        mavenLocal()
		maven { name "Alibaba" ; url "https://maven.aliyun.com/repository/public" }
		maven { name "Bstek" ; url "http://nexus.bsdn.org/content/groups/public/" }
		mavenCentral()
    }

	buildscript { 
		repositories { 
			maven { name "Alibaba" ; url 'https://maven.aliyun.com/repository/public' }
			maven { name "Bstek" ; url 'http://nexus.bsdn.org/content/groups/public/' }
			maven { name "M2" ; url 'https://plugins.gradle.org/m2/' }
		}
	}
}

file:///D:/maven/maven_repository/maven_repository’}

指定你本地的maven仓库目录

这里借助idea来编译kafka的源码,因为编译的是kafka2.1,所以对应的scala也需要选择scala2.1.x 版本的,避免不必要的依赖问题出现

开始编译

idea编译kafka源码项目

开始编译

编译成功之后就可以了

测试

在没报错的前提下,测试kafka源码环境是否搭建成功了

core模块下的这个Kafka是kafka服务的启动主类

需要我们配置启动

但是在启动之前还需要安装Zookeeper环境

安装Zookeeper环境

https://download.csdn.net/download/weixin_44445168/16182602

下载后直接解压,修改配置,就可以运行了

修改zk的配置

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=D:\Server\zookeeper-3.4.14\data
dataLogDir=D:\Server\zookeeper-3.4.14\log
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

主要是为了修改

dataDir=D:\Server\zookeeper-3.4.14\data
dataLogDir=D:\Server\zookeeper-3.4.14\log

启动zookeeper

双击便可以执行,然后就可以使用kafka了

运行kafka

正常启动之后,kafka源码包中提供了生产者和消费者的测试包

可以看到数据的输出打印就说明源码环境搭建成功了