kafka：springboot集成kafka，并完成单机版安装

于 2019-06-02 23:34:13 发布
阅读量1.1k 收藏
点赞数 1
分类专栏： kafka springBoot 文章标签： kafka springboot集成kafka
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_38750084/article/details/90745184
版权

因为我的环境：

springboot版本：

<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.0.4.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>

kafka的依赖：

 <!--这里不需要设置依赖包的版本也可，spring-boot-starter-parent已经帮我们添加了版本的管理。但是最好自己要加上，最好和kafka环境上的版本一致-->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
            <version>2.1.7.RELEASE</version>
        </dependency>
    </dependencies>

我一开始安装的是0.8版本的kafka，安装jdk是1.7的，导致我一启动springboot时，kafka就一直报错如下，同时也不可以send消息，总是timeout：


[2019-06-02 18:40:03,639] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)
[2019-06-02 18:40:04,794] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)
[2019-06-02 18:40:05,647] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)
[2019-06-02 18:40:06,653] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)
[2019-06-02 18:40:07,512] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)
[2019-06-02 18:40:08,617] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)
[2019-06-02 18:40:09,771] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)
[2019-06-02 18:40:10,677] ERROR Closing socket for /10.10.10.63 because of error (kafka.network.Processor)
kafka.common.KafkaException: Wrong request type 18
	at kafka.api.RequestKeys$.deserializerForKey(RequestKeys.scala:57)
	at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:50)
	at kafka.network.Processor.read(SocketServer.scala:353)
	at kafka.network.Processor.run(SocketServer.scala:245)
	at java.lang.Thread.run(Thread.java:745)

[root@sparkproject1 local]# 
[root@sparkproject1 local]#

分析原因：应该是用高版本的kafka client向0.8.1的kafka server端发送数据，版本不兼容的问题。

后来我就把kafka更换版本为：

kafka_2.11-2.2.0

启动报错说我jdk版本低，kafka不能启动起来，所以我又将jdk版本换为1.8，然后kafka就可以顺利启动了。

jdk下载（我的是linux 32位）：

https://download.csdn.net/download/qq_40580023/10754106

64位：

https://download.csdn.net/download/yang8468/10531023?utm_source=bbsseo

官网：

https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

springboot项目启动也不报错，同时kafka也可以收到消息：

使用kafkatool工具查看：

 

kafka新版本消费者启动命令：

bin/​​​​​​​kafka-console-consumer.sh --bootstrap-server sparkproject1:9092 --topic TestTopic--from-beginning

 

 

参考：

版本不兼容报错问题参考：

https://blog.csdn.net/zhuimeng_by/article/details/83589512

 

 

kafka新版本消费者启动命令参考：

https://blog.csdn.net/csdn_sunlighting/article/details/81516646

https://www.jianshu.com/p/80a10811d5cb

其他：

https://www.baidu.com/link?url=seUzu-w-TEYsSLforsBdKn8tqgXVLv2HTq7I3WvvlZPJ1bVIA-L6WtYrs1WRHzxS3EIYBAFTWd5HuFpbHLJNPa&wd=&eqid=da66f55800003fe0000000035cf3ae85

https://www.cnblogs.com/kangoroo/p/7353330.html

https://blog.csdn.net/lv_1093964643/article/details/83177280

kafka配置说明：

https://www.cnblogs.com/rilley/p/5391268.html

 

 

springboot整合kafka过程中出现的一个错误：

 

springboot版本：

<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.0.4.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>


kafka版本：

<dependency>
          <groupId>org.springframework.kafka</groupId>
          <artifactId>spring-kafka</artifactId>
          <version>1.1.1.RELEASE</version>
      </dependency>

启动报错：

D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\bin\java.exe -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:54880,suspend=y,server=n -javaagent:C:\Users\Administrator\.IdeaIC2018.2\system\captureAgent\debugger-agent.jar=file:/C:/Users/Administrator/AppData/Local/Temp/capture.props -Dfile.encoding=UTF-8 -classpath "D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\charsets.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\deploy.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\access-bridge-64.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\cldrdata.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\dnsns.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\jaccess.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\jfxrt.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\localedata.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\nashorn.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\sunec.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\sunjce_provider.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\sunmscapi.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\sunpkcs11.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\ext\zipfs.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\javaws.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\jce.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\jfr.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\jfxswt.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\jsse.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\management-agent.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\plugin.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\resources.jar;D:\Aimsen\myDevelopKit\1.software\jdk\64位\1.8\jdk-8-windows-x64\jre\lib\rt.jar;D:\workspace\dubbospringboot\dubbo-provider\target\classes;D:\workspace\dubbospringboot\dubbo-dao\target\classes;D:\workspace\dubbospringboot\dubbo-api\target\classes;D:\repository\log4j\log4j\1.2.17\log4j-1.2.17.jar;D:\workspace\dubbospringboot\dubbo-common\target\classes;D:\repository\org\apache\commons\commons-lang3\3.7\commons-lang3-3.7.jar;D:\repository\com\alibaba\fastjson\1.2.49\fastjson-1.2.49.jar;D:\repository\com\belerweb\pinyin4j\2.5.0\pinyin4j-2.5.0.jar;D:\repository\org\springframework\boot\spring-boot-starter-web\2.0.4.RELEASE\spring-boot-starter-web-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot-starter\2.0.4.RELEASE\spring-boot-starter-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot\2.0.4.RELEASE\spring-boot-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot-autoconfigure\2.0.4.RELEASE\spring-boot-autoconfigure-2.0.4.RELEASE.jar;D:\repository\org\springframework\boot\spring-boot-starter-logging\2.0.4.RELEASE\spring-boot-starter-logging-2.0.4.RELEASE.jar;D:\repository\ch\qos\logback\logback-classic\1.2.3\logback-classic-1.2.3.jar;D:\repository\ch\qos\logback\logback-core\1.2.3\logback-core-1.2.3.jar;D:\repository\org\apache\logging\log4j\log4j-to-slf4j\2.10.0\log4j-to-slf4j-2.10.0.jar;D:\repository\org\apache\logging\log4j\log4j-api\2.10.0\log4j-api-2.10.0.jar;D:\repository\org\slf4j\jul-to-slf4j\1.7.25\jul-to-slf4j-1.7.25.jar;D:\repository\javax\annotation\javax.annotation-api\1.3.2\javax.annotation-api-1.3.2.jar;D:\repository\org\yaml\snakeyaml\1.19\snakeyaml-1.19.jar;D:\repository\org\springframework\boot\spring-boot-starter-json\2.0.4.RELEASE\spring-boot-starter-json-2.0.4.RELEASE.jar;D:\repository\com\fasterxml\jackson\core\jackson-databind\2.9.6\jackson-databind-2.9.6.jar;D:\repository\com\fasterxml\jackson
最低0.47元/天 解锁文章
花和尚也有春天
关注
  • 1
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值