kafka单服务器搭建
kafka搭建需要jdk, zookeeper
因此搭建前需先准备好JDK环境
由于kafka的安装包中已经内置一个zk,因此可以不用另外搭建, 当然如果想独立处理,或者已经拥有zk也可以接入使用
下载地址:
http://kafka.apache.org/downloads
1.下载
下载带有scala的安装包
Binary downloads:
Scala 2.11 - kafka_2.11-2.3.0.tgz (asc, sha512)
Scala 2.12 - kafka_2.12-2.3.0.tgz (asc, sha512)
2.安装JDK
mkdir /app
tar -zxvf jdk-8u221-linux-x64.tar.gz
添加环境变量
vi ~/.bash_profile
export JAVA_HOME=/app/jdk1.8.0_221
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=.:${JAVA_HOME}/bin:$ZOOKEEPER_HOME/bin:$PATH
加载环境变量
source ~/.bash_profile
验证java环境
# java --version
Unrecognized option: --version
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
[root@kafka kafka_2.11-2.2.0]#
3.解压安装
tar -zxf kafka_2.11-2.2.0.tgz
cd kafka_2.11-2.2.0/
4.启动zk
bin/zookeeper-server-start.sh config/zookeeper.properties
[root@kafka kafka_2.11-2.2.0]# bin/zookeeper-server-start.sh config/zookeeper.properties
[2019-09-01 15:42:13,338] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2019-09-01 15:42:13,344] INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager)
[2019-09-01 15:42:13,344] INFO autopurge.purgeInterval set to 0 (org.apache.zookeeper.server.DatadirCleanupManager)
[2019-09-01 15:42:13,344] INFO Purge task is not scheduled. (org.apache.zookeeper.server.DatadirCleanupManager)
[2019-09-01 15:42:13,344] WARN Either no config or no quorum defined in config, running in standalone mode (org.apache.zookeeper.server.quorum.QuorumPeerMain)
[2019-09-01 15:42:13,361] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2019-09-01 15:42:13,361] INFO Starting server (org.apache.zookeeper.server.ZooKeeperServerMain)
....
[2019-09-01 15:42:33,437] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2019-09-01 15:42:33,444] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)
5.启动kafka
bin/kafka-server-start.sh config/server.propertie
[2019-09-01 15:58:04,457] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-09-01 15:58:04,501] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-09-01 15:58:04,504] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2019-09-01 15:58:04,621] INFO [/config/changes-event-process-thread]: Starting (kaf