1. 准备安装包
csd包:
http://archive.cloudera.com/csds/kafka/
parcel包:
http://archive.cloudera.com/kafka/parcels/latest/ (根据自己的集群的系统版本下载 )
KAFKA-2.0.2-1.2.0.2.p0.5-wheezy.parcel
KAFKA-2.0.2-1.2.0.2.p0.5-wheezy.parcel.sha
manifest.json
2. CDH集成Kafka
2.1 将原来Kafka的在CDH中删除
2.2、csd包放到cm安装节点下的 /opt/cloudera/csd目录下
cp KAFKA-1.2.0.jar /opt/cloudera/csd/
2.3、将parcel包放到cm安装节点下的/opt/cloudera/parcel-repo目录下
cp KAFKA-2.0.2-1.2.0.2.p0.5-wheezy.parcel /opt/cloudera/parcel-repo/
cp KAFKA-2.0.2-1.2.0.2.p0.5-wheezy.parcel.sha /opt/cloudera/parcel-repo/
2.4、使用manifest.json中对应的部分,添加到/opt/cloudera/parcel-repomanifest.json中
vi /opt/cloudera/parcel-repo/manifest.json
添加
{
"parcelName": "KAFKA-2.0.2-1.2.0.2.p0.5-wheezy.parcel",
"components": [
{
"pkg_version": "0.9.0+kafka2.0.2",
"pkg_release": "1.2.0.2.p0.5",
"name": "kafka",
"version": "0.9.0-kafka2.0.2"
}
],
"depends": "CDH (>= 5.2), CDH (<< 6.0)",
"replaces": "CLABS_KAFKA",
"hash": "8756ae832d388880a8f8c38b6569f461996c84bc"
}
2.5、启动Cloudera Manager,检查更新parcel,分配并激活percel包
注意此处一定要激活才能使用:
CM主页 -> Hosts -> parcel -> 检查新parcel -> 配置 -> 激活
3.安装kafka
如果启动时出现内存溢出错误(java.lang.OutOfMemoryError: Java heap space)
主页服务列表 -> kafka -> 配置 -> Kafka-Broker -> Java Heap Size of Broker
修改参数到256以上即可