golang基础-kafka、zookeeper搭建、go终端发送数据给kafka

最新推荐文章于 2024-09-26 19:23:28 发布

易辰_

最新推荐文章于 2024-09-26 19:23:28 发布

阅读量5k

点赞数

分类专栏： Go基础

本文链接：https://blog.csdn.net/u013210620/article/details/78570820

版权

Go基础专栏收录该内容

69 篇文章 12 订阅

订阅专栏

zookeeper搭建

1、安装JAVA-JDK，从oracle下载最新的SDK安装(我用的是1.8的)
2、安装zookeeper3.3.6，下载地址：http://apache.fayea.com/zookeeper/
3、重命名conf/zoo_sample.cfg 为conf/zoo.cfg
4、编辑 conf/zoo.cfg，修改dataDir=D:\zookeeper-3.3.6\data\
4、运行bin/zkServer.cmd

这里写图片描述

启动zookeeper如下
这里写图片描述

kafka搭建

1、打开链接：http://kafka.apache.org/downloads.html下载kafka2.1.2
2、打开config目录下的server.properties，修改log.dirs为D:\kafka_logs,

############################# Log Basics #############################
# A comma seperated list of directories under which to store log files
log.dirs=/tmp/kafka-logs
#log.dirs=E:\develop\kafka\kafka_logs

3、修改advertised.host.name=服务器ip

# root directory for all kafka znodes.
zookeeper.connect=localhost:2181

# Timeout in ms for connecting to zookeeper
zookeeper.connection.timeout.ms=6000

advertised.host.name=192.168.11.28

4、启动kafka ./bin/windows/kafka-server-start.bat ./config/server.preperties

这里写图片描述

kafka链接zookeeper

这里写图片描述

在go终端写入kafka

初始化NewConfig配置 sarama.NewConfig
创建生产者sarama.NewSyncProducer
创建消息sarama.ProducerMessage
发送消息client.SendMessage

package main

import (
    "fmt"
    "github.com/Shopify/sarama"
)

func main() {
    config := sarama.NewConfig()
    config.Producer.RequiredAcks = sarama.WaitForAll
    config.Producer.Partitioner = sarama.NewRandomPartitioner
    config.Producer.Return.Successes = true

    msg := &sarama.ProducerMessage{}
    msg.Topic = "nginx_log"
    msg.Value = sarama.StringEncoder("this is a good test, my message is good")

    client, err := sarama.NewSyncProducer([]string{"192.168.11.28:9092"}, config)
    if err != nil {
        fmt.Println("producer close, err:", err)
        return
    }

    defer client.Close()

    pid, offset, err := client.SendMessage(msg)
    if err != nil {
        fmt.Println("send message failed,", err)
        return
    }

    fmt.Printf("pid:%v offset:%v\n", pid, offset)
}

终端执行如下：我这里有个问题

PS E:\golang\go_pro\src\safly> go run main.go
send message failed, kafka server: In the middle of a leadership election, there is currently no leader for this partition and hence it is unavailable for writes.
PS E:\golang\go_pro\src\safly> go run main.go
pid:2 offset:0
pid:0 offset:0
pid:5 offset:0
pid:3 offset:0
pid:7 offset:0
pid:0 offset:1
pid:2 offset:1
pid:0 offset:2
pid:3 offset:1
pid:2 offset:2
pid:5 offset:1
pid:1 offset:0