C/C++ 使用librdkafka库实现的消费者

本文档详细介绍了如何使用C/C++编程语言,结合librdkafka库来实现Kafka消息消费者的步骤。通过示例代码consumer.c,展示了如何连接到Kafka集群,订阅主题并消费消息。
摘要由CSDN通过智能技术生成
/*
 * librdkafka - Apache Kafka C library
 *
 * Copyright (c) 2019, Magnus Edenhill
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are met:
 *
 * 1. Redistributions of source code must retain the above copyright notice,
 *    this list of conditions and the following disclaimer.
 * 2. Redistributions in binary form must reproduce the above copyright notice,
 *    this list of conditions and the following disclaimer in the documentation
 *    and/or other materials provided with the distribution.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
 * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
 * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
 * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
 * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
 * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
 * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
 * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
 * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
 * POSSIBILITY OF SUCH DAMAGE.
 */

/**
 * Simple high-level balanced Apache Kafka consumer
 * using the Kafka driver from librdkafka
 * (https://github.com/edenhill/librdkafka)
 */

#include <stdio.h>
#include <signal.h>
#include <string.h>
#include <ctype.h>


/* Typical include path would be <librdkafka/rdkafka.h>, but this program
 * is builtin from within the librdkafka source tree and thus differs. */
//#include <librdkafka/rdkafka.h>
#include "rdkafka.h"


static volatile sig_atomic_t run = 1;

/**
 * @brief Signal termination of program
 */
static void stop (int sig) {
   
        run = 0;
}



/**
 * @returns 1 if all bytes are printable, else 0.
 */
static int is_printable (const char *buf, size_t size) {
   
        size_t i;

        for (i = 0 ; i < size ; i++)
                if (!isprint((int)buf[i]))
                        return 0;

        return 1;
}


int main (int argc, char **argv) {
   
        rd_kafka_t *rk;          /* Consumer instance handle */
        rd_kafka_conf_t *conf;   /* Temporary configuration object */
        rd_kafka_resp_err_t err; /* librdkafka API error code */
        char errstr[512];        /* librdkafka API error reporting buffer */
        const char *brokers;     /
librdkafka 是一个 C/C++ 语言编写的 Apache Kafka 客户端,它提供了高级别和低级别的 API,使得 Kafka 的生产者和消费者的开发变得更加简单和灵活。下面介绍 librdkafka 中的消费者类: librdkafka 中的消费者类是 `RdKafka::Consumer`,它封装了消费者的相关操作。使用该类需要先创建一个 `RdKafka::Conf` 对象来设置消费者的配置,然后创建一个 `RdKafka::KafkaConsumer` 对象来实例化一个消费者。 具体的代码如下: ```c++ RdKafka::Conf *conf = RdKafka::Conf::create(RdKafka::Conf::CONF_GLOBAL); conf->set("bootstrap.servers", "localhost:9092"); // 设置 Kafka broker 地址 conf->set("group.id", "test-group"); // 设置消费者组 ID std::string errstr; RdKafka::KafkaConsumer *consumer = RdKafka::KafkaConsumer::create(conf, errstr); if (consumer == nullptr) { std::cerr << "Failed to create consumer: " << errstr << std::endl; exit(1); } // 订阅主题 std::vector<std::string> topics = {"test-topic"}; RdKafka::ErrorCode err = consumer->subscribe(topics); if (err != RdKafka::ERR_NO_ERROR) { std::cerr << "Failed to subscribe topic: " << RdKafka::err2str(err) << std::endl; exit(1); } // 开始消费消息 while (true) { RdKafka::Message *msg = consumer->consume(1000); // 每隔 1 秒轮询一次 if (msg == nullptr) { continue; } if (msg->err() == RdKafka::ERR_NO_ERROR) { std::cout << "Received message: " << std::string(static_cast<char *>(msg->payload()), msg->len()) << std::endl; } else { std::cerr << "Failed to consume message: " << RdKafka::err2str(msg->err()) << std::endl; } delete msg; } // 清理资源 consumer->close(); delete consumer; delete conf; ``` 上述代码中,首先创建了一个 `RdKafka::Conf` 对象,并设置了 `bootstrap.servers` 和 `group.id` 两个配置项。然后使用该对象创建一个 `RdKafka::KafkaConsumer` 对象,并订阅了一个名为 `test-topic` 的主题。最后进入消息消费循环,每隔 1 秒钟轮询一次消息,如果有消息到来则打印消息内容。 需要注意的是,消费者使用完毕后需要调用 `close` 方法来清理资源。
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值