php7操作Kafka 2.13安装与实例

最新推荐文章于 2024-08-22 16:38:35 发布

张耘华

最新推荐文章于 2024-08-22 16:38:35 发布

阅读量269

点赞数

分类专栏： PHP

本文链接：https://blog.csdn.net/oqzuser12345678999q/article/details/107605379

版权

PHP 专栏收录该内容

21 篇文章 1 订阅

订阅专栏

安装Php 扩展：

pecl install rdkafa 如果报错 re2c-1.0.2 之类的错误，请安装 re2c-1.0.2.tar

PHP例子：也可以参考官方的文档，https://arnaud.le-blanc.net/php-rdkafka-doc/phpdoc/rdkafka.examples-high-level-consumer.html

//生产例子
$conf = new RdKafka\Conf();
$conf->set('metadata.broker.list', '192.168.0.33:9092,192.168.0.185:9092,192.168.0.193:9092');
$objRdKafka = new RdKafka\Producer($conf);
$topic = $objRdKafka->newTopic("test");
//$oObjTopic = $objRdKafka->newTopic("test");
// 发送消息
$sMsg = json_encode(["code" => 200, "data" => "sfsfsfs"]);
$topic->produce(RD_KAFKA_PARTITION_UA, 0, $sMsg);
$objRdKafka->poll(0);
$result = $objRdKafka->flush(10000);
var_dump($result);

//消息例子
$conf = new RdKafka\Conf();

// Set a rebalance callback to log partition assignments (optional)
$conf->setRebalanceCb(function (RdKafka\KafkaConsumer $kafka, $err, array $partitions = null) {
    switch ($err) {
        case RD_KAFKA_RESP_ERR__ASSIGN_PARTITIONS:
            echo "Assign: ";
            var_dump($partitions);
            $kafka->assign($partitions);
            break;

        case RD_KAFKA_RESP_ERR__REVOKE_PARTITIONS:
            echo "Revoke: ";
            var_dump($partitions);
            $kafka->assign(NULL);
            break;

        default:
            throw new \Exception($err);
    }
});

// Configure the group.id. All consumer with the same group.id will consume
// different partitions.
$conf->set('group.id', 'myConsumerGroup');

// Initial list of Kafka brokers
$conf->set('metadata.broker.list', '192.168.0.33:9092,192.168.0.185:9092,192.168.0.193:9092');

// Set where to start consuming messages when there is no initial offset in
// offset store or the desired offset is out of range.
// 'smallest': start from the beginning
$conf->set('auto.offset.reset', 'smallest');

$consumer = new RdKafka\KafkaConsumer($conf);

// Subscribe to topic 'test'
$consumer->subscribe(['test']);

echo "Waiting for partition assignment... (make take some time when\n";
echo "quickly re-joining the group after leaving it.)\n";

while (true) {
    $message = $consumer->consume(120 * 1000);
    switch ($message->err) {
        case RD_KAFKA_RESP_ERR_NO_ERROR:
            var_dump($message);
            break;
        case RD_KAFKA_RESP_ERR__PARTITION_EOF:
            echo "No more messages; will wait for more\n";
            break;
        case RD_KAFKA_RESP_ERR__TIMED_OUT:
            echo "Timed out\n";
            break;
        default:
            throw new Exception($message->errstr(), $message->err);
            break;
    }
}

Kafka集群的安装见我的另一篇BLOG，常用命令如：

注：kafka是否启动成功，如果未成功需根据错误查找解决的办法。

启动服务器
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-server-start.sh -daemon ../config/server.properties

关闭服务器
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-server-stop.sh

创建Topic
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic test

列出所有Topic：
bin/kafka-topics.sh -list -zookeeper 192.168.0.185:2181

启动Producer并发送消息：
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

启动Consumer并接收消息：

kafka-console-consumer.sh --bootstrap-server 192.168.0.185:9092,192.168.0.33:9092,192.168.0.193:9092 --topic test --from-beginning