常用中间件 环境搭建 和 简单 使用

目录

简述

消息中间件Kafka

windows 简单搭建 单机 kafka 环境

Java客户端代码

生产者：

消费者

kafka集群配置

消息中间件rocketmq

window搭建rocketmq

Java 客户端代码

生产者代码

消费者代码

---

 

简述

如果提供一个中间件，一定会有一个 server 服务端，提供服务；一个 client 客户端 提供api，用来和服务端做交互。不论是 缓存redis、注册中心 zookeeper、消息中间件 kafka/rocketmq，都是这种。

搭建环境总体思路：

第一步：官网下载服务jar包

第二步：配置环境变量、配置文件

第三步：命令行 启动服务端

第四步：命令行 和服务端交互

如果在项目里使用，则环境搭建好后，在项目中 引入依赖包，使用其提供的 客户端API 来完成功能即可。

消息中间件Kafka

windows 简单搭建 单机 kafka 环境

1. 下载 zookeeper  3.7.0 /kafka_2.12-2.8.0

2. windows 环境下 启动zookeeper 

zkServer.cmd

3. windows 环境下 启动 kafka

.\bin\windows\kafka-server-start.bat .\config\server.properties

4. kafka 启动后，创建一个 topic

kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

5. topic 创建后，启动 producer

kafka-console-producer.bat --broker-list localhost:9092 --topic test

6. topic创建后，启动 consumer

kafka-console-consumer.bat --bootstrap-server localhost:9092 --topic test

7. producer 生成消息，consume消费消息

简单的 kafka windows 单机环境 就搭建好了。

其中，有很多配置 都是默认的，如果需要，则修改 配置文件就可以了。

Java客户端代码

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka_2.13</artifactId>
            <version>2.8.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.zookeeper</groupId>
            <artifactId>zookeeper</artifactId>
            <version>3.7.0</version>
        </dependency>

生产者：

package wxj.test.kafka;

import org.apache.kafka.clients.producer.*;
import org.apache.kafka.common.serialization.StringSerializer;

import java.util.Properties;

public class TestProducer {

    static volatile boolean send = false;

    public static void main(String[] args) {
        /**
         * 1. 构造一个 生产者 对象
         */

        /**
         * 1.1 属性配置
         */
        Properties properties = new Properties();
        properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
        properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());

        /**
         * 1.2 构造实例
         */
        KafkaProducer<String,String> producer = new KafkaProducer<String, String>(properties);

        /**
         * 2. 包装 需要发送的消息
         */
        ProducerRecord<String,String> record = new ProducerRecord<>("test","new message");


        /**
         * 3. 生产者发送消息
         */
        producer.send(record, new Callback() {
            @Override
            public void onCompletion(RecordMetadata recordMetadata, Exception e) {
                if (null != e){
                    System.out.println("error");
                }else {
                    System.out.println("success");
                }
                send = true;
            }
        });
        while (!send){
        }
        producer.close();

    }
}

消费者

package wxj.test.kafka;

import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.time.Duration;
import java.util.Arrays;
import java.util.Iterator;
import java.util.Properties;

public class TestConsumer {


    public static void main(String[] args) {
        /**
         * 1. 构造一个 消费者对象
         */

        /**
         * 1.1 属性配置
         */
        Properties properties = new Properties();
        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
        properties.put(ConsumerConfig.GROUP_ID_CONFIG,"0");
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());

        /**
         * 1.2 构造实例
         */
        KafkaConsumer<String,String> consumer = new KafkaConsumer<String, String>(properties);

        /**
         * 1.3 设置订阅的 topic
         */
        consumer.subscribe(Arrays.asList("test"));

        while (true){
            ConsumerRecords<String, String> records = consumer.poll(Duration.ofSeconds(1));
            Iterator<ConsumerRecord<String, String>> iterator = records.iterator();
            while (iterator.hasNext()){
                ConsumerRecord<String, String> record = iterator.next();
                System.out.println("has record : " + record);
            }
        }

    }
}

查看kafka消息堆积情况

kafka-consumer-groups.bat --bootstrap-server localhost:9092 --describe --group 0

kafka集群配置

单机就是一个 broker，配置文件不用改

集群，主要修改配置文件：

broker.id=0

log.dirs=/tmp/kafka0-logs

listeners=PLAINTEXT://:9092

zookeeper地址要配成一样的，比如本地的话就都设置为：

zookeeper.connect=localhost:2181

消息中间件rocketmq

window搭建rocketmq

1. 下载 rocketmq

2. 配置环境变量

ROCKETMQ_HOME="D:\rocketmq"
NAMESRV_ADDR="localhost:9876"

3.  runserver.cmd 和 runbroker.cmd 稍作修改

JAVA_HOME 和 CLASSPATH 的地方 加 ""

如：

set CLASSPATH=.;%BASE_DIR%conf;"%CLASSPATH%"

set "JAVA_OPT=%JAVA_OPT% -Djava.ext.dirs=%BASE_DIR%lib;"%JAVA_HOME%"\jre\lib\ext"

4. 启动 rocketmq

>mqnamesrv.cmd

>mqbroker.cmd -n localhost:9876 autoCreateTopicEnable=true

5. 生产者发消息

>tools.cmd org.apache.rocketmq.example.quickstart.Producer

6. 消费者消费

>tools.cmd org.apache.rocketmq.example.quickstart.Consumer

完毕

Java 客户端代码

加入依赖包

        <dependency>
            <groupId>org.apache.rocketmq</groupId>
            <artifactId>rocketmq-client</artifactId>
            <version>4.3.0</version>
        </dependency>

生产者代码

private static void oneWayProducer() throws Exception {
        //Instantiate with a producer group name.
        DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
        // Specify name server addresses.
        producer.setNamesrvAddr("localhost:9876");
        //Launch the instance.
        producer.start();
        for (int i = 0; i < 100; i++) {
            //Create a message instance, specifying topic, tag and message body.
            Message msg = new Message("TopicTest" /* Topic */,
                    "TagA" /* Tag */,
                    ("Hello RocketMQ " +
                            i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
            );
            //Call send message to deliver message to one of brokers.
            producer.sendOneway(msg);
        }
        //Wait for sending to complete
        Thread.sleep(5000);
        producer.shutdown();
    }

private static void asyncProducer() throws Exception {
        //Instantiate with a producer group name.
        DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");
        // Specify name server addresses.
        producer.setNamesrvAddr("localhost:9876");
        //Launch the instance.
        producer.start();
        producer.setRetryTimesWhenSendAsyncFailed(0);

        int messageCount = 100;
        final CountDownLatch countDownLatch = new CountDownLatch(messageCount);
        for (int i = 0; i < messageCount; i++) {
            try {
                final int index = i;
                Message msg = new Message("Jodie_topic_1023",
                        "TagA",
                        "OrderID188",
                        "Hello world".getBytes(RemotingHelper.DEFAULT_CHARSET));
                producer.send(msg, new SendCallback() {
                    @Override
                    public void onSuccess(SendResult sendResult) {
                        countDownLatch.countDown();
                        System.out.printf("%-10d OK %s %n", index, sendResult.getMsgId());
                    }

                    @Override
                    public void onException(Throwable e) {
                        countDownLatch.countDown();
                        System.out.printf("%-10d Exception %s %n", index, e);
                        e.printStackTrace();
                    }
                });
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        countDownLatch.await(5, TimeUnit.SECONDS);
        producer.shutdown();
    }

private static void syncProducer() throws Exception{
        //Instantiate with a producer group name.
        DefaultMQProducer producer = new
                DefaultMQProducer("please_rename_unique_group_name");
        // Specify name server addresses.
        producer.setNamesrvAddr("localhost:9876");
        //Launch the instance.
        producer.start();
        for (int i = 0; i < 100; i++) {
            //Create a message instance, specifying topic, tag and message body.
            Message msg = new Message("TopicTest" /* Topic */,
                    "TagA" /* Tag */,
                    ("Hello RocketMQ " +
                            i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
            );
            //Call send message to deliver message to one of brokers.
            SendResult sendResult = producer.send(msg);
            System.out.printf("%s%n", sendResult);
        }
        //Shut down once the producer instance is not longer in use.
        producer.shutdown();
    }

消费者代码

private static void consumer() throws MQClientException {
        // Instantiate with specified consumer group name.
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("please_rename_unique_group_name");

        // Specify name server addresses.
        consumer.setNamesrvAddr("localhost:9876");

        // Subscribe one more more topics to consume.
        consumer.subscribe("TopicTest", "*");
        // Register callback to execute on arrival of messages fetched from brokers.
        consumer.registerMessageListener(new MessageListenerConcurrently() {

            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
                                                            ConsumeConcurrentlyContext context) {
                System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), msgs);
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });

        //Launch the consumer instance.
        consumer.start();

        System.out.printf("Consumer Started.%n");
    }