java牛客网⑤ Kafka，构建TB级异步消息系统

第5章 Kafka，构建TB级异步消息系统

5.1 阻塞队列

生产者与消费者模式 demo

package com.nowcoder.community;

import java.util.Random;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;

public class BlockingQueueTests {
    public static void main(String[] args) {
        // 队列中最多只能存十个数
        BlockingQueue queue = new ArrayBlockingQueue(10);
        new Thread(new Prodocer(queue)).start();
        new Thread(new Consumer(queue)).start();
        new Thread(new Consumer(queue)).start();
        new Thread(new Consumer(queue)).start();


    }
}

class Prodocer implements Runnable{

    private BlockingQueue<Integer> queue;

    public Prodocer(BlockingQueue<Integer> queue){
        this.queue = queue;
    }

    @Override
    public void run() {
        try {
            for (int i=0; i<100; i ++) {
                Thread.sleep(20);
                queue.put(i);
                System.out.println(Thread.currentThread().getName() + "生产者" +queue.size());
            }

        }catch (Exception e){
            e.printStackTrace();
        }
    }
}

class Consumer implements Runnable{

    private BlockingQueue<Integer> queue;

    public Consumer(BlockingQueue<Integer> queue){
        this.queue = queue;
    }


    @Override
    public void run() {
        try {
            while (true) {
                Thread.sleep(new Random().nextInt(1000));
                queue.take();
                System.out.println(Thread.currentThread().getName() + "消费者" + queue.size());
            }
        }catch (Exception e){
            e.printStackTrace();
        }
    }
}

5.5 Kafka入门

broker kafka的服务器，每一台服务器为一个broker
Zookeeper 管理kafka的集群，kafka有内置的Zookeeper

Topic 可以理解为消息存放位置
Partition 为主题的分区，分区追加数据
Oddset 消息存放在Partiton的 index 或 序列

Replica 副本存多份，提高容错率
Leader Replica 存数据做相响应， Follower Replica 只是做备份，当主服务挂掉，备份上

kafka的下载地址 http://archive.apache.org/dist/kafka/

1.启动zookeeper

2. 启动kafka

D:\work_program\kafka_2.13-2.5.0>bin\windows\kafka-server-start.bat config\server.properties

3. 创建Topic 主题

kafka-topics.bat --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic test)

4. 查看主题

kafka-topics.bat --list --bootstrap-server localhost:9092

5. 启动生产者

D:\work_program\kafka_2.13-2.5.0\bin\windows>kafka-console-producer.bat --broker-list localhost:9092 -topic test

6. 启动消费者

kafka-console-consumer.bat --bootstrap-server localhost:9092 -topic test from-beginning

5.9 Spring整合Kafka