spring cloud stream+Kafka实现消息传递

(一)  为什么要消息传递在构建微服务应用程序中很重要

         回答这个问题之前,首先介绍一个概念

事件驱动架构(EDA):使用异步消息实现事件之间的通信,也被称为消息驱动架构(MDA).

       而基于EDA的方法允许开发人员构建高度解耦的系统,它可以对变更做出反应,而不需要与特定的库或者服务紧密耦合.当与微服务结合之后,.EDA通过让服务监听由应用程序发出的事件流(消息)的方式,允许开发人员迅速地向应用程序中添加新功能.

      也就是说,基于EDA的方法构建的系统具有的高度的解耦性以及扩展性;

 

(二)  Spring Cloud Stream 简介

1 、Spring Cloud Stream是一个由注解驱动的架构,它允许开发人员在Spring应用程序中构建消息发布者和消费者

2、Spring Cloud可以通过Spring Cloud Stream将消息传递集成到Spring 的微服务中

3、Spring Cloud Stream可以使用多个消息平台(包括Apache  Kafka和RabbitMQ),而平台的具体实现细节则被排除在应用程序之外.在应用程序中实现消息的发布和消费是通过平台无关的Spring接口实现的.

 

(三) 安装ZooKeeper与kafka

zookeeper安装与入门:https://blog.csdn.net/douya2016/article/details/102983218

kafka安装与学习:https://www.cnblogs.com/qingyunzong/p/9004509.html#_label3_1

(四)  Spring Cloud Stream + Kafka 实例

消息生产者生产消息,并向kafka发布,消息消费者1从kafka中消费消息,并负责中转;中转之后,将其发布到kafka,然后由comsumerGroup2组中的消息消费者消费;

项目结构如下:

其中

messgae-producer:消息生产者,负责向kafka中发布消息;

message-comsumer1:消息消费者1,负责消息中转

messgae-comsumer2:消息消费者2

messgae-comsumer3:消息消费者3

 

Spring Cloud Stream 与kafka依赖:

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>

1.message-producer项目

application.yml

server:
   port: 8750
spring:
   application:
      name: producer
   cloud:
      stream:
         bindings:
            output: #通道名称,使用stream默认的通道名称,可以自定义
               destination: stream-demo #要写入的消息队列的名称
               content-type: application/json #发送或接受什么类型的消息
         kafka: #使用kafka作为服务中的消息总线
            binder:
               zkNodes: localhost:2181 #zookeeper的网络位置,如果是集群,逗号分割
               brokers: localhost:9092 #kafka的网络位置   
               auto-create-topics: true

启动类

package com.dy.producer;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

/**
 * EnableBinding注解告诉Spring Cloud Steam将应用程序绑定到消息代理
 * @author dy
 *
 */
@SpringBootApplication
public class ProducerService {
	
	public static void main(String[] args) {
		SpringApplication.run(ProducerService.class, args);
	}
}

控制器MessageProducerController

package com.dy.producer.controller;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

import com.dy.producer.entiy.Message;
import com.dy.producer.service.SendService;

@RestController
public class MessageProducerController {

	@Autowired
	private SendService sendService;

	@PostMapping("/sendMsg")
	public void send(@RequestBody Message message) {
		sendService.sendMsg(message);
	}
	
}

服务类SendServiceImpl,通过默认的Source类上定义的一组通道与消息代理进行通信

package com.dy.producer.service;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.messaging.Source;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Service;

import com.dy.producer.entiy.Message;

@Service
@EnableBinding(Source.class)
public class SendServiceImpl implements SendService {

	@Autowired
	private Source source;

	@Override
	public void sendMsg(Message msg) {
		source.output().send(MessageBuilder.withPayload(msg).build());

	}

}

实体类Message

package com.dy.stream.entiy;

public class Message {
	String sender;
	String reciver;
	String messgaeName;
	String messageContent;
	
	public Message(String sender, String reciver, String messgaeName, String messageContent) {
		super();
		this.sender = sender;
		this.reciver = reciver;
		this.messgaeName = messgaeName;
		this.messageContent = messageContent;
	}
	
	public String getSender() {
		return sender;
	}
	public void setSender(String sender) {
		this.sender = sender;
	}
	public String getReciver() {
		return reciver;
	}
	public void setReciver(String reciver) {
		this.reciver = reciver;
	}
	public String getMessgaeName() {
		return messgaeName;
	}
	public void setMessgaeName(String messgaeName) {
		this.messgaeName = messgaeName;
	}
	public String getMessageContent() {
		return messageContent;
	}
	public void setMessageContent(String messageContent) {
		this.messageContent = messageContent;
	}
	
	@Override
	public String toString() {
		return sender + "-" + reciver + "-" + messgaeName + "-" + messageContent;
	}
}

2.message-comsumer1 消息中转

application.yml

server:
   port: 9751
spring:
   application:
      name: comsumer1
   cloud:
      stream:
         bindings:
            input: #通道名称,使用stream默认的通道名称,可以自定义, 接受消息生产者生产的消息 
               destination: stream-demo #要写入的消息队列的名称
 #              group: comsumerGroup1 #该属性确保服务只处理一次
            output:
               destination: stream-demo-trans #转发
         kafka: #使用kafka作为服务中的消息总线
            binder:
               zkNodes: localhost:2181 #zookeeper的网络位置,如果是集群,逗号分割
               brokers: localhost:9092 #kafka的网络位置
               auto-create-topics: true

启动类

package com.dy.stream;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class ComsumerApplication1 {
	public static void main(String[] args) {
		SpringApplication.run(ComsumerApplication1.class, args);
	}
}

SendToBinder:消息代理借口

package com.dy.stream.entiy;

import org.springframework.cloud.stream.annotation.Input;
import org.springframework.cloud.stream.annotation.Output;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.SubscribableChannel;

public interface SendToBinder {
	@Output("out")
	MessageChannel output();
	
	@Input("input")
	SubscribableChannel input();
}

TransFormService:绑定自己的SendToBinder接口，然后监听input,返回ACK表示中转站收到消息了，再转发消息出去

package com.dy.stream.entiy;

import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.messaging.handler.annotation.SendTo;

/**
 * 定自己的SendToBinder接口，然后监听input,返回ACK表示中转站收到消息了，再转发消息出去，代码如下：
 * @author dy
 *
 */
@EnableBinding(SendToBinder.class)
public class TransFormService {

	@StreamListener("input")
	//SendTo注解:给别人一个反馈ACK
	@SendTo("output")
	public Object transform(Message payload) {
		System.out.println("消息中转：" + payload.toString());
		return payload;
	}

}

3.message-comsumer2

application.yml

server:
   port: 9752
spring:
   application:
      name: comsumer2
   cloud:
      stream:
         bindings:
            input: #通道名称,使用stream默认的通道名称,可以自定义
               destination: stream-demo-trans #要写入的消息队列的名称
#            content-type: text/plain   #发送或接受什么类型的消息
               group: comsumerGroup2 #该属性确保服务只处理一次
         kafka: #使用kafka作为服务中的消息总线
            binder:
               zkNodes: localhost:2181 #zookeeper的网络位置,如果是集群,逗号分割
               brokers: localhost:9092 #kafka的网络位置    
               auto-create-topics: true

启动类

package com.dy.stream;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;

@SpringBootApplication
public class ComsumerApplication2 {	
	public static void main(String[] args) {
		SpringApplication.run(ComsumerApplication2.class, args);
	}
}

RecieveService:使用Spring Sink接口监听默认通道input上传入的消息:

package com.dy.stream.entiy;

import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.annotation.StreamListener;
import org.springframework.cloud.stream.messaging.Sink;

@EnableBinding(Sink.class)
public class RecieveService {
	
	// 每次收到来自input通道消息时, Spring Cloud Stream将执行此方法
	@StreamListener(Sink.INPUT)
	public void recieve(Message payload) {
		System.out.println(payload.toString());
	}
}

4.message-comsumer3

同message-comsumer2

 

分别启动以上项目,发送messgae

各模块运行结果如下:

message-producer:

message-comsumer1:消息中转

message-comsumer2:消息消费者2

message-comsumer3:消息消费者3

从以上模块的结果可以看出,message-producer将消息分布给kafka,message-comsumer1从kafka中接收消息后对消息进行了中转,而message-comsumer2和message-comsumer3由于同属于一个组,因此只有一个消费者从kafka中订阅到了消息;

 

项目下载地址:

https://github.com/xdouya/Spring-Cloud-Stream-kafka/tree/master