spring cloud stream kafka 动态写入不同的topic(Using dynamically bound destinations)

引入依赖：

 <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>

编写一个controller:

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.stream.annotation.EnableBinding;
import org.springframework.cloud.stream.binding.BinderAwareChannelResolver;
import org.springframework.http.HttpStatus;
import org.springframework.messaging.support.GenericMessage;
import org.springframework.web.bind.annotation.*;

import static org.springframework.web.bind.annotation.RequestMethod.POST;

@EnableBinding
@RestController
public class DynamicDestinationKafkaProducerController {


    @Autowired
    private BinderAwareChannelResolver resolver;

    /**
     * @param body   kafka消息内容
     * @param target 目标topic
     */
    @RequestMapping(path = "/topics/{target}", method = POST, consumes = "*/*")
    @ResponseStatus(HttpStatus.ACCEPTED)
    public void sendKafkaMessage(@RequestBody byte[] body, @PathVariable("target") String target) {

        sendMessage(body, target);
    }




    private void sendMessage(String body, String target) {
        sendMessage(body.getBytes(), target);
    }

    private void sendMessage(byte[] body, String target) {
        resolver.resolveDestination(target).send(new GenericMessage<>(body));
    }


}

application.yml配置：

server:
  port: 7007
management:
  security:
    enabled: false
spring:
  cloud:
    stream:
      default:
        binder: kafka
        producer:
          #为了兼容非java端消费消息，使用原生encoding，不会有content-type头信息在消息体中，bytearray序列化，消费端直接获取原始字符串自己处理。
          useNativeEncoding: true
      #指定用kafka stream来作为默认消息中间件
      default-binder: kafka
      kafka:
        #来自KafkaBinderConfigurationProperties
        binder:
          brokers: @kafka.brokers@
          zkNodes: @kafka.zkNodes@
logging:
  path: /data/logs/${spring.application.name}
  level:
    com.netflix.discovery.shared.resolver.aws.ConfigClusterResolver: WARN

启动服务，发送请求到http://localhost:7007/topics/your_topic,消息体为任意字符串放在request body中，这样会自动将消息体发送到your_topic这个topic里面去

接下来我们在源码层面分析一下实现原理，首先是：

resolver.resolveDestination(target)

这个方法的实现:

@Override
public MessageChannel resolveDestination(String channelName) {
   try {
      return super.resolveDestination(channelName);
   }
   catch (DestinationResolutionException e) {
      // intentionally empty; will check again while holding the monitor
   }
   synchronized (this) {
      DestinationResolutionException destinationResolutionException;
      try {
         return super.resolveDestination(channelName);
      }
      catch (DestinationResolutionException e) {
         destinationResolutionException = e;
      }
      MessageChannel channel = null;
      if (this.beanFactory != null) {
         String[] dynamicDestinations = null;
         BindingServiceProperties bindingServiceProperties = this.bindingService
               .getBindingServiceProperties();
         if (bindingServiceProperties != null) {
            dynamicDestinations = bindingServiceProperties.getDynamicDestinations();
         }
         boolean dynamicAllowed = ObjectUtils.isEmpty(dynamicDestinations)
               || ObjectUtils.containsElement(dynamicDestinations, channelName);
         if (dynamicAllowed) {
            channel = this.bindingTargetFactory.createOutput(channelName);
            this.beanFactory.registerSingleton(channelName, channel);
            channel = (MessageChannel) this.beanFactory.initializeBean(channel, channelName);
            Binding<MessageChannel> binding = this.bindingService.bindProducer(channel, channelName);
            this.dynamicDestinationsBindable.addOutputBinding(channelName, binding);
         }
         else {
            throw destinationResolutionException;
         }
      }
      return channel;
   }
}

大致意思是首先在spring容器中查找有没有bean name=channelName(即我们传进来的target参数)的MessageChannel bean定义，有就直接返回，没有的话就会创建一个然后注册到容器当中去。

这里的channelName在配置文件里面对应的就是spring.cloud.stream.bindings.<channelName>相关的配置。例如:

spring:
  cloud:
    stream:
      default:
        binder: kafka1
        producer:
          useNativeEncoding: true
      #指定用kafka stream来作为默认消息中间件
      default-binder: kafka1
      #属性来自BindingProperties
      bindings:
        #与@StreamListener注解中的value一致，是绑定的渠道名
        input_1:
          binder: kafka1
          consumer:
            headerMode: raw
          #绑定的kafka topic名称为test
          destination: cloud-test10

其中的input_1就是channelName,input_1下面的配置对应的就是这个MessageChannel的配置,每一个channel的配置都是一个BindingProperties的实例，有哪些可配置的项就看这个类有哪些属性就行了。

所以如果有匹配到bindings相应的channelName配置，那么就使用该ChannelName，否则就创建一个新的Channel,使用默认配置。

dd

那么默认配置是怎么配置的呢？再看看源码,

刚才resolveDestination方法中有一段代码：
channel = this.bindingTargetFactory.createOutput(channelName); 

用来创建一个channel,具体实现在SubscribableChannelBindingTargetFactory.createOutput :

@Override
public SubscribableChannel createOutput(String name) {
   SubscribableChannel subscribableChannel = new DirectChannel();
   this.messageChannelConfigurer.configureOutputChannel(subscribableChannel, name);
   return subscribableChannel;
}

这里可以看到创建的是一个DirectChannel,即在当前线程中同步发送消息，另外还有一个实现时ExecutorChannel另起线程发送消息。

我们重点看一下下面这行代码：
this.messageChannelConfigurer.configureOutputChannel(subscribableChannel, name);
实现在MessageConverterConfigurer.configureMessageChannel：

/**
 * Setup data-type and message converters for the given message channel.
 *
 * @param channel message channel to set the data-type and message converters
 * @param channelName the channel name
 */
private void configureMessageChannel(MessageChannel channel, String channelName, boolean input) {
   Assert.isAssignable(AbstractMessageChannel.class, channel.getClass());
   AbstractMessageChannel messageChannel = (AbstractMessageChannel) channel;
   final BindingProperties bindingProperties = this.bindingServiceProperties.getBindingProperties(
         channelName);
   final String contentType = bindingProperties.getContentType();
   ProducerProperties producerProperties = bindingProperties.getProducer();
   if (!input && producerProperties != null && producerProperties.isPartitioned()) {
      messageChannel.addInterceptor(new PartitioningInterceptor(bindingProperties,
            getPartitionKeyExtractorStrategy(producerProperties),
            getPartitionSelectorStrategy(producerProperties)));
   }
   if (StringUtils.hasText(contentType)) {
      messageChannel.addInterceptor(new ContentTypeConvertingInterceptor(contentType, input));
   }
}

这个方法中的这行代码：

final BindingProperties bindingProperties = this.bindingServiceProperties.getBindingProperties(
         channelName);

根据channelName去获取对应的bingingProperties，我们点进去看getBindingProperties的实现：

public BindingProperties getBindingProperties(String bindingName) {
   BindingProperties bindingProperties = new BindingProperties();
   if (this.bindings.containsKey(bindingName)) {
      BeanUtils.copyProperties(this.bindings.get(bindingName), bindingProperties);
   }
   if (bindingProperties.getDestination() == null) {
      bindingProperties.setDestination(bindingName);
   }
   return bindingProperties;
}

注意这里的this.bindings其实不是TreeMap实例，而是 EnvironmentEntryInitializingTreeMap实例,在这个方法中实例化的：

@Override
public void setEnvironment(Environment environment) {
   if (environment instanceof ConfigurableEnvironment) {
      // override the bindings store with the environment-initializing version if in
      // a Spring context
      Map<String, BindingProperties> delegate = new TreeMap<String, BindingProperties>(
            String.CASE_INSENSITIVE_ORDER);
      delegate.putAll(this.bindings);
      this.bindings = new EnvironmentEntryInitializingTreeMap<>((ConfigurableEnvironment) environment,
            BindingProperties.class, "spring.cloud.stream.default", delegate);
   }
}

而EnvironmentEntryInitializingTreeMap实现了重写了get和containsKey方法：

@Override
public T get(Object key) {
   if (!this.delegate.containsKey(key) && key instanceof String) {
      T entry = BeanUtils.instantiate(entryClass);
      RelaxedDataBinder defaultsDataBinder = new RelaxedDataBinder(entry, defaultsPrefix);
      defaultsDataBinder.bind(new PropertySourcesPropertyValues(environment.getPropertySources()));
      this.delegate.put((String) key, entry);
   }
   return this.delegate.get(key);
}

@Override
public boolean containsKey(Object key) {
   return get(key) != null;
}

从get方法的实现可以看出，如果bindings里面没有包含指定channelName的key的时候，就会自动创建一个BindingProperties对象，然后塞到bindings中去,默认的BindingProperties配置也是在这生成的，获取的是defaultsPrefix前缀中定义的属性,defaultPrefix=spring.cloud.stream.default，即读取了

spring:
  cloud:
    stream:
      default:
        binder: kafka
        producer:
          #为了兼容非java端消费消息，使用原生encoding，不会有content-type头信息在消息体中，bytearray序列化，消费端直接获取原始字符串自己处理。
          useNativeEncoding: true

这里的default下面的对于binding channel的默认配置

我们在回过头来看getBindingProperties这个方法里面的这段代码：

if (this.bindings.containsKey(bindingName)) {
   BeanUtils.copyProperties(this.bindings.get(bindingName), bindingProperties);
}

由于 EnvironmentEntryInitializingTreeMapd的覆盖实现,所以这个if条件永远都成立，然后拷贝属性到bindingProperties中去作为这个新创建的channel的默认配置属性.

然后我们在分析一下为什么传入的target参数就是对应写入的topic,这点很简单由于我们没有配置bindingProperties中的

destination

属性，那么就会用channelName作为默认的topic name,这点在上面分析的那个getBindingProperties方法中有体现：

public BindingProperties getBindingProperties(String bindingName) {
   BindingProperties bindingProperties = new BindingProperties();
   if (this.bindings.containsKey(bindingName)) {
      BeanUtils.copyProperties(this.bindings.get(bindingName), bindingProperties);
   }
   if (bindingProperties.getDestination() == null) {
      bindingProperties.setDestination(bindingName);
   }
   return bindingProperties;
}

其中：

 if (bindingProperties.getDestination() == null) {
      bindingProperties.setDestination(bindingName);
   }

至此真个流程就大致分析完了。

参考http://cloud.spring.io/spring-cloud-static/Dalston.SR3/#dynamicdestination