springboot怎么使用Tomcat集群数据同步功能？

前言

在上一文中我们介绍了Tomcat的集群功能以及原理，现在可以使用借助这个功能在springboot项目实现广播的功能。

需求如下：

类似一个远程集合对象，我往集合当中设置某个值，其他节点就能收到这个改变。

前置知识

1. springboot使用的阉割版的嵌入式tomcat,功能并没有那么全，如果需要使用Tomcat的集群功能，需要引入以下几个包。

<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-tribes</artifactId>
    <version>9.0.60</version>
</dependency>
<dependency>
    <groupId>org.apache.tomcat</groupId>
    <artifactId>tomcat-catalina-ha</artifactId>
    <version>9.0.60</version>
</dependency>
<dependency>
    <groupId>org.apache.tomcat.embed</groupId>
    <artifactId>tomcat-embed-jasper</artifactId>
    <version>9.0.60</version>
</dependency>

2. tomcat是很多层的tomcat的启动时都是一层一层加载，这个层级就是server.xml当中的 xml的元素例如

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<!-- Note:  A "Server" is not itself a "Container", so you may not
     define subcomponents such as "Valves" at this level.
     Documentation at /docs/config/server.html
 -->
<Server port="8005" shutdown="SHUTDOWN">
   .
   .
   .
   .
  <Service name="Catalina">
    <Connector port="8080" protocol="HTTP/1.1"
               connectionTimeout="20000"
               redirectPort="8443" />
    <Engine name="Catalina" defaultHost="localhost">
      <Host name="localhost"  appBase="webapps"
            unpackWARs="true" autoDeploy="true">
      </Host>
    </Engine>
  </Service>
</Server>

3. 原生的tomcat 使用tomcat的集群功能需要在server.xml的 Engine或者Host当中配置

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">

          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>
          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>
            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
          </Channel>
          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=""/>
          <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>
          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>

实现思路 找到这个Host对象，然后将这个xml所表示的设置到其中去。

springboot怎么创建Tomcat的呢？

通过查看web容器工厂源码可以看到

@Override
	public WebServer getWebServer(ServletContextInitializer... initializers) {
		if (this.disableMBeanRegistry) {
			Registry.disableRegistry();
		}
		Tomcat tomcat = new Tomcat();
		File baseDir = (this.baseDirectory != null) ? this.baseDirectory : createTempDir("tomcat");
		tomcat.setBaseDir(baseDir.getAbsolutePath());
		for (LifecycleListener listener : this.serverLifecycleListeners) {
			tomcat.getServer().addLifecycleListener(listener);
		}
		Connector connector = new Connector(this.protocol);
		connector.setThrowOnFailure(true);
		tomcat.getService().addConnector(connector);
		customizeConnector(connector);
		tomcat.setConnector(connector);
		tomcat.getHost().setAutoDeploy(false);
		configureEngine(tomcat.getEngine());
		for (Connector additionalConnector : this.additionalTomcatConnectors) {
			tomcat.getService().addConnector(additionalConnector);
		}
        //容器构造之前  所以这一块应该有个扩展点 我们继续看代码
        //并且可以看到这个Host已经很明显了 我们是可以拿到的
		prepareContext(tomcat.getHost(), initializers);
		return getTomcatWebServer(tomcat);
	}

protected void prepareContext(Host host, ServletContextInitializer[] initializers) {
        .
        .
        .
        host.addChild(context);
		configureContext(context, initializersToUse);
        //果然给我们发现了这个扩展点,且这个context是Host的子类
		postProcessContext(context);
	}

所以我们可以自定义自己的Tomcat容器工厂代码如下

@Component
public class MyTomcatServletWebServerFactory extends TomcatServletWebServerFactory{
    @Override
    protected void postProcessContext(Context context) {
        StandardHost host = (StandardHost) context.getParent();
        Engine parent = (Engine) host.getParent();
        //然后我们将xml当中的配置设置到Engine当中
        SimpleTcpCluster tcpCluster = new SimpleTcpCluster();
        tcpCluster.setChannelSendOptions(67);
        BackupManager backupManager=new BackupManager();
        backupManager.setMapSendOptions(6);
        backupManager.setContext(context);
        tcpCluster.registerManager(backupManager);
        DeltaManager deltaManager=new DeltaManager();
        deltaManager.setNotifySessionListenersOnReplication(true);
        deltaManager.setExpireSessionsOnShutdown(true);
        deltaManager.setContext(context);
        tcpCluster.registerManager(deltaManager);
        context.setManager(deltaManager);
        GroupChannel groupChannel=new GroupChannel();
        McastService mcastService=new McastService();
        mcastService.setAddress("228.0.0.4");
        mcastService.setPort(45564);
        mcastService.setFrequency(500);
        mcastService.setDropTime(3000);
        groupChannel.setMembershipService(mcastService);
        NioReceiver nioReceiver=new NioReceiver();
        nioReceiver.setAddress("auto");
        nioReceiver.setPort(4000);
        nioReceiver.setAutoBind(100);
        nioReceiver.setSelectorTimeout(5000);
        nioReceiver.setMaxThreads(10);
        groupChannel.setChannelReceiver(nioReceiver);
        ReplicationTransmitter channelSender=new ReplicationTransmitter();
        PooledParallelSender pooledParallelSender=new PooledParallelSender();
        channelSender.setTransport(pooledParallelSender);
        groupChannel.addInterceptor(new TcpFailureDetector());
        groupChannel.addInterceptor(new MessageDispatchInterceptor());
        groupChannel.setChannelSender(channelSender);
        tcpCluster.setChannel(groupChannel);
        parent.setCluster(tcpCluster);
    }
}

这个 ReplicationValve是同步集群session的写死了，这个我们只能自己去实现了。

1. 首先我定义了一个Spring事件,当你想给集群节点发送信息时，通过发这个事件就可以了。

package com.example.springboot;


import org.springframework.context.ApplicationEvent;

public class KeyValueChangeEvent extends ApplicationEvent {
    private String key;
    private String value;

    public KeyValueChangeEvent(String key, String value) {
        super(value);
    }
    public String getKey() {
        return key;
    }

    public void setKey(String key) {
        this.key = key;
    }

    public String getValue() {
        return value;
    }

    public void setValue(String value) {
        this.value = value;
    }
}

2. 自己实现 Valve这个类,监听了KeyValueChangeEvent这个事件，间接的发送给其他节点

package com.example.springboot;

import org.apache.catalina.connector.Request;
import org.apache.catalina.connector.Response;
import org.apache.catalina.ha.CatalinaCluster;
import org.apache.catalina.ha.ClusterMessageBase;
import org.apache.catalina.ha.ClusterValve;
import org.apache.catalina.valves.ValveBase;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;

import javax.servlet.ServletException;
import java.io.IOException;

@Component
public class MyMapValue extends ValveBase implements ClusterValve, ApplicationListener<KeyValueChangeEvent> {
    private CatalinaCluster cluster;

    @Override
    public CatalinaCluster getCluster() {
        return cluster;
    }

    @Override
    public void setCluster(CatalinaCluster cluster) {
        this.cluster=cluster;
    }

    @Override
    public void invoke(Request request, Response response) throws ServletException, IOException {
        getNext().invoke(request, response);
    }


    @Override
    public void onApplicationEvent(KeyValueChangeEvent event) {
        String key = event.getKey();
        String value = event.getValue();
        cluster.send(new MyKeyValueClusterMessage(key, value));
    }

    public static class MyKeyValueClusterMessage extends ClusterMessageBase {
        private final String key;
        private final String value;

        public MyKeyValueClusterMessage(String key, String value) {
            this.key=key;
            this.value=value;
        }

        @Override
        public String getUniqueId() {
            return System.currentTimeMillis()+key;
        }

        public String getKey() {
            return key;
        }

        public String getValue() {
            return value;
        }
    }
}

3. 然后将这个对象设置到集群当中 MyTomcatServletWebServerFactory

 myMapValue.setCluster(tcpCluster);

4. 那我怎么去接到集群当中其他节点发送过来的数据呢？通过实现ChannelListener，并且将这个监听设置到ChannelGroup当中，这个是我的实现

package com.example.springboot;

import org.apache.catalina.tribes.ChannelListener;
import org.apache.catalina.tribes.Member;
import org.apache.catalina.tribes.tipis.AbstractReplicatedMap;

import java.io.Serializable;

public class MapClusterListener implements ChannelListener {
    @Override
    public void messageReceived(Serializable msg, Member sender) {
        if(msg instanceof MyMapValue.MyKeyValueClusterMessage){
            MyMapValue.MyKeyValueClusterMessage mapMessage= (MyMapValue.MyKeyValueClusterMessage) msg;
            Serializable key = mapMessage.getKey();
            System.out.println("收到集群消息");
            System.out.println(key);
            System.out.println(mapMessage.getValue());
        }
    }

    @Override
    public boolean accept(Serializable msg, Member sender) {
        return msg instanceof MyMapValue.MyKeyValueClusterMessage;
    }
}

将监听设置到GroupChannel当中

groupChannel.addChannelListener(new MapClusterListener());

5. 打完收工我们开始验证 这个是我的springboot应用类

package com.example.springboot;

import org.springframework.beans.BeansException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.context.ServletContextAware;

import javax.servlet.ServletContext;
import javax.servlet.http.HttpSession;
import java.util.Random;

@SpringBootApplication
@RestController
public class Application implements ApplicationContextAware {


    private ApplicationContext applicationContext;

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @GetMapping
    public String getMaping(){
        applicationContext.publishEvent(new KeyValueChangeEvent("testKey",String.valueOf(System.currentTimeMillis())));
        return "abcd";
    }

    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
        this.applicationContext=applicationContext;
    }
}

开始实验

可以看到1513这个节点收是收到通知了为啥数据是空的呢？卧槽通过debug发现 我特么的KeyValueChangeEvent这个没设置值，有成功写了一个bug,哈哈

修改代码再试试代码如下

package com.example.springboot;


import org.springframework.context.ApplicationEvent;

public class KeyValueChangeEvent extends ApplicationEvent {
    private String key;
    private String value;

    public KeyValueChangeEvent(String key, String value) {
        super(value);
        this.key=key;
        this.value=value;
    }
    public String getKey() {
        return key;
    }

    public void setKey(String key) {
        this.key = key;
    }

    public String getValue() {
        return value;
    }

    public void setValue(String value) {
        this.value = value;
    }
}

okok了 一个小程序就这样ok了

项目源码地址 study-project/ tomcat-spring-boot