Spring-Cloud-Gateway+Ribbon负载均衡-2

最新推荐文章于 2024-06-25 17:17:56 发布

SIGN-zxd

最新推荐文章于 2024-06-25 17:17:56 发布

阅读量920

点赞数

文章标签： spring ribbon gateway

本文链接：https://blog.csdn.net/htmlxxxx/article/details/115379226

版权

Spring-Cloud-Gateway源码系列学习

版本 v2.2.6.RELEASE

LoadBalancerClientFilter源码分析

public class LoadBalancerClientFilter implements GlobalFilter, Ordered {

	/**
	 * @see RouteToRequestUrlFilter#filter(org.springframework.web.server.ServerWebExchange, org.springframework.cloud.gateway.filter.GatewayFilterChain)
	 */
	@Override
	@SuppressWarnings("Duplicates")
	public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
		//拿出目标url
		URI url = exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR);
		//获取协议，如http、lb
		String schemePrefix = exchange.getAttribute(GATEWAY_SCHEME_PREFIX_ATTR);
		if (url == null
				|| (!"lb".equals(url.getScheme()) && !"lb".equals(schemePrefix))) {
			return chain.filter(exchange);
		}
		// preserve the original url
		//把原始的目标url保存到exchange上下文,如lb://user-service
		addOriginalRequestUrl(exchange, url);

		if (log.isTraceEnabled()) {
			log.trace("LoadBalancerClientFilter url before: " + url);
		}

		//通过负载均衡器拿到一个服务实例
		final ServiceInstance instance = choose(exchange);

		if (instance == null) {
			throw NotFoundException.create(properties.isUse404(),
					"Unable to find instance for " + url.getHost());
		}

		URI uri = exchange.getRequest().getURI();

		// if the `lb:<scheme>` mechanism was used, use `<scheme>` as the default,
		// if the loadbalancer doesn't provide one.
		String overrideScheme = instance.isSecure() ? "https" : "http";
		if (schemePrefix != null) {
			overrideScheme = url.getScheme();
		}

		//获取真实的目标请求地址，如http://localhost:1000/user/get?id=1
		URI requestUrl = loadBalancer.reconstructURI(
				new DelegatingServiceInstance(instance, overrideScheme), uri);

		if (log.isTraceEnabled()) {
			log.trace("LoadBalancerClientFilter url chosen: " + requestUrl);
		}

		//把目标url保存到exchange上下文
		exchange.getAttributes().put(GATEWAY_REQUEST_URL_ATTR, requestUrl);
		//下一个filter
		return chain.filter(exchange);
	}

	/**
	 * 通过负载均衡器拿到一个实例，示例数据：exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR) = lb://user-service/user/get?id=1
	 * @see GatewayLoadBalancerClientAutoConfiguration
	 * @see RibbonLoadBalancerClient
	 * @see RibbonNacosAutoConfiguration
	 */
	protected ServiceInstance choose(ServerWebExchange exchange) {
		return loadBalancer.choose(
				((URI) exchange.getAttribute(GATEWAY_REQUEST_URL_ATTR)).getHost()); //示例数据：getHost拿到的是user-service
	}

}

Spring-Cloud负载均衡设计

从上面LoadBalancerClientFilter的源码分析中我们可以看到，负载均衡核心代码是LoadBalancerClient#choose，LoadBalancerClient是Spring-Cloud定义的负载均衡客户端，继承了ServiceInstanceChooser接口

public interface LoadBalancerClient extends ServiceInstanceChooser {

	//request相当于一个Function，需要自己实现，参数是一个负载均衡得到的ServiceInstance
	<T> T execute(String serviceId, LoadBalancerRequest<T> request) throws IOException;

	//指定serviceId和serviceInstance
	<T> T execute(String serviceId, ServiceInstance serviceInstance,
			LoadBalancerRequest<T> request) throws IOException;

	//根据ServiceInstance和原始URI(如：http://user-service)得到真实的uri(http://192.168.1.91:8181)
	URI reconstructURI(ServiceInstance instance, URI original);

}
public interface ServiceInstanceChooser {

	//根据serviceId 经过一定的负载均衡算法，获得一个ServiceInstance
	ServiceInstance choose(String serviceId);

}

RibbonLoadBalancerClient#choose分析

Spring-Cloud目前负载均衡好像就只有Ribbon一个组件

public class RibbonLoadBalancerClient implements LoadBalancerClient {

	private SpringClientFactory clientFactory;

    //SpringClientFactory可以生产ILoadBalancer
	public RibbonLoadBalancerClient(SpringClientFactory clientFactory) {
		this.clientFactory = clientFactory;
	}

    //通过服务id获取一个服务实例
	@Override
	public ServiceInstance choose(String serviceId) {
		return choose(serviceId, null);
	}

	//通过服务id获取一个服务实例
	public ServiceInstance choose(String serviceId, Object hint) {
        //核心代码，getLoadBalancer创建一个ILoadBalancer
        //委托ILoadBalancer去负载均衡获取一个Ribbon的Server
		Server server = getServer(getLoadBalancer(serviceId), hint);
		if (server == null) {
			return null;
		}
        //组装返回一个RibbonServer，RibbonServer属于ServiceInstance子类
		return new RibbonServer(serviceId, server, isSecure(server, serviceId),
				serverIntrospector(serviceId).getMetadata(server));
	}

	// Note: This method could be removed?
	protected Server getServer(String serviceId) {
		return getServer(getLoadBalancer(serviceId), null);
	}

	protected Server getServer(ILoadBalancer loadBalancer) {
		return getServer(loadBalancer, null);
	}

	protected Server getServer(ILoadBalancer loadBalancer, Object hint) {
		if (loadBalancer == null) {
			return null;
		}
		//核心代码，委托ILoadBalancer去负载均衡获取一个Ribbon的Server
		return loadBalancer.chooseServer(hint != null ? hint : "default");
	}

    //通过clientFactory生产一个ILoadBalancer
	protected ILoadBalancer getLoadBalancer(String serviceId) {
		return this.clientFactory.getLoadBalancer(serviceId);
	}

}

ILoadBalancer及其子类体系

在这里插入图片描述

ILoadBalancer：接口，定义了客户端负载均衡需要使用到的一系列抽象操作
AbstractLoadBalancer：主要是增加了一个服务实例分类枚举ServerGroup，包含三种类型：ALL(所有服务实例)，STATUS_UP(健康的服务实例)，STATUS_NOT_UP(停止服务的服务实例)
BaseLoadBalancer：基础的ILoadBalancer实现类，后面子类一些情况也会调用它的实现，里面主要提供了以下功能：
- 定义并维护了两个服务实例列表，一个是全部服务实例列表，一个是健康的服务实例列表
```
@Monitor(name = PREFIX + "AllServerList", type = DataSourceType.INFORMATIONAL)
protected volatile List<Server> allServerList = Collections
	.synchronizedList(new ArrayList<Server>());
@Monitor(name = PREFIX + "UpServerList", type = DataSourceType.INFORMATIONAL)
protected volatile List<Server> upServerList = Collections
	.synchronizedList(new ArrayList<Server>());
```
- 实现了ILoadBalancer接口定义的负载均衡器中应具备的一系列基础操作方法
- 定义了检查服务实例是否正常服务的IPing对象和SerialPingStrategy(实现IPingStrategy)，并在构造函数里面开启setupPingTask()定期检测各个服务实例的状态
- 定义了负载均衡的处理规则IRule对象，chooseServer方法核心也是使用IRule对象去获取一个经过IRule负载均衡算法的Server，Ribbon提供的IRule实现类有：
  - BestAvailableRule：选择最小请求数的Server
  - RandomRule：随机选择Server
  - RoundRobinRule：轮询选择Server
  - RetryRule：根据轮询的方式重试
  - WeightedResponseTimeRule：根据响应时间去分配Weight，Weight越高，被选择的可能性就越大
  - ZoneAvoidanceRule：根据Server的zone区域和可用性来轮询选择，如果只有一个zone区域，行为跟RoundRobinRule一致
  - AvailabilityFilteringRule：过滤掉那些因为一直连接失败(标记了circuit tripped)和高并发(activeConnections 超过配置的阈值)的Server，尽量保证可用性
DynamicServerListLoadBalancer：继承于BaseLoadBalancer，它对基础负载均衡器做了扩展。在该负载均衡器，实现了服务实例清单在运行期间的动态更新；同时还具备了对服务实例清单的过滤功能，也就是说可以通过过滤器来选择获取一批服务实例清单。
ZoneAwareLoadBalancer：对DynamicServerListLoadBalancer的扩展。在DynamicServerListLoadBalancer中没有区分zone区域，而ZoneAwareLoadBalancer主要扩展的功能就是增加了zone区域过滤。

ILoadBalancer源码分析

public interface ILoadBalancer {

	//向负载均衡器中维护的实例列表增加服务实例
	public void addServers(List<Server> newServers);
	
	//通过某种策略，从负载均衡器中挑选出一个具体的服务实例
	public Server chooseServer(Object key);
	
	//用来通知和标识负载均衡器中某个具体实例已经停止服务，不然负载均衡器在下一次获取服务实例清单前都会认为服务实例均是正常服务的
	public void markServerDown(Server server);
	
	//根据参数判断返回可正常服务的实例列表还是全部服务实例列表
	@Deprecated
	public List<Server> getServerList(boolean availableOnly);

	//返回当前可正常服务的实例列表
    public List<Server> getReachableServers();

    //返回所有已知的服务实例列表，包括正常服务和停止服务的实例
	public List<Server> getAllServers();
}

从RibbonClientConfiguration类研究Ribbon负载均衡器选择逻辑

默认的IRule是ZoneAvoidanceRule，默认的ILoadBalancer是ZoneAwareLoadBalancer

public class RibbonClientConfiguration {
	
	@Autowired
	private PropertiesFactory propertiesFactory;
	
    //IRule，默认创建ZoneAvoidanceRule
	@Bean
	@ConditionalOnMissingBean
	public IRule ribbonRule(IClientConfig config) {
        //如果设置了ILoadBalancer类，即name.ribbon.NFLoadBalancerRuleClassName则创建用户配置的负载均衡规则实现类
		if (this.propertiesFactory.isSet(IRule.class, name)) {
			return this.propertiesFactory.get(IRule.class, config, name);
		}
        //否则默认创建ZoneAvoidanceRule
		ZoneAvoidanceRule rule = new ZoneAvoidanceRule();
		rule.initWithNiwsConfig(config);
		return rule;
	}
	
    //核心代码，默认选择ZoneAwareLoadBalancer
	@Bean
	@ConditionalOnMissingBean
	public ILoadBalancer ribbonLoadBalancer(IClientConfig config,
			ServerList<Server> serverList, ServerListFilter<Server> serverListFilter,
			IRule rule, IPing ping, ServerListUpdater serverListUpdater) {
        //如果设置了ILoadBalancer类，即name.ribbon.NFLoadBalancerClassName则创建用户配置的负载均衡实现类
		if (this.propertiesFactory.isSet(ILoadBalancer.class, name)) {
			return this.propertiesFactory.get(ILoadBalancer.class, config, name);
		}
        //否则默认创建ZoneAwareLoadBalancer
		return new ZoneAwareLoadBalancer<>(config, rule, ping, serverList,
				serverListFilter, serverListUpdater);
	}
}

负载均衡配置示例

示例代码仓库：https://gitee.com/wyusig/spring-webflux-demo.git

负载均衡例子的代码模块是：

sc-gateway-registry
sc-user-service

IDEA 多实例运行sc-user-service，正常运行sc-gateway-registry

测试链接：http://localhost:8888/user-service/user/robbinTest

Ribbon yml配置文件配置

#user-service为服务名
user-service:
  ribbon:
    NFLoadBalancerRuleClassName: com.netflix.loadbalancer.RandomRule

Ribbon java config 配置

首先新建个包，需要和Spring-Boot Application启动类不在同一目录下，可以参考sc-gateway-registry

编写默认负载均衡规则Configuration

@Configuration
public class DefaultRibbonClientConfiguration {

    @Bean
    public IRule ribbonDefaultRule() {
        return new RoundRobinRule();
    }
}

编写单独服务的负载均衡规则Configuration

@Configuration
public class UserServiceRibbonClientConfiguration {

    @Bean
    @Primary
    public IRule ribbonCustomRule() {
        return new RandomRule();
    }
}

编写Ribbon配置类

@Configuration
@RibbonClients(
        value = {
                @RibbonClient(name = "user-service", configuration = UserServiceRibbonClientConfiguration.class)
        },
        defaultConfiguration = DefaultRibbonClientConfiguration.class
)
public class RibbonConfiguration {
}

BaseLoadBalancer#chooseServer源码分析

核心就是调用IRule#choose

public class BaseLoadBalancer extends AbstractLoadBalancer implements
        PrimeConnections.PrimeConnectionListener, IClientConfigAware {
	public Server chooseServer(Object key) {
        if (counter == null) {
            //计数器为空就新建一个
            counter = createCounter();
        }
        //+1
        counter.increment();
        if (rule == null) {
            //如果IRule对象为空，即负载均衡策略对象找不到，直接返回null
            return null;
        } else {
            try {
                //调用IRule#choose，让负载均衡策略对象返回一个服务实例
                return rule.choose(key);
            } catch (Exception e) {
                //示例数据：name=user-service， key=deault
                logger.warn("LoadBalancer [{}]:  Error choosing server for key {}", name, key, e);
                return null;
            }
        }
    }
}

ZoneAwareLoadBalancer简单源码分析

public class ZoneAwareLoadBalancer<T extends Server> extends DynamicServerListLoadBalancer<T>{
	@Override
    public Server chooseServer(Object key) {
        //如果未开启ZoneAware(默认开启)或者只有一个区域
        if (!ENABLED.get() || getLoadBalancerStats().getAvailableZones().size() <= 1) {
            logger.debug("Zone aware logic disabled or there is only one zone");
            //调用BaseLoadBalancer#chooseServer
            return super.chooseServer(key);
        }
        Server server = null;
        try {
            LoadBalancerStats lbStats = getLoadBalancerStats();
            Map<String, ZoneSnapshot> zoneSnapshot = ZoneAvoidanceRule.createSnapshot(lbStats);
            logger.debug("Zone snapshots: {}", zoneSnapshot);
            if (triggeringLoad == null) {
                triggeringLoad = DynamicPropertyFactory.getInstance().getDoubleProperty(
                        "ZoneAwareNIWSDiscoveryLoadBalancer." + this.getName() + ".triggeringLoadPerServerThreshold", 0.2d);
            }

            if (triggeringBlackoutPercentage == null) {
                triggeringBlackoutPercentage = DynamicPropertyFactory.getInstance().getDoubleProperty(
                        "ZoneAwareNIWSDiscoveryLoadBalancer." + this.getName() + ".avoidZoneWithBlackoutPercetage", 0.99999d);
            }
            //获取可用的区域的服务实例列表
            Set<String> availableZones = ZoneAvoidanceRule.getAvailableZones(zoneSnapshot, triggeringLoad.get(), triggeringBlackoutPercentage.get());
            logger.debug("Available zones: {}", availableZones);
            if (availableZones != null &&  availableZones.size() < zoneSnapshot.keySet().size()) {
                //随机获取一个区域
                String zone = ZoneAvoidanceRule.randomChooseZone(zoneSnapshot, availableZones);
                logger.debug("Zone chosen: {}", zone);
                if (zone != null) {
                    //构造一个BaseLoadBalancer，里面的IRule使用用户配置或默认的ZoneAvoidanceRule
                    BaseLoadBalancer zoneLoadBalancer = getLoadBalancer(zone);
                    //BaseLoadBalancer负载均衡获得一个服务实例
                    server = zoneLoadBalancer.chooseServer(key);
                }
            }
        } catch (Exception e) {
            logger.error("Error choosing server using zone aware logic for load balancer={}", name, e);
        }
        if (server != null) {
            //返回
            return server;
        } else {
            logger.debug("Zone avoidance logic is not invoked.");
            //调用BaseLoadBalancer#chooseServer
            return super.chooseServer(key);
        }
    }
    
    //构造一个BaseLoadBalancer
    @VisibleForTesting
    BaseLoadBalancer getLoadBalancer(String zone) {
        zone = zone.toLowerCase();
        BaseLoadBalancer loadBalancer = balancers.get(zone);
        if (loadBalancer == null) {
        	// We need to create rule object for load balancer for each zone
            //克隆一份IRule(负载均衡规则)，里面的IRule使用用户配置或默认的ZoneAvoidanceRule
        	IRule rule = cloneRule(this.getRule());
            loadBalancer = new BaseLoadBalancer(this.getName() + "_" + zone, rule, this.getLoadBalancerStats());
            BaseLoadBalancer prev = balancers.putIfAbsent(zone, loadBalancer);
            if (prev != null) {
            	loadBalancer = prev;
            }
        } 
        return loadBalancer;        
    }
}

Nacos是如何将服务列表配置给Ribbon的

答案就是跟前面的Ribbon java config配置一样，使用@RibbonClients，具体代码细节在 RibbonNacosAutoConfiguration、NacosRibbonClientConfiguration、RibbonClientConfiguration 三个配置类上

RibbonNacosAutoConfiguration

//核心代码：@RibbonClients(defaultConfiguration = NacosRibbonClientConfiguration.class)
@Configuration(proxyBeanMethods = false)
@EnableConfigurationProperties
@ConditionalOnBean(SpringClientFactory.class)
@ConditionalOnRibbonNacos
@ConditionalOnNacosDiscoveryEnabled
@AutoConfigureAfter(RibbonAutoConfiguration.class)
@RibbonClients(defaultConfiguration = NacosRibbonClientConfiguration.class)
public class RibbonNacosAutoConfiguration {

}

NacosRibbonClientConfiguration

@Configuration(proxyBeanMethods = false)
@ConditionalOnRibbonNacos
public class NacosRibbonClientConfiguration {

	@Autowired
	private PropertiesFactory propertiesFactory;

    //由Nacos创建ServerList的Bean
	@Bean
	@ConditionalOnMissingBean
	public ServerList<?> ribbonServerList(IClientConfig config,
			NacosDiscoveryProperties nacosDiscoveryProperties) {
		if (this.propertiesFactory.isSet(ServerList.class, config.getClientName())) {
			ServerList serverList = this.propertiesFactory.get(ServerList.class, config,
					config.getClientName());
			return serverList;
		}
		NacosServerList serverList = new NacosServerList(nacosDiscoveryProperties);
		serverList.initWithNiwsConfig(config);
		return serverList;
	}

}

RibbonClientConfiguration

public class RibbonClientConfiguration
	//构造器依赖注入Nacos创建的ServerList
	@Bean
	@ConditionalOnMissingBean
	public ILoadBalancer ribbonLoadBalancer(IClientConfig config,
			ServerList<Server> serverList, ServerListFilter<Server> serverListFilter,
			IRule rule, IPing ping, ServerListUpdater serverListUpdater) {
		if (this.propertiesFactory.isSet(ILoadBalancer.class, name)) {
			return this.propertiesFactory.get(ILoadBalancer.class, config, name);
		}
		return new ZoneAwareLoadBalancer<>(config, rule, ping, serverList,
				serverListFilter, serverListUpdater);
	}
}

Ribbon是如何更新服务实例的

原因就是DynamicServerListLoadBalancer，从上面可以看到，DynamicServerListLoadBalancer注入了Nacos的ServerList，我们来看一下ServerList的源码(BaseLoadBalancer里面也有IPing对已有的服务进行健康监测)

public interface ServerList<T extends Server> {

    //获取初始化的服务实例列表
    public List<T> getInitialListOfServers();
    
    /**
     * Return updated list of servers. This is called say every 30 secs
     * (configurable) by the Loadbalancer's Ping cycle
     * 
     */
    //获取新的服务实例列表，默认每30秒调用一次
    public List<T> getUpdatedListOfServers();   

}

我们再来看一下DynamicServerListLoadBalancer里面的代码

public class DynamicServerListLoadBalancer<T extends Server> extends BaseLoadBalancer {

	volatile ServerList<T> serverListImpl;
	
    //DynamicServerListLoadBalancer实现的UpdateAction，update里面调用updateListOfServers
	protected final ServerListUpdater.UpdateAction updateAction = new ServerListUpdater.UpdateAction() {
        @Override
        public void doUpdate() {
            updateListOfServers();
        }
    };
    
    public void enableAndInitLearnNewServersFeature() {
        LOGGER.info("Using serverListUpdater {}", serverListUpdater.getClass().getSimpleName());
        serverListUpdater.start(updateAction);
    }
    
    //updateListOfServers方法核心逻辑就是调用ServerList#getUpdatedListOfServers
    @VisibleForTesting
    public void updateListOfServers() {
        List<T> servers = new ArrayList<T>();
        if (serverListImpl != null) {
            //调用ServerList#getUpdatedListOfServers
            servers = serverListImpl.getUpdatedListOfServers();
            LOGGER.debug("List of Servers for {} obtained from Discovery client: {}",
                    getIdentifier(), servers);

            if (filter != null) {
                servers = filter.getFilteredListOfServers(servers);
                LOGGER.debug("Filtered List of Servers for {} obtained from Discovery client: {}",
                        getIdentifier(), servers);
            }
        }
        //更新所有服务实例
        updateAllServerList(servers);
    }
    
    //更新所有服务实例
    protected void updateAllServerList(List<T> ls) {
        // other threads might be doing this - in which case, we pass
        if (serverListUpdateInProgress.compareAndSet(false, true)) {
            try {
                for (T s : ls) {
                    s.setAlive(true); // set so that clients can start using these
                                      // servers right away instead
                                      // of having to wait out the ping cycle.
                }
                setServersList(ls);
                super.forceQuickPing();
            } finally {
                serverListUpdateInProgress.set(false);
            }
        }
    }
}

SIGN-zxd

关注

0
点赞
踩
3

收藏

觉得还不错? 一键收藏
0
评论
Spring-Cloud-Gateway+Ribbon负载均衡-2

Spring-Cloud-Gateway源码系列学习版本 v2.2.6.RELEASELoadBalancerClientFilter源码分析public class LoadBalancerClientFilter implements GlobalFilter, Ordered { /** * @see RouteToRequestUrlFilter#filter(org.springframework.web.server.ServerWebExchange, org.springfra
复制链接

扫一扫