限流组件Sentinel生产应用实践(一) : 动态规则源持久化(Zookeeper)改造接入

介绍

Sentinel是阿里巴巴去年开源的一款轻量级限流组件。引用它官网的一段简要介绍：

As distributed systems become increasingly popular, the reliability between services is becoming more important than ever before. Sentinel takes “flow” as breakthrough point, and works on multiple fields including flow control, circuit breaking and system adaptive protection, to guarantee reliability of microservices.

Sentinel has the following features:

• Rich applicable scenarios: Sentinel has been wildly used in Alibaba, and has covered almost all the core-scenarios in Double-11 (11.11) Shopping Festivals in the past 10 years, such as “Second Kill” which needs to limit burst flow traffic to meet the system capacity, message peak clipping and valley fills, circuit breaking for unreliable downstream services, cluster flow control, etc.
• Real-time monitoring: Sentinel also provides real-time monitoring ability. You can see the runtime information of a single machine in real-time, and the aggregated runtime info of a cluster with less than 500 nodes.
  Widespread open-source ecosystem: Sentinel provides out-of-box integrations with commonly-used frameworks and libraries such as Spring Cloud, Dubbo and gRPC. You can easily use Sentinel by simply add the adapter dependency to your services.
• Various SPI extensions: Sentinel provides easy-to-use SPI extension interfaces that allow you to quickly customize your logic, for example, custom rule management, adapting data sources, and so on.

如何使用Sentinel这里就不介绍了, 可以参考官方文档快速入门。这里主要介绍的是动态规则源zookeeper持久化的Dashboard模块的接入改造生产实践。

Dashboad介绍

从github仓库下载最新的代码到本地，启动dashoard模块应用入口类DashboardApplication, 该引用是基于springboot 的web应用, 并加入启动参数-Dserver.port=8080 -Dcsp.sentinel.dashboard.server=localhost:8080 -Dproject.name=sentinel-dashboard 参数具体意义可参考文档。访问http://localhost:8080即可 用户密码均为sentinel, 如下所示

可以看到这里可以配置多种规则，比如流控规则

但是这里的规则设置，都是存储在对应的客户端内存中的，这样有两个缺点: 1.规则会随着客户端的重启而丢失 2.若同一个app有多个实例(多个实例微服务很常见)，就需要每个客户端实例都得手动设置规则，非常痛苦，因此不能直接用于生产环境，需要做相应的改造。目前的sentinel已经支持多种持久化组件了，比如zookeeper,nacos,apollo,redis,rdbms等,而且留有相应的扩展，只需对dashboard做些改造即可。

Dashboard改造

其实sentinel dashboard源码中已经有了一个针对流控规则改造的简单例子，就是com.alibaba.csp.sentinel.dashboard.controller.v2.FlowControllerV2这个controller

com.alibaba.csp.sentinel.dashboard.rule.DynamicRuleProvider 动态规则源提获取接口以及com.alibaba.csp.sentinel.dashboard.rule.DynamicRulePublisher动态规则源提变更接口，实现这两个接口即可。

Zookeeper接入改造

这里先做个zookeeper的接入，其它数据源举一反三即可。

1. 实现DynamicRuleProvider和DynamicRulePublisher这两个接口

• 先在dashboard模块中引入zookeeper数据源扩展依赖

<dependency>
   <groupId>com.alibaba.csp</groupId>
   <artifactId>sentinel-datasource-zookeeper</artifactId>
   <exclusions>
       <exclusion>
           <groupId>org.slf4j</groupId>
           <artifactId>slf4j-api</artifactId>
       </exclusion>
       <exclusion>
           <groupId>org.slf4j</groupId>
           <artifactId>slf4j-log4j12</artifactId>
       </exclusion>
       <exclusion>
           <groupId>log4j</groupId>
           <artifactId>log4j</artifactId>
       </exclusion>
   </exclusions>
</dependency>

• 实现DynamicRuleProvider

package com.alibaba.csp.sentinel.dashboard.rule.zookeeper.flow;

import com.alibaba.csp.sentinel.dashboard.datasource.entity.rule.FlowRuleEntity;
import com.alibaba.csp.sentinel.dashboard.rule.DynamicRuleProvider;
import com.alibaba.csp.sentinel.dashboard.rule.zookeeper.ZookeeperConfigUtils;
import com.alibaba.csp.sentinel.datasource.Converter;
import org.apache.curator.framework.CuratorFramework;
import org.springframework.beans.factory.annotation.Autowired;

import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
import java.util.List;


/**
 * @author rodbate
 */
public class FlowRuleZookeeperProvider implements DynamicRuleProvider<List<FlowRuleEntity>> {
   

    @Autowired
    private CuratorFramework zkClient;

    @Autowired
    private Converter<String, List<FlowRuleEntity>> converter;

    @Override
    public List<FlowRuleEntity> getRules(String appName) throws Exception {
   
        byte[] data = zkClient.getData().forPath(ZookeeperConfigUtils.getFlowRuleZkPath(appName));
        if (data == null || data.length == 0) {
   
            return new ArrayList<>();
        }
        return converter.convert(new String(data, StandardCharsets.UTF_8));
    }
}

• 实现DynamicRulePublisher

package com.alibaba.csp.sentinel.dashboard.rule.zookeeper.flow;

import com.alibaba.csp.sentinel.dashboard.datasource.entity.rule.FlowRuleEntity;
import com.alibaba.csp.sentinel.dashboard.rule.DynamicRulePublisher;
import com.alibaba.csp.sentinel.dashboard.rule.zookeeper.ZookeeperConfigUtils;
import com.alibaba.csp.sentinel.datasource.Converter;
import org.apache.curator.framework.CuratorFramework;
import org.apache.zookeeper.CreateMode;
import org.apache.zookeeper.data.Stat;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.util.CollectionUtils;

import java.nio.charset.StandardCharsets;
import java.util.List;


/**
 * @author rodbate
 */
public class FlowRuleZookeeperPublisher implements DynamicRulePublisher<List<FlowRuleEntity>> {
   

    @Autowired
    private CuratorFramework zkClient;

    @Autowired
    private Converter<List<FlowRuleEntity>, String> converter;

    @Override
    public void publish(String app, List<FlowRuleEntity> rules) throws Exception {
   
        String zkPath = ZookeeperConfigUtils.getFlowRuleZkPath(app);
        Stat stat = zkClient.checkExists().forPath(zkPath);
        if (stat == null) {
   
            zkClient.create().creatingParentsIfNeeded().withMode(CreateMode.PERSISTENT).forPath(zkPath, null);
        }
        byte[] data = null;
        if (!CollectionUtils.isEmpty(rules)) {
   
            data = converter.convert(rules).getBytes(StandardCharsets.UTF_8);
        }
        zkClient.setData().forPath(zkPath, data);
    }

}

• 相应的bean初始化

package com.alibaba.csp.sentinel.dashboard.rule.zookeeper;


import com.alibaba.csp.sentinel.dashboard.datasource.entity.rule.FlowRuleEntity;
import com.alibaba.csp.sentinel.dashboard.rule.zookeeper.flow.FlowRuleZookeeperProvider;
import com.alibaba.csp.sentinel.dashboard.rule.zookeeper.flow.FlowRuleZookeeperPublisher;
import com.alibaba.csp.sentinel.datasource.Converter;
import com.alibaba.fastjson.JSON;
import lombok.extern.slf4j.Slf4j;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org