Zuul网关集成Sentinel持久化规则数据到nacos中

Zuul网关集成Sentinel持久化规则数据到nacos中

本项目环境：

• Zuul 1.3.1
• SentinelDashBoard 1.8.3
• nacos 2.0.4

本篇文章目前已实现功能

• 网关限流和网关API分组规则数据持久化到nacos
• 实现了nacos和SentinelDashboard的数据双向互通

目前遗留问题：

• SentinelDashboard网关限流规则参数实体类与客户端的网关限流规则参数实例类字段不一致

1.网关POM文件加入依赖

Sentinel 提供了 Zuul 1.x 的适配模块，为 Zuul Gateway 只提供了两种资源维度的限流。特别注意的是默认不支持 URL 粒度，本篇文主要讲的是基于api分组实现网关流控效果：

• route 维度：即在 Spring 配置文件中配置的路由条目，资源名为对应的 route ID（对应 RequestContext 中的 proxy 字段）
• 自定义 API 维度：用户可以利用 Sentinel 提供的 API 来自定义一些 API 分组

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-zuul</artifactId>
        </dependency>
		<!-- sentinel核心依赖 -->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-sentinel</artifactId>
            <version>2.2.7.RELEASE</version>
        </dependency>
		<!-- sentinel适配nacos数据源依赖 -->
         <dependency>
            <groupId>com.alibaba.csp</groupId>
            <artifactId>sentinel-datasource-nacos</artifactId>
            <version>1.8.3</version>
        </dependency>
		<!-- sentinel适配zuul依赖 -->
        <dependency>
            <groupId>com.alibaba.csp</groupId>
            <artifactId>sentinel-zuul-adapter</artifactId>
            <version>1.8.3</version>
        </dependency>
		<!-- sentinel网关依赖 -->
        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-alibaba-sentinel-gateway</artifactId>
        </dependency>

2.创建配置类

@Configuration
public class ZuulConfig {
/**
*将三个Sentinel实现的ZuulFilter 注入到 Spring 环境中
**/
    @Bean
    public ZuulFilter sentinelZuulPreFilter() {
        // We can also provider the filter order in the constructor.
        return new SentinelZuulPreFilter();
    }

    @Bean
    public ZuulFilter sentinelZuulPostFilter() {
        return new SentinelZuulPostFilter();
    }

    @Bean
    public ZuulFilter sentinelZuulErrorFilter() {
        return new SentinelZuulErrorFilter();
    }
    //初始化自定义限流异常返回信息
    @PostConstruct
    public void doInit() {
        // 注册 FallbackProvider
        ZuulBlockFallbackManager.registerProvider(new MyBlockFallbackProvider());
    }
}

public class MyBlockFallbackProvider implements ZuulBlockFallbackProvider {
    
    @Override
    public String getRoute() {
        return "/book/app";
    }

    @Override
    public BlockResponse fallbackResponse(String route, Throwable cause) {
        RecordLog.info(String.format("[Sentinel DefaultBlockFallbackProvider] Run fallback route: %s", route));
        if (cause instanceof BlockException) {
            return new BlockResponse(429, "Sentinel block exception", route);
        } else {
            return new BlockResponse(500, "System Error", route);
        }
    }
}

3.环境配置

spring:
  cloud:
    sentinel:
      transport:
      #Sentinel 控制台地址
        dashboard: localhost:9111
        #Sentinel与客户端交互的端口
        port: 8720
      datasource:
      	# 网关流控配置文件
        ds1:
          nacos:
            # nacos地址
            server-addr: addr
            # nacos 命名空间 没有可不写
            namespace：namespace
            # nacos中配置文件的data-id
            dataId: sentinel-gateway
            # nacos 分组
            groupId: DEFAULT_GROUP
            # 规则类型 流控 rule-type类型 参考com.alibaba.cloud.sentinel.datasource.RuleType
            # 网关流控规则数据源类型是 gw-flow，若将网关流控规则数据源指定为 flow 则不生效。
            rule-type: gw-flow
            data-type: json
        # 网关API分组配置文件
        ds2:
          nacos:
            server-addr: addr
            namespace: namespace
            dataId: sentinel-gateway-api
            groupId: DEFAULT_GROUP
            rule-type: GW_API_GROUP
            data-type: json
      # 取消Sentinel控制台懒加载
      eager: true
      

4.nacos配置

创建网关流控配置文件和网关API分组配置文件文件内容设置为空对象即可

5.下载sentinel1.8.3源码

6.改造Sentinel使其支持动态数据源

1.改造后端

1.复制官方demo代码

test/com.alibaba.csp.sentinel.dashboard.rule.nacos -> com.alibaba.csp.sentinel.dashboard.rule.nacos

2.修改application配置文件并修改POM文件

在application.properties中添加上我们的nacos信息

nacos.address=
nacos.namespace=
nacos.username=
nacos.password=

        <!--修改pom.xml(sentinel-dashboard) 找到 如下依赖 把test注释掉-->
		<dependency>
            <groupId>com.alibaba.csp</groupId>
            <artifactId>sentinel-datasource-nacos</artifactId>
            <!--<scope>test</scope>-->
        </dependency>

3.修改NacosConfig

改之前

/**
 * @author Eric Zhao
 * @since 1.4.0
 */
@Configuration
public class NacosConfig {

    @Bean
    public Converter<List<FlowRuleEntity>, String> flowRuleEntityEncoder() {
        return JSON::toJSONString;
    }

    @Bean
    public Converter<String, List<FlowRuleEntity>> flowRuleEntityDecoder() {
        return s -> JSON.parseArray(s, FlowRuleEntity.class);
    }

    @Bean
    public ConfigService nacosConfigService() throws Exception {
        return ConfigFactory.createConfigService("localhost");
    }
}

改之后

/*
 * Copyright 1999-2018 Alibaba Group Holding Ltd.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *      http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package com.alibaba.csp.sentinel.dashboard.rule.nacos;

import com.alibaba.csp.sentinel.dashboard.datasource.entity.gateway.GatewayFlowRuleEntity;
import com.alibaba.csp.sentinel.dashboard.datasource.entity.rule.*;
import com.alibaba.csp.sentinel.dashboard.domain.ParamFlowRuleCorrectEntity;
import com.alibaba.csp.sent inel.datasource.Converter;
import com.alibaba.fastjson.JSON;
import com.alibaba.nacos.api.PropertyKeyConst;
import com.alibaba.nacos.api.config.ConfigFactory;
import com.alibaba.nacos.api.config.ConfigService;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import java.util.List;
import java.util.Properties;

/**
 * @author Eric Zhao
 * @since 1.4.0
 */
@Configuration
public class NacosConfig {

    @Value("${nacos.address}")
    private String address;

    @Value("${nacos.namespace}")
    private String namespace;

    @Value("${nacos.username}")
    private String username;

    @Value("${nacos.password}")
    private String password;
    /**
     * Converter类型转换器 用于讲控制台配置的规则数据和数据源中存储的JSON规则字符串相互转换
     * @return
     */
     
    @Bean
    public Converter<List<FlowRuleEntity>, String> flowRuleEntityEncoder() {
        return JSON::toJSONString;
    }

    @Bean
    public Converter<List<AuthorityRuleEntity>, String> authorityRuleEntityEncoder() {
        return JSON::toJSONString;
    }

    @Bean
    public Converter<List<DegradeRuleEntity>, String> degradeRuleEntityEncoder() {
        return JSON::toJSONString;
    }

    @Bean
    public Converter<List<ParamFlowRuleEntity>, String> paramFlowRuleEntityEncoder() {
        return JSON::toJSONString;
    }

    @Bean
    public Converter<List<SystemRuleEntity>, String> systemRuleEntityEncoder() {
        return JSON::toJSONString;
    }
    @Bean
    public Converter<String, List<FlowRuleEntity>> flowRuleEntityDecoder() {
        return s -> JSON.parseArray(s, FlowRuleEntity.class);
    }

    @Bean
    public Converter<List<GatewayFlowRuleEntity>, String> gatewayFlowRuleEntityEncoder() {
        return JSON::toJSONString;
    }

    @Bean
    public Converter<String, List<GatewayFlowRuleEntity>> gatewayFlowRuleEntityDecoder() {
        return s -> JSON.parseArray(s, GatewayFlowRuleEntity.class);
    }


    /**
     * 创建nacos连接配置 由于NacosConfig中填写的Nacos服务地址是localhost，我们将其改成读取配置形式
     * @return
     * @throws Exception
     */
    @Bean
    public ConfigService nacosConfigService() throws Exception {
        Properties properties = new Properties();
        properties.put(PropertyKeyConst.SERVER_ADDR, address);
        properties.put(PropertyKeyConst.NAMESPACE, namespace);
        properties.put(PropertyKeyConst.USERNAME, username);
        properties.put(PropertyKeyConst.PASSWORD, password);
        return ConfigFactory.createConfigService(properties);
    }
}

4.创建GatewayFlowRuleNacosProvider

复制FlowRuleNacosProvider -> GatewayFlowRuleNacosProvider

原代码

/**
 * @author Eric Zhao
 * @since 1.4.0
 */
@Component("flowRuleNacosProvider")
public class FlowRuleNacosProvider implements DynamicRuleProvider<List<FlowRuleEntity>> {

    @Autowired
    private ConfigService configService;
    @Autowired
    private Converter<String, List<FlowRuleEntity>> converter;

    @Override
    public List<FlowRuleEntity> getRules(String appName) throws Exception {
        String rules = configService.getConfig(appName + NacosConfigUtil.FLOW_DATA_ID_POSTFIX,
            NacosConfigUtil.GROUP_ID, 3000);
        if (StringUtil.isEmpty(rules)) {
            return new ArrayList<>();
        }
        return converter.convert(rules);
    }
}

改造后的代码

@Component("gatewayFlowRuleNacosProvider")
public class GatewayFlowRuleNacosProvider implements DynamicRuleProvider<List<GatewayFlowRuleEntity>> {

    @Autowired
    private ConfigService configService;
    // 规则对象的转换器，获取到的数据根据使用的数据类型的不同，需要用不同的转换器转化后使用
    @Autowired
    private Converter<String, List<GatewayFlowRuleEntity>> converter;

	
    /*
     * 连接数据源获取规则，第一个参数是DATA_ID；第二个参数是nacos分组id；
     */
    @Override
    public List<GatewayFlowRuleEntity> getRules(String appName) throws Exception {
        String rules = configService.getConfig(DATA_ID,GROUP_ID, 3000);
        if (StringUtil.isEmpty(rules)) {
            return new ArrayList<>(0);
        }
        return converter.convert(rules);
    }
}

5.创建GatewayFlowRuleNacosPublisher

复制FlowRuleNacosPublisher-> GatewayFlowRuleNacosPublisher

原代码

/**
 * @author Eric Zhao
 * @since 1.4.0
 */
@Component("flowRuleNacosPublisher")
public class FlowRuleNacosPublisher implements DynamicRulePublisher<List<FlowRuleEntity>> {
    
    @Autowired
    private ConfigService configService;
    @Autowired
    private Converter<List<FlowRuleEntity>, String> converter;

    @Override
    public void publish(String app, List<FlowRuleEntity> rules) throws Exception {
        AssertUtil.notEmpty(app, "app name cannot be empty");
        if (rules == null) {
            return;
        }
        configService.publishConfig(app + NacosConfigUtil.FLOW_DATA_ID_POSTFIX,
            NacosConfigUtil.GROUP_ID, converter.convert(rules));
    }
}

改造后代码

/**
 * @author Eric Zhao
 * @since 1.4.0
 */
@Component("gatewayFlowRuleNacosPublisher")
public class GatewayFlowRuleNacosPublisher implements DynamicRulePublisher<List<GatewayFlowRuleEntity>> {
    // 数据源的配置服务
    @Autowired
    private ConfigService configService;
    /**
     * <p>数据通信的转换器。
     * <p>在config包下的NacosConfig类中声明的Spring Bean对象。
     * <p>负责将实体对象转换为json格式的字符串</p>
     */
    @Autowired
    private Converter<List<GatewayFlowRuleEntity>, String> converter;

    @Override
    public void publish(String app, List<GatewayFlowRuleEntity> rules) throws Exception {
        AssertUtil.notEmpty(app, "app name cannot be empty");
        if (rules == null) {
            return;
        }
        /*
         * 将规则发布到动态数据源作持久化，第一个参数是DATA_ID后缀；
         * 第二个参数是nacos分组id，；最后一个参数是数据转换
         * 器，要将对象转换成统一的格式后，网络传输到nacos。
         */
        configService.publishConfig(DATA_ID,GROUP_ID, converter.convert(rules));
    }

}

6.创建GatewayApiNacosProvider

@Component
public class GatewayApiNacosProvider {
    @Autowired
    private ConfigService configService;
    
    /**
     * 从nacos拉取API分组配置
     * @param appName
     * @return
     * @throws Exception
     */
    public List<ApiDefinitionEntity> fetchApis(String appName) throws Exception {
        // 
        String rules = configService.getConfig(DATA_ID,DEFAULT_GROUP, 3000);
        if (StringUtil.isEmpty(rules)) {
            return new ArrayList<>();
        }

        List<ApiDefinitionEntity> list = new ArrayList<>();

        List result = JSON.parseObject(rules, List.class);

        if (result != null && !result.isEmpty()){
            for (Object o : result) {
                ApiDefinitionEntity apiDefinitionEntity = JSON.toJavaObject((JSON) o, ApiDefinitionEntity.class);
                list.add(apiDefinitionEntity);
            }
        }

        return list;
    }

}

7.创建GatewayApiNacosPublisher

@Component
public class GatewayApiNacosPublisher {
    @Autowired
    private ConfigService configService;
    public static final String DEFAULT_GROUP = "DEFAULT_GROUP";

    public Boolean modifyApis(String app, List<ApiDefinitionEntity> rules) {
        AssertUtil.notEmpty(app, "app name cannot be empty");
        if (rules == null) {
            return false;
        }
        try {
            // 随便取得名字，用来保存对应配置，记得先在nacos里创建好，初始值为[]
            configService.publishConfig("sentinel-gateway-api",
                    DEFAULT_GROUP, JSON.toJSONString(rules));
            return true;
        } catch (NacosException e) {
            e.printStackTrace();
            return false;
        }
    }
}

8.创建GatewayFlowRuleControllerV2

复制com.alibaba.csp.sentinel.dashboard.controller.gateway.GatewayFlowRuleController -> com.alibaba.csp.sentinel.dashboard.controller.v2.GatewayFlowRuleControllerV2

//接口地址添加v2
@RequestMapping(value = "/v2/gateway/flow")

//注入以下依赖   
	@Autowired
    @Qualifier("gatewayFlowRuleNacosProvider")
    private DynamicRuleProvider<List<GatewayFlowRuleEntity>> ruleProvider;
    @Autowired
    @Qualifier("gatewayFlowRuleNacosPublisher")
    private DynamicRulePublisher<List<GatewayFlowRuleEntity>> rulePublisher;
    //删除以下依赖
    @Autowired
    private SentinelApiClient sentinelApiClient;

GatewayFlowRuleControllerV2接口更改

9.创建GatewayApiControllerV2

复制com.alibaba.csp.sentinel.dashboard.controller.gateway.GatewayApiController ->

com.alibaba.csp.sentinel.dashboard.controller.v2.GatewayApiControllerV2

//接口添加v2
@RequestMapping(value = "/v2/gateway/api") 
//注入以下依赖
@Autowired
private GatewayApiNacosProvider gatewayApiNacosProvider;
@Autowired
private GatewayApiNacosPublisher gatewayApiNacosPublisher;
//删除原有依赖
@Autowired
private SentinelApiClient sentinelApiClient;

接口改造

后端最终效果

2.改造前端

1.修改app.js

找到resources/app/scripts/app.js 添加网关限流和网关API分组state

 // 新增网关限流state
      .state('dashboard.gatewayFlowV2', {
                templateUrl: 'app/views/gateway/flow_v2.html',
                url: '/gateway/flow/:app',
                controller: 'GatewayFlowCtlV2',
                resolve: {
                    loadMyFiles: ['$ocLazyLoad', function ($ocLazyLoad) {
                        return $ocLazyLoad.load({
                            name: 'sentinelDashboardApp',
                            files: [
                                'app/scripts/controllers/gateway/flow_v2.js',
                            ]
                        });
                    }]
                }
            })

    // 新增网关限流state
      .state('dashboard.gatewayApiV2', {
                templateUrl: 'app/views/gateway/api_v2.html',
                url: '/gateway/api/:app',
                controller: 'GatewayApiCtlV2',
                resolve: {
                    loadMyFiles: ['$ocLazyLoad', function ($ocLazyLoad) {
                        return $ocLazyLoad.load({
                            name: 'sentinelDashboardApp',
                            files: [
                                'app/scripts/controllers/gateway/api_v2.js',
                            ]
                        });
                    }]
                }
            })

2.修改sidebar.html

找到resources/app/scripts/directives/sidebar/sidebar.html页面中的流控规则和API管理，复制相应菜单并更改相应的state路由

3.创建flow_v2.js

复制resources/app/scripts/controllers/gateway/flow.js -> resources/app/scripts/controllers/gateway/flow_v2.js，将里面的GatewayFlowCtl，GatewayFlowService，GatewayApiService 改成 GatewayFlowCtlV2，GatewayFlowServiceV2，GatewayApiServiceV2 （建议全局搜索替换）

4.创建api_v2.js

复制resources/app/scripts/controllers/gateway/api.js -> resources/app/scripts/controllers/gateway/api_v2.js，将里面的GatewayApiCtl，GatewayApiService 改成 GatewayApiCtlV2，GatewayApiServiceV2

5.创建api_service_v2.js

复制resources/app/scripts/services/gateway/api_service.js -> resources/app/scripts/services/gateway/api_service_v2.js，将GatewayApiService 改成 GatewayApiServiceV2 ，并且所有的请求地址前加上V2

6.创建flow_service_v2.js

复制resources/app/scripts/services/gateway/flow_service.js -> resources/app/scripts/services/gateway/flow_service_v2.js，将GatewayFlowService 改成 GatewayFlowServiceV2，并且所有的请求地址前加上V2

7.修改gulpfile.js

在resources/gulpfile.js中的 JS_APP 中添加

  'app/scripts/services/gateway/flow_service_v2.js',
  'app/scripts/services/gateway/api_service_v2.js'

7.启动SentinelDashboard控制台

java -Dserver.port=9111 -jar sentinel-dashboard-1.8.3.jar -Dcsp.sentinel.app.type=1

通过浏览器打开http://localhost:9111/即可访问Sentinel控制台，默认用户名和密码都是sentinel

8.启动网关项目

由于当前项目为网关项目，需要标识，启动项目时需要设置 -Dcsp.sentinel.app.type=1 或者在启动类中添加System.setProperty(“csp.sentinel.app.type”, “1”);

9.使用网关限流

由于Zuul只支持RoutID和API分组，所以我们需要先在网关API管理中新增一个API分组

然后在网关流控规则中新增一个限流规则 API类型选择API分组即可