sentinel源码分析-02客户端处理服务端下发请求

客户端处理服务端下发请求

sentinel客户端中会存储一些限流规则而这些限流规则是我们通过页面配置后由服务端下发下来的，下面让我们来看看客户端是如何接收服务端下发的命令的

上面我们分析了在执行init方法时HeartbeatSenderInitFunc心跳处理，现在我们继续分析一下CommandCenterInitFunc方法来完成sentinel服务端发送过来的请求相关操作

@InitOrder(-1)
public class CommandCenterInitFunc implements InitFunc {

    @Override
    public void init() throws Exception {
        CommandCenter commandCenter = CommandCenterProvider.getCommandCenter();

        if (commandCenter == null) {
            RecordLog.warn("[CommandCenterInitFunc] Cannot resolve CommandCenter");
            return;
        }
        //注册处理器
        commandCenter.beforeStart();
        //启动命令处理
        commandCenter.start();
        RecordLog.info("[CommandCenterInit] Starting command center: "
                + commandCenter.getClass().getCanonicalName());
    }
}

beforeStart方法会将所有处理器已key-value的形式注册到handlermap中，其中key为CommandMapping注解配置的名称，value为对应的命令处理器

public void beforeStart() throws Exception {
    // Register handlers 注册处理器
    Map<String, CommandHandler> handlers = CommandHandlerProvider.getInstance().namedHandlers();
    registerCommands(handlers);
}


public Map<String, CommandHandler> namedHandlers() {
    Map<String, CommandHandler> map = new HashMap<String, CommandHandler>();
    for (CommandHandler handler : serviceLoader) {
        String name = parseCommandName(handler);
        if (!StringUtil.isEmpty(name)) {
            map.put(name, handler);
        }
    }
    return map;
}

private String parseCommandName(CommandHandler handler) {
    //通过注解获取handler名称
    CommandMapping commandMapping = handler.getClass().getAnnotation(CommandMapping.class);
    if (commandMapping != null) {
        return commandMapping.name();
    } else {
        return null;
    }
}

注册处理器完成后执行start方法，启动命令中心

public void start() throws Exception {
    //cpu核数
    int nThreads = Runtime.getRuntime().availableProcessors();
    //创建处理请求线程池
    this.bizExecutor = new ThreadPoolExecutor(nThreads, nThreads, 0L, TimeUnit.MILLISECONDS,
        new ArrayBlockingQueue<Runnable>(10),
        new NamedThreadFactory("sentinel-command-center-service-executor"),
        new RejectedExecutionHandler() {
            @Override
            public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
                CommandCenterLog.info("EventTask rejected");
                throw new RejectedExecutionException();
            }
        });

    Runnable serverInitTask = new Runnable() {
        int port;

        {
            try {
                //从配置文件中获取配置的监听端口
                port = Integer.parseInt(TransportConfig.getPort());
            } catch (Exception e) {
                port = DEFAULT_PORT;
            }
        }

        @Override
        public void run() {
            boolean success = false;
            //创建ServerSocket
            ServerSocket serverSocket = getServerSocketFromBasePort(port);

            if (serverSocket != null) {
                CommandCenterLog.info("[CommandCenter] Begin listening at port " + serverSocket.getLocalPort());
                socketReference = serverSocket;
                executor.submit(new ServerThread(serverSocket));
                success = true;
                port = serverSocket.getLocalPort();
            } else {
                CommandCenterLog.info("[CommandCenter] chooses port fail, http command center will not work");
            }

            if (!success) {
                port = PORT_UNINITIALIZED;
            }

            TransportConfig.setRuntimePort(port);
            executor.shutdown();
        }

    };

    new Thread(serverInitTask).start();
}

 private static ServerSocket getServerSocketFromBasePort(int basePort) {
    int tryCount = 0;
    while (true) {
        try {
            //根据配置端口开始每个端口重试3次，如果失败端口不停增长继续尝试
            ServerSocket server = new ServerSocket(basePort + tryCount / 3, 100);
            server.setReuseAddress(true);
            return server;
        } catch (IOException e) {
            tryCount++;
            try {
                TimeUnit.MILLISECONDS.sleep(30);
            } catch (InterruptedException e1) {
                break;
            }
        }
    }
    return null;
}

这里会获取配置文件配置的端口，如果端口不可用则尝试3次，未成功将端口增长1继续尝试

class ServerThread extends Thread {

    private ServerSocket serverSocket;

    ServerThread(ServerSocket s) {
        this.serverSocket = s;
        setName("sentinel-courier-server-accept-thread");
    }

    @Override
    public void run() {
        while (true) {
            Socket socket = null;
            try {
                //等待客户端接入
                socket = this.serverSocket.accept();
                setSocketSoTimeout(socket);
                //封装处理请求数据结构
                HttpEventTask eventTask = new HttpEventTask(socket);
                bizExecutor.submit(eventTask);
            } catch (Exception e) {
                CommandCenterLog.info("Server error", e);
                if (socket != null) {
                    try {
                        socket.close();
                    } catch (Exception e1) {
                        CommandCenterLog.info("Error when closing an opened socket", e1);
                    }
                }
                try {
                    // In case of infinite log.
                    Thread.sleep(10);
                } catch (InterruptedException e1) {
                    // Indicates the task should stop.
                    break;
                }
            }
        }
    }
}

最终将服务端下发请求，封装到HttpEventTask处理，我们进入HttpEventTask类，该类实现了Runnable接口

public void run() {
    if (socket == null) {
        return;
    }

    BufferedReader in = null;
    PrintWriter printWriter = null;
    try {
        long start = System.currentTimeMillis();
        //输入流
        in = new BufferedReader(new InputStreamReader(socket.getInputStream(), SentinelConfig.charset()));
        OutputStream outputStream = socket.getOutputStream();

        printWriter = new PrintWriter(
            new OutputStreamWriter(outputStream, Charset.forName(SentinelConfig.charset())));

        String line = in.readLine();
        CommandCenterLog.info("[SimpleHttpCommandCenter] socket income: " + line
            + "," + socket.getInetAddress());
        //解析请求
        CommandRequest request = parseRequest(line);

        if (line.length() > 4 && StringUtil.equalsIgnoreCase("POST", line.substring(0, 4))) {
            // Deal with post method
            // Now simple-http only support form-encoded post request.
            String bodyLine = null;
            boolean bodyNext = false;
            boolean supported = false;
            int maxLength = 8192;
            while (true) {
                // Body processing
                if (bodyNext) {
                    if (!supported) {
                        break;
                    }
                    char[] bodyBytes = new char[maxLength];
                    int read = in.read(bodyBytes);
                    String postData = new String(bodyBytes, 0, read);
                    parseParams(postData, request);
                    break;
                }

                bodyLine = in.readLine();
                if (bodyLine == null) {
                    break;
                }
                // Body seperator
                if (StringUtil.isEmpty(bodyLine)) {
                    bodyNext = true;
                    continue;
                }
                // Header processing
                int index = bodyLine.indexOf(":");
                if (index < 1) {
                    continue;
                }
                String headerName = bodyLine.substring(0, index);
                String header = bodyLine.substring(index + 1).trim();
                if (StringUtil.equalsIgnoreCase("content-type", headerName)) {
                    if (StringUtil.equals("application/x-www-form-urlencoded", header)) {
                        supported = true;
                    } else {
                        // not support request
                        break;
                    }
                } else if (StringUtil.equalsIgnoreCase("content-length", headerName)) {
                    try {
                        int len = new Integer(header);
                        if (len > 0) {
                            maxLength = len;
                        }
                    } catch (Exception e) {
                    }
                }
            }
        }

        // Validate the target command.
        //校验请求
        String commandName = HttpCommandUtils.getTarget(request);
        if (StringUtil.isBlank(commandName)) {
            badRequest(printWriter, "Invalid command");
            return;
        }

        // Find the matching command handler.
        //根据请求名称获取请求处理器
        CommandHandler<?> commandHandler = SimpleHttpCommandCenter.getHandler(commandName);
        if (commandHandler != null) {
            //执行处理请求逻辑，然后返回结果
            CommandResponse<?> response = commandHandler.handle(request);
            handleResponse(response, printWriter, outputStream);
        } else {
            // No matching command handler.
            badRequest(printWriter, "Unknown command `" + commandName + '`');
        }
        printWriter.flush();

        long cost = System.currentTimeMillis() - start;
        CommandCenterLog.info("[SimpleHttpCommandCenter] Deal a socket task: " + line
            + ", address: " + socket.getInetAddress() + ", time cost: " + cost + " ms");
    } catch (Throwable e) {
        CommandCenterLog.warn("[SimpleHttpCommandCenter] CommandCenter error", e);
        try {
            if (printWriter != null) {
                String errorMessage = SERVER_ERROR_MESSAGE;
                if (!writtenHead) {
                    internalError(printWriter, errorMessage);
                } else {
                    printWriter.println(errorMessage);
                }
                printWriter.flush();
            }
        } catch (Exception e1) {
            CommandCenterLog.warn("[SimpleHttpCommandCenter] Close server socket failed", e);
        }
    } finally {
        closeResource(in);
        closeResource(printWriter);
        closeResource(socket);
    }
}

这里代码比较长，其实主要逻辑就是将字符串请求解析为CommandRequest请求，根据请求获取对应的处理器，如果没有找到返回错误请求，如果有对象请求处理器，则执行命令处理然后返回请求结果

这里我们以设置流控规则命令来分析，会执行ModifyRulesCommandHandler的handle方法

public CommandResponse<String> handle(CommandRequest request) {
    //类型
    String type = request.getParam("type");
    // rule data in get parameter
    //流控配置
    String data = request.getParam("data");
    if (StringUtil.isNotEmpty(data)) {
        try {
            data = URLDecoder.decode(data, "utf-8");
        } catch (Exception e) {
            RecordLog.info("Decode rule data error", e);
            return CommandResponse.ofFailure(e, "decode rule data error");
        }
    }

    RecordLog.info(String.format("Receiving rule change (type: %s): %s", type, data));

    String result = "success";

    if (FLOW_RULE_TYPE.equalsIgnoreCase(type)) {//限流
        List<FlowRule> flowRules = JSONArray.parseArray(data, FlowRule.class);
        FlowRuleManager.loadRules(flowRules);
        if (!writeToDataSource(getFlowDataSource(), flowRules)) {
            result = WRITE_DS_FAILURE_MSG;
        }
        return CommandResponse.ofSuccess(result);
    } else if (AUTHORITY_RULE_TYPE.equalsIgnoreCase(type)) {//授权
        List<AuthorityRule> rules = JSONArray.parseArray(data, AuthorityRule.class);
        AuthorityRuleManager.loadRules(rules);
        if (!writeToDataSource(getAuthorityDataSource(), rules)) {
            result = WRITE_DS_FAILURE_MSG;
        }
        return CommandResponse.ofSuccess(result);
    } else if (DEGRADE_RULE_TYPE.equalsIgnoreCase(type)) {//熔断
        List<DegradeRule> rules = JSONArray.parseArray(data, DegradeRule.class);
        DegradeRuleManager.loadRules(rules);
        if (!writeToDataSource(getDegradeDataSource(), rules)) {
            result = WRITE_DS_FAILURE_MSG;
        }
        return CommandResponse.ofSuccess(result);
    } else if (SYSTEM_RULE_TYPE.equalsIgnoreCase(type)) {//系统规则
        List<SystemRule> rules = JSONArray.parseArray(data, SystemRule.class);
        SystemRuleManager.loadRules(rules);
        if (!writeToDataSource(getSystemSource(), rules)) {
            result = WRITE_DS_FAILURE_MSG;
        }
        return CommandResponse.ofSuccess(result);
    }
    return CommandResponse.ofFailure(new IllegalArgumentException("invalid type"));
}

如果是限流命令，则会执行FlowRuleManager.loadRules(flowRules)方法

public static void loadRules(List<FlowRule> rules) {
    currentProperty.updateValue(rules);
}

public boolean updateValue(T newValue) {
    if (isEqual(value, newValue)) {
        return false;
    }
    RecordLog.info("[DynamicSentinelProperty] Config will be updated to: " + newValue);

    value = newValue;
    for (PropertyListener<T> listener : listeners) {
        listener.configUpdate(newValue);
    }
    return true;
}

最终会执行到FlowPropertyListener类的configUpdate方法

 public void configUpdate(List<FlowRule> value) {
    //构造限流规则
    Map<String, List<FlowRule>> rules = FlowRuleUtil.buildFlowRuleMap(value);
    if (rules != null) {
        flowRules.clear();
        //将限流规则集合放入flowRules中以key-value的形式存入
        flowRules.putAll(rules);
    }
    RecordLog.info("[FlowRuleManager] Flow rules received: " + flowRules);
}


public static <K> Map<K, List<FlowRule>> buildFlowRuleMap(List<FlowRule> list, Function<FlowRule, K> groupFunction,
                                                              Predicate<FlowRule> filter, boolean shouldSort) {
    Map<K, List<FlowRule>> newRuleMap = new ConcurrentHashMap<>();
    if (list == null || list.isEmpty()) {
        return newRuleMap;
    }
    Map<K, Set<FlowRule>> tmpMap = new ConcurrentHashMap<>();
    //遍历限流规则
    for (FlowRule rule : list) {
        if (!isValidRule(rule)) {
            RecordLog.warn("[FlowRuleManager] Ignoring invalid flow rule when loading new flow rules: " + rule);
            continue;
        }
        if (filter != null && !filter.test(rule)) {
            continue;
        }
        if (StringUtil.isBlank(rule.getLimitApp())) {
            rule.setLimitApp(RuleConstant.LIMIT_APP_DEFAULT);
        }
        //根据不同的限流类型创建不同的控制器
        TrafficShapingController rater = generateRater(rule);
        rule.setRater(rater);

        K key = groupFunction.apply(rule);
        if (key == null) {
            continue;
        }
        Set<FlowRule> flowRules = tmpMap.get(key);

        if (flowRules == null) {
            // Use hash set here to remove duplicate rules.
            flowRules = new HashSet<>();
            tmpMap.put(key, flowRules);
        }

        flowRules.add(rule);
    }
    Comparator<FlowRule> comparator = new FlowRuleComparator();
    for (Entry<K, Set<FlowRule>> entries : tmpMap.entrySet()) {
        List<FlowRule> rules = new ArrayList<>(entries.getValue());
        if (shouldSort) {
            // Sort the rules.
            Collections.sort(rules, comparator);
        }
        newRuleMap.put(entries.getKey(), rules);
    }

    return newRuleMap;
}

遍历限流规则，根据不同的限流类型创建不同的控制器

 private static TrafficShapingController generateRater(/*@Valid*/ FlowRule rule) {
    //QPS限流
    if (rule.getGrade() == RuleConstant.FLOW_GRADE_QPS) {
        switch (rule.getControlBehavior()) {
            case RuleConstant.CONTROL_BEHAVIOR_WARM_UP://预热模式
                return new WarmUpController(rule.getCount(), rule.getWarmUpPeriodSec(),
                    ColdFactorProperty.coldFactor);
            case RuleConstant.CONTROL_BEHAVIOR_RATE_LIMITER://匀速排队
                return new RateLimiterController(rule.getMaxQueueingTimeMs(), rule.getCount());
            case RuleConstant.CONTROL_BEHAVIOR_WARM_UP_RATE_LIMITER://预热模式+匀速排队
                return new WarmUpRateLimiterController(rule.getCount(), rule.getWarmUpPeriodSec(),
                    rule.getMaxQueueingTimeMs(), ColdFactorProperty.coldFactor);
            case RuleConstant.CONTROL_BEHAVIOR_DEFAULT:
            default:
                // Default mode or unknown mode: default traffic shaping controller (fast-reject).
        }
    }
    //默认达到阈值直接拒绝
    return new DefaultController(rule.getCount(), rule.getGrade());
}

到这里我们分析完了服务端下发命令到客户端后客户端是如何处理请求和保存数据的，我们发现这些数据都保存在内存中，组件重启就会丢失，所以需要外部持久化存储规则数据。