蓝绿和数据隔离

K8S支持蓝绿

Ingress-nginx的canary-*注解

配置

nginx.ingress.kubernetes.io/canary: 'true'
nginx.ingress.kubernetes.io/canary-by-header: version
nginx.ingress.kubernetes.io/canary-by-header-value: blue

注意

• 配置中，除了string是支持的类型，不需要加引号，boolean值需要加引号
• 要使配置生效，Nginx中必须要有不带canary配置或者 带有canary但是canary-by-header-value不一样的的另一套环境配置，单独的使用是无效的。

访问的时候只要在header中加上version = blue 。就会被Nginx代理到blue环境。

此方案的优点是可以在k8s中将所有路由配置在一个yaml中，然后可以直接切全套环境的蓝绿。

参考

• 如何通过NginxIngressController实现应用服务的灰度发布_容器服务 Kubernetes 版 ACK(ACK)-阿里云帮助中心​​​​​​

Web获取蓝绿标签

HandlerInterceptor+ThreadLocal

提供2种实现方式：从cookie中获取蓝绿和从header中获取蓝绿。

从cookie中获取蓝绿

@Slf4j
public class RequestCookieHandler implements HandlerInterceptor {

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
        Cookie[] cookies = request.getCookies();
        if(null == cookies) {
            return true;
        }
        for (Cookie cookie : cookies) {
            String name = cookie.getName();
            if(BLUE_HEADER_KEY.equals(name)) {
                RequestHeaderThreadLocal.put(cookie.getValue());
                log.info("请求cookie的蓝绿属性值： {}", cookie.getValue());
            }
        }
        return true;
    }

    @Override
    public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex) throws Exception {
        RequestHeaderThreadLocal.remove();
    }


}

从header中获取蓝绿

@Slf4j
public class RequestHeaderHandler implements HandlerInterceptor {

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
        String header = request.getHeader(BLUE_HEADER_KEY);
        RequestHeaderThreadLocal.put(header);
        log.info("请求header的蓝绿属性值： {}", header);
        return true;
    }

    @Override
    public void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView) throws Exception {
        RequestHeaderThreadLocal.remove();
    }
}

自动装配

@Slf4j
@Configuration
public class RequestAutoConfigure {

    @Value("${bg.type:header}")
    private String bgType;

    @Bean
    public HandlerInterceptor requestBgHandler() {
        log.info("启动自动装配获取requestHandlerType = {}", bgType);
        if(bgType.equals("cookie")) {
            return requestCookieHandler();
        }
        if(bgType.equals("header")) {
            return requestHeaderHandler();
        }
        throw new RuntimeException("未知bgHandler");
    }

    @Bean
    public RequestCookieHandler requestCookieHandler() {
        return new RequestCookieHandler();
    }

    @Bean
    public RequestHeaderHandler requestHeaderHandler() {
        return new RequestHeaderHandler();
    }

}

Feign下沉

如果将threadLocal中的蓝绿下沉到feign服务层中？

feign.RequestInterceptor

@Slf4j
public class FeignHeaderInterceptor implements RequestInterceptor {
    @Override
    public void apply(RequestTemplate template) {
        template.header(RequestHeaderThreadLocal.BLUE_HEADER_KEY, RequestHeaderThreadLocal.get());
        log.info("feign 调用增加header_version信息 ： {}", RequestHeaderThreadLocal.get());
    }
}

通过实现RequestInterceptor 来自定义请求的header信息，将蓝绿参数传入feign调用的request.header中。

自动装配

@Configuration
@Slf4j
public class FeignRequestAutoConfigure {

    /*
       自动装配feign.requestInterceptor
    */
    @Bean(name = "feignHeaderInterceptor")
    public RequestInterceptor requestInterceptor() {
        log.info("自动装配feign调用请求头增加蓝绿拦截器");
        return new FeignHeaderInterceptor();
    }

}

Feign服务层使用上述的 <从header中获取蓝绿>的办法，获取到蓝绿标签。

数据隔离

org.apache.ibatis.plugin.Interceptor

要实现蓝绿数据隔离，在不影响业务逻辑的前提下，很自然的想到的就是mybatis-plugin。通过自定义plugin来完成数据的分离。

在前面说明中，我们在feign层中已经获取到了当前线程的蓝绿标签。后续拓展的思路是：

在需要支持蓝绿数据分离的表中增加字段bg,同时实体类型也需要加属性bg。

然后通过mybatis的拦截器去拦截当前实体对象是否有bg属性，如果有就获取当前线程的蓝绿标识根据sql的类型重新组装sql。

Insert语句处理

@Intercepts({
        @Signature(type = Executor.class, method = "update", args = {MappedStatement.class, Object.class}),
})
@Slf4j
public class InsertInterceptor implements Interceptor {

    @Override
    public Object intercept(Invocation invocation) throws Throwable {
        MappedStatement mappedStatement = (MappedStatement)invocation.getArgs()[0];
        SqlCommandType commandType = mappedStatement.getSqlCommandType();
        if(!SqlCommandType.INSERT.equals(commandType)) {
            return invocation.proceed();
        }
        Object param = invocation.getArgs()[1];

        String id = mappedStatement.getId();
        String className = id.substring(0, id.lastIndexOf("."));
        boolean hasBg = InterceptorUtil.hasBgField(className);
        if(hasBg) {
            BeanUtils.setProperty(param, BG_FIELD, RequestHeaderThreadLocal.getBG());
        }

        return invocation.proceed();
    }

    @Override
    public Object plugin(Object target) {
        return Plugin.wrap(target, this);
    }

    @Override
    public void setProperties(Properties properties) {
    }

}

此处贴出hasField代码：

public class InterceptorUtil {

    public static final String BG_FIELD = "bg";

    public static boolean hasBgField(String className) throws ClassNotFoundException {
        Class<?> classObj = Class.forName(className);
        Class baseEntity = null;
        for (Type type : classObj.getGenericInterfaces()) {
            if(type instanceof ParameterizedType) {
                ParameterizedType inter = (ParameterizedType) type;
                Type t = inter.getActualTypeArguments()[0];
                baseEntity = (Class) t;
            }
        }
        Field[] fields = baseEntity.getDeclaredFields();
        for (Field field : fields) {
            if(field.getName().equals(BG_FIELD)) {
                return true;
            }
        }
        return false;
    }

}

非insert语句处理

@Slf4j
@Intercepts({
        @Signature(type = StatementHandler.class, method = "prepare", args = {Connection.class, Integer.class}),
})
public class NoInsertInterceptor implements Interceptor {

    @Override
    public Object intercept(Invocation invocation) throws Throwable {
        Object target = invocation.getTarget();
        StatementHandler realTarget = realTarget(target);
        MetaObject metaObject = SystemMetaObject.forObject(realTarget);
        processByField(realTarget, metaObject);
        return invocation.proceed();
    }

    private void processByField(StatementHandler handler, MetaObject metaObject) throws ClassNotFoundException {
        MappedStatement mappedStatement = (MappedStatement) metaObject
                .getValue("delegate.mappedStatement");
        SqlCommandType sqlCommandType = mappedStatement.getSqlCommandType();
        if(SqlCommandType.INSERT.equals(sqlCommandType)) {
            return;
        }
        String id = mappedStatement.getId();
        String className = id.substring(0, id.lastIndexOf("."));
        boolean hasBg = InterceptorUtil.hasBgField(className);
        if(hasBg) {
            resetSql(handler, metaObject);
        }
    }

    private void resetSql(StatementHandler handler, MetaObject metaObject) {
        BoundSql boundSql = handler.getBoundSql();
        String sql = boundSql.getSql().toLowerCase();
        if(sql.contains(" where")) {
            sql += " and `bg` = '" + RequestHeaderThreadLocal.getBG() + "'";
        } else {
            sql += " where `bg` = '" + RequestHeaderThreadLocal.getBG() + "'";
        }
        metaObject.setValue("delegate.boundSql.sql", sql);
    }

    public static <T> T realTarget(Object target) {
        if (Proxy.isProxyClass(target.getClass())) {
            MetaObject metaObject = SystemMetaObject.forObject(target);
            return realTarget(metaObject.getValue("h.target"));
        }
        return (T) target;
    }

    @Override
    public Object plugin(Object target) {
        return Plugin.wrap(target, this);
    }

    @Override
    public void setProperties(Properties properties) {
    }
}

自动装配

@Configuration
@Slf4j
@ConditionalOnBean(SqlSessionFactory.class)
public class MybatisConfig {

    @Resource
    private SqlSessionFactory sqlSessionFactory;
    @Bean
    public void addInterceptor() {
        log.info("mybatis 初始化添加bgInterceptor拦截器");
        org.apache.ibatis.session.Configuration configuration = sqlSessionFactory.getConfiguration();
        configuration.addInterceptor(new NoInsertInterceptor());
        configuration.addInterceptor(new InsertInterceptor());
    }
}

到此，从蓝绿请求的分发到数据隔离就完成了。