后端业务防重的思考

大米饭66

已于 2023-07-20 17:14:03 修改

阅读量297

点赞数

文章标签： java 防重

于 2023-07-20 17:13:25 首次发布

本文链接：https://blog.csdn.net/oschina_40730821/article/details/131833976

版权

1.请求防重

同一请求路径下，短时间内出现相同参数的请求操作。某些重复变更请求如果编码不严谨会导致底层数据和逻辑出现错误。

相同参数的请求直接在最外层做防重拦截。

解决方案：

- 请求参数唯一标识，前端生成，后端拦截判断一定时间范围内重复拦截

- 重复请求后端拦截，后端解析请求参数，生成唯一码，拦截判断重复

2.业务防重

业务数据重复变更、重复插入，导致业务错误。

解决方案：

- 数据库唯一索引（数据防重）

- 状态机进行防重（数据防重）

- 数据版本管理、乐观锁。在变更操作时携带数据版本号，版本号匹配时才能修改目标数据（数据防重）

- 加锁，同一时间资源访问限制（逻辑防重）

项目实践：

请求防重和业务防重都需要做。请求防重可以做成通用逻辑or组件抽取出来。这里只介绍请求防重的一个具体实现。

整体大致逻辑：

1、后端读取所有请求参数

2、使用hash算法将请求参数和路径 hash后生成请求唯一码

3、利用redis的单线程执行命令的机制，使用 setNx 判断是否存在相同key，不存在就插入

实现碰到的问题

如何拦截目标请求？

因为项目使用 spring ，直接使用扩展点 HandlerInterceptor ，直接使用 preHandle 扩展点进行拦截判断方法上是否标注 @RepeatRequest 注解。

注解

import org.springframework.core.annotation.AliasFor;

import java.lang.annotation.*;

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
public @interface RepeatRequest {

    @AliasFor("interval")
    long value() default 1000L;

    /**
     * 重复请求的判定间隔时间  不能 <= 0 ，否则失效
     */
    @AliasFor("value")
    long interval() default 1000L;

    /**
     * 判定 重复请求后的提示信息
     *
     * @return
     */
    String message() default "请勿重复提交,请稍后再试";
}

拦截器

public interface HandlerInterceptor {


    /**
    *  主要拦截逻辑
    * 
    **/
    default boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
        return true;
    }

    default void postHandle(HttpServletRequest request, HttpServletResponse response, Object handler, @Nullable ModelAndView modelAndView) throws Exception {
    }

    default void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, @Nullable Exception ex) throws Exception {
    }
}

具体列子



/**
 * @author: yu
 * @description: 防重拦截  短时间内 重复提交 重复请求
 * @create: 2023-07-18 16:02
 **/
public class RepeatSubmitInterceptor implements HandlerInterceptor {


    private RedisService redisService;

    static final String REPEAT_SUBMIT_KEY = "repeat_submit_key:%s_%s";

    private static final Logger logger = LoggerFactory.getLogger(RepeatSubmitInterceptor.class);

    public RepeatSubmitInterceptor(RedisService redisService) {
        this.redisService = redisService;
    }

    @Override
    public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {

        if (!(handler instanceof HandlerMethod)) {
            return true;
        }

        HandlerMethod handlerMethod = (HandlerMethod) handler;
        Method method = handlerMethod.getMethod();

        RepeatRequest annotation = AnnotationUtils.getAnnotation(method, RepeatRequest.class);
        if (annotation == null) {
            return true;
        }

        long interval = annotation.interval();
        if (interval <= 0L) {
            return true;
        }
        String message = annotation.message();

        String controllerMethodPath = handlerMethod.toString();
        // 这里使用了相对路径 ，没有用绝对路径，参数相同的不同调用方也会拦住
        String requestURI = request.getRequestURI();
        Map<String, String[]> parameterMap = request.getParameterMap();
        String messagePayload = getMessagePayload(request);
        String paramsMap = JSONUtil.toJsonStr(parameterMap);
        String appName = SpringUtil.getApplicationContext().getEnvironment().getProperty("app.name");
        String redisKey = String.format(REPEAT_SUBMIT_KEY, appName, controllerMethodPath);

        String value = requestURI + paramsMap + messagePayload;
        String encodeValue = Hashing.md5().hashString(value, StandardCharsets.UTF_8).toString();
        boolean exist = false;
        try {
            // 后续 节省空间 看以替换成 bitmap
            exist = !redisService.setNx(redisKey + encodeValue, encodeValue, interval, TimeUnit.MILLISECONDS);
        } catch (Exception e) {
            // redis 不可用，不阻塞主流程 ，只打印日志
            logger.error("redis 异常", e);
        }

        if (exist) {
            logger.info("重复请求 rediskey:{},value ：{}", redisKey+encodeValue, value);
            throw new BaseRunTimeException(BaseExceptionEnum.SYS_ERROR.getCode(), message);
        }


        logger.info("paramsMap:{},messagePayload:{}", paramsMap, messagePayload);
        return true;
    }


        /**
         * 1.处理注解信息
         *
         * 1.取出所有参数  和 url
         * 2.拼接成 唯一码 ，存入redis
         * 3.判断是否存在
         */
    

    protected String getMessagePayload(HttpServletRequest request) {
        KdmpContentCachingRequestWrapper wrapper = WebUtils.getNativeRequest(request, KdmpContentCachingRequestWrapper.class);
        if (wrapper == null) {
            return null;
        }
        byte[] buf = wrapper.getContentAsByteArray();
        if (buf.length > 0) {
            int length = buf.length;
            try {
                return new String(buf, 0, length, wrapper.getCharacterEncoding());
            } catch (UnsupportedEncodingException ex) {
                return "[unknown]";
            }
        }

        return null;

    }

  

}

后端如何读取所有请求参数？

request对象中的 body 数据不支持重复读取，所以需要首先实现可以重复读取body的 request 对象。

使用 HttpServletRequestWrapper 的功能，同时参考 spring-web utils包中 ContentCachingRequestWrapper 的实现。主要缓存了流的内容，实现可重复读取的流。

实现 MyContentCachingRequestWrapper

import org.apache.commons.io.IOUtils;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.lang.Nullable;
import org.springframework.web.util.ContentCachingResponseWrapper;
import org.springframework.web.util.WebUtils;

import javax.servlet.ReadListener;
import javax.servlet.ServletInputStream;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletRequestWrapper;
import java.io.*;
import java.net.URLEncoder;
import java.util.*;

/**
 * {@link javax.servlet.http.HttpServletRequest} wrapper that caches all content read from
 * the {@linkplain #getInputStream() input stream} and {@linkplain #getReader() reader},
 * and allows this content to be retrieved via a {@link #getContentAsByteArray() byte array}.
 *
 * <p>This class acts as an interceptor that only caches content as it is being
 * read but otherwise does not cause content to be read. That means if the request
 * content is not consumed, then the content is not cached, and cannot be
 * retrieved via {@link #getContentAsByteArray()}.
 *
 * <p>Used e.g. by {@link org.springframework.web.filter.AbstractRequestLoggingFilter}.
 * Note: As of Spring Framework 5.0, this wrapper is built on the Servlet 3.1 API.
 *
 * @author Juergen Hoeller
 * @author Brian Clozel
 * @see ContentCachingResponseWrapper
 * <p>
 * 重写下，可以重复读取流
 * @since 4.1.3
 */
public class MyContentCachingRequestWrapper extends HttpServletRequestWrapper {

    private final ByteArrayOutputStream cachedContent;


    @Nullable
    private ServletInputStream inputStream;

    @Nullable
    private BufferedReader reader;


    /**
     * Create a new ContentCachingRequestWrapper for the given servlet request.
     *
     * @param request the original servlet request
     */
    public MyContentCachingRequestWrapper(HttpServletRequest request) {
        super(request);
        int contentLength = request.getContentLength();
        this.cachedContent = new ByteArrayOutputStream(contentLength >= 0 ? contentLength : 1024);

        // 直接缓存流
        try {
            if (isFormBody()) {
                ServletInputStream inputStream = this.getRequest().getInputStream();
                IOUtils.copy(inputStream, this.cachedContent);
            }

        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }


    @Override
    public ServletInputStream getInputStream() throws IOException {
        // 每次都是一个新流
        this.inputStream = new ContentCachingInputStream(cachedContent.toByteArray());
        return this.inputStream;
    }

    @Override
    public String getCharacterEncoding() {
        String enc = super.getCharacterEncoding();
        return (enc != null ? enc : WebUtils.DEFAULT_CHARACTER_ENCODING);
    }

    @Override
    public BufferedReader getReader() throws IOException {
        if (this.reader == null) {
            this.reader = new BufferedReader(new InputStreamReader(getInputStream(), getCharacterEncoding()));
        }
        return this.reader;
    }

    @Override
    public String getParameter(String name) {

        return super.getParameter(name);
    }


    @Override
    public Map<String, String[]> getParameterMap() {
        return super.getParameterMap();
    }

    @Override
    public Enumeration<String> getParameterNames() {
        return super.getParameterNames();
    }

    @Override
    public String[] getParameterValues(String name) {

        return super.getParameterValues(name);
    }

    private boolean isFormBody() {
        String contentType = getContentType();
        return (contentType != null &&
                (contentType.contains(MediaType.APPLICATION_JSON_VALUE)
                        || contentType.contains(MediaType.APPLICATION_XML_VALUE)
                )
        );
    }


    private boolean isFormPost() {
        String contentType = getContentType();
        return (contentType != null && (contentType.contains(MediaType.APPLICATION_FORM_URLENCODED_VALUE) || contentType.contains(MediaType.MULTIPART_FORM_DATA_VALUE)) &&
                (HttpMethod.POST.matches(getMethod())
                        || HttpMethod.GET.matches(getMethod())
                        || HttpMethod.PUT.matches(getMethod())
                        || HttpMethod.DELETE.matches(getMethod())
                        || HttpMethod.HEAD.matches(getMethod())
                        || HttpMethod.PATCH.matches(getMethod())
                )
        );
    }


    /**
     * Return the cached request content as a byte array.
     * <p>The returned array will never be larger than the content cache limit.
     * <p><strong>Note:</strong> The byte array returned from this method
     * reflects the amount of content that has has been read at the time when it
     * is called. If the application does not read the content, this method
     * returns an empty array.
     *
     * @see #ContentCachingRequestWrapper(HttpServletRequest, int)
     */
    public byte[] getContentAsByteArray() {
        return this.cachedContent.toByteArray();
    }

    /**
     * Template method for handling a content overflow: specifically, a request
     * body being read that exceeds the specified content cache limit.
     * <p>The default implementation is empty. Subclasses may override this to
     * throw a payload-too-large exception or the like.
     *
     * @param contentCacheLimit the maximum number of bytes to cache per request
     *                          which has just been exceeded
     * @see #ContentCachingRequestWrapper(HttpServletRequest, int)
     * @since 4.3.6
     */
    protected void handleContentOverflow(int contentCacheLimit) {
    }


    private class ContentCachingInputStream extends ServletInputStream {

        private final ByteArrayInputStream is;

        private boolean overflow = false;

        public ContentCachingInputStream(byte[] content) {
            this.is = new ByteArrayInputStream(content);
        }

        @Override
        public int read() throws IOException {
            int ch = this.is.read();
            return ch;
        }

        @Override
        public int read(byte[] b) throws IOException {
            int count = this.is.read(b);
            return count;
        }


        @Override
        public int read(final byte[] b, final int off, final int len) throws IOException {
            int count = this.is.read(b, off, len);
            return count;
        }

        @Override
        public int readLine(final byte[] b, final int off, final int len) throws IOException {
            int count = this.is.read(b, off, len);
            return count;
        }

        @Override
        public boolean isFinished() {
            return this.is.available() == 0;
        }

        @Override
        public boolean isReady() {
            return true;
        }

        @Override
        public void setReadListener(ReadListener readListener) {

        }
    }

}

public class RequestWrapFilter extends OncePerRequestFilter implements OrderedFilter {

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {

        HttpServletRequest requestToUse = request;
        if (!(request instanceof MyContentCachingRequestWrapper)) {
            requestToUse =  new MyContentCachingRequestWrapper(request);
        }
        logger.debug("RequestWrapperFilter success");
        filterChain.doFilter(requestToUse, response);
    }

    @Override
    public int getOrder() {
        return REQUEST_WRAPPER_FILTER_MAX_ORDER ;
    }
}

结尾：上述代码虽然不完整，但是已经包含了核心逻辑和思想。

迭代想法：组件化可以封装成 springboot -starter 开放调用、代码步骤抽象、判重逻辑抽象支持自定义如 mysql 等其他方式