一、什么是幂等性?
幂等性:对一个接口执行重复的多次请求,与一次请求所产生的结果是相同的
二、哪些请求天生就是幂等的?
查询操作:一般都是天然幂等的
删除操作:删除请求在大多数情况下也是幂等的,但是ABA场景下除外
比如,先请求了一次删除A的操作,但由于响应超时,又自动请求了一次删除A的操作,如果在两次请求之间,又插入了一次A,而实际上新插入的这一次A,是不应该被删除的,这就是ABA问题,不过,在大多数业务场景中,ABA问题都是可以忽略的。
更新操作:在大多数场景下也是天然幂等的,其例外是也会存在ABA的问题,更重要的是,比如执行update table set a = a + 1 where v = 1
这样的更新就非幂等了。
插入操作:插入大多数情况下都是非幂等的,除非是利用数据库唯一索引来保证数据不会重复产生。
三、为什么需要幂等
1.超时重试
当发起一次RPC请求时,难免会因为网络不稳定而导致请求失败,一般遇到这样的问题我们希望能够重新请求一次,正常情况下没有问题,但有时请求实际上已经发出去了,只是在请求响应时网络异常或者超时,此时,请求方如果再重新发起一次请求,那被请求方就需要保证幂等了。
2.异步回调
异步回调是提升系统接口吞吐量的一种常用方式,很明显,此类接口一定是需要保证幂等性的。
3.消息队列
现在常用的消息队列框架,比如:Kafka、RocketMQ、RabbitMQ在消息传递时都会采取At least once原则(也就是至少一次原则,在消息传递时,不允许丢消息,但是允许有重复的消息),既然消息队列不保证不会出现重复的消息,那消费者自然要保证处理逻辑的幂等性了。
4.表单提交
短时间内,如果重复提交多次,如果不用业务主键控制,就会重复插入相同的数据,这种情况其实是需要保证接口幂等的。
四、实现幂等的关键因素
关键因素1
幂等唯一标识,可以叫它幂等号或者幂等令牌或者全局ID,总之就是客户端与服务端一次请求时的唯一标识,一般情况下由客户端来生成,也可以让第三方来统一分配。
关键因素2
有了唯一标识以后,服务端只需要确保这个唯一标识只被使用一次即可,一种常见的方式就是利用数据库的唯一索引。
五、接口幂等性实现
5.1 针对RPC接口调用或者后台服务见接口调用
下面演示一种利用AOP+Redis来实现的方式。
1.自定义注解
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Target(value = ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface Idempotent {
/**
* 参数名,表示将从哪个参数中获取属性值。
* 获取到的属性值将作为KEY。
*
* @return
*/
String name() default "";
/**
* 属性,表示将获取哪个属性的值。
*
* @return
*/
String field() default "";
/**
* 参数类型
*
* @return
*/
Class type();
}
2.统一的请求入参对象
@Data
public class RequestData<T> {
private Header header;
private T body;
}
@Data
public class Header {
private String token;
}
@Data
public class Order {
String orderNo;
}
3.AOP处理
import com.springboot.micrometer.annotation.Idempotent;
import com.springboot.micrometer.entity.RequestData;
import com.springboot.micrometer.idempotent.RedisIdempotentStorage;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.stereotype.Component;
import javax.annotation.Resource;
import java.lang.reflect.Method;
import java.util.Map;
@Aspect
@Component
public class IdempotentAspect {
@Resource
private RedisIdempotentStorage redisIdempotentStorage;
@Pointcut("@annotation(com.springboot.micrometer.annotation.Idempotent)")
public void idempotent() {
}
@Around("idempotent()")
public Object methodAround(ProceedingJoinPoint joinPoint) throws Throwable {
MethodSignature signature = (MethodSignature) joinPoint.getSignature();
Method method = signature.getMethod();
Idempotent idempotent = method.getAnnotation(Idempotent.class);
String field = idempotent.field();
String name = idempotent.name();
Class clazzType = idempotent.type();
String token = "";
Object object = clazzType.newInstance();
Map<String, Object> paramValue = AopUtils.getParamValue(joinPoint);
if (object instanceof RequestData) {
RequestData idempotentEntity = (RequestData) paramValue.get(name);
token = String.valueOf(AopUtils.getFieldValue(idempotentEntity.getHeader(), field));
}
if (redisIdempotentStorage.delete(token)) {
return joinPoint.proceed();
}
return "重复请求";
}
}
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.reflect.CodeSignature;
import java.lang.reflect.Field;
import java.util.HashMap;
import java.util.Map;
public class AopUtils {
public static Object getFieldValue(Object obj, String name) throws Exception {
Field[] fields = obj.getClass().getDeclaredFields();
Object object = null;
for (Field field : fields) {
field.setAccessible(true);
if (field.getName().toUpperCase().equals(name.toUpperCase())) {
object = field.get(obj);
break;
}
}
return object;
}
public static Map<String, Object> getParamValue(ProceedingJoinPoint joinPoint) {
Object[] paramValues = joinPoint.getArgs();
String[] paramNames = ((CodeSignature) joinPoint.getSignature()).getParameterNames();
Map<String, Object> param = new HashMap<>(paramNames.length);
for (int i = 0; i < paramNames.length; i++) {
param.put(paramNames[i], paramValues[i]);
}
return param;
}
}
4.Token值生成
import com.springboot.micrometer.idempotent.RedisIdempotentStorage;
import com.springboot.micrometer.util.IdGeneratorUtil;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import javax.annotation.Resource;
@RestController
@RequestMapping("/idGenerator")
public class IdGeneratorController {
@Resource
private RedisIdempotentStorage redisIdempotentStorage;
@RequestMapping("/getIdGeneratorToken")
public String getIdGeneratorToken() {
String generateId = IdGeneratorUtil.generateId();
redisIdempotentStorage.save(generateId);
return generateId;
}
}
public interface IdempotentStorage {
void save(String idempotentId);
boolean delete(String idempotentId);
}
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Component;
import javax.annotation.Resource;
import java.io.Serializable;
import java.util.concurrent.TimeUnit;
@Component
public class RedisIdempotentStorage implements IdempotentStorage {
@Resource
private RedisTemplate<String, Serializable> redisTemplate;
@Override
public void save(String idempotentId) {
redisTemplate.opsForValue().set(idempotentId, idempotentId, 10, TimeUnit.MINUTES);
}
@Override
public boolean delete(String idempotentId) {
return redisTemplate.delete(idempotentId);
}
}
import java.util.UUID;
public class IdGeneratorUtil {
public static String generateId() {
return UUID.randomUUID().toString();
}
}
5. 请求示例
调用接口之前,先申请一个token,然后带着服务端返回的token值,再去请求。
import com.springboot.micrometer.annotation.Idempotent;
import com.springboot.micrometer.entity.Order;
import com.springboot.micrometer.entity.RequestData;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/order")
public class OrderController {
@RequestMapping("/saveOrder")
@Idempotent(name = "requestData", type = RequestData.class, field = "token")
public String saveOrder(@RequestBody RequestData<Order> requestData) {
return "success";
}
}
请求获取token值->带着token值,第一次请求成功->第二次请求失败。
5.2 针对表单重复提交
利用RepeaterFilter +拦截器 +注解实现(借鉴若依系统构建可重复读的inputstream,流一般只可被读取一次)
1.Repeatable 过滤器
@Component
@WebFilter
public class RepeatableFilter implements Filter {
@Override
public void init(FilterConfig filterConfig) {
}
@Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain)
throws IOException, ServletException {
ServletRequest requestWrapper = null;
if (request instanceof HttpServletRequest && StringUtils.startsWithIgnoreCase(request.getContentType(), MediaType.APPLICATION_JSON_VALUE)) {
requestWrapper = new RepeatedlyRequestWrapper((HttpServletRequest) request, response);
}
if (null == requestWrapper) {
chain.doFilter(request, response);
} else {
chain.doFilter(requestWrapper, response);
}
}
@Override
public void destroy() {
}
}
2. 构建可重复读取inputStream的request
public class RepeatedlyRequestWrapper extends HttpServletRequestWrapper
{
private final byte[] body;
public RepeatedlyRequestWrapper(HttpServletRequest request, ServletResponse response) throws IOException
{
super(request);
request.setCharacterEncoding("UTF-8");
response.setCharacterEncoding("UTF-8");
body = HttpHelper.getBodyString(request).getBytes("UTF-8");
}
@Override
public BufferedReader getReader() throws IOException
{
return new BufferedReader(new InputStreamReader(getInputStream()));
}
@Override
public ServletInputStream getInputStream() throws IOException
{
final ByteArrayInputStream bais = new ByteArrayInputStream(body);
return new ServletInputStream()
{
@Override
public int read() throws IOException
{
return bais.read();
}
@Override
public boolean isFinished()
{
return false;
}
@Override
public boolean isReady()
{
return false;
}
@Override
public void setReadListener(ReadListener readListener)
{
}
};
}
}
3.自定义注解防止表单重复提交
/**
* 自定义注解防止表单重复提交
*
* @author
*
*/
@Inherited
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface RepeatSubmit
{
/** 间隔时间 毫秒**/
public int interval() default 2000;
/** 提示消息**/
public String message() default "不允许重复提交,请稍后再试";
}
4. 防止重复提交拦截器
@Component
public abstract class RepeatSubmitInterceptor implements HandlerInterceptor
{
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception
{
if (handler instanceof HandlerMethod)
{
HandlerMethod handlerMethod = (HandlerMethod) handler;
Method method = handlerMethod.getMethod();
RepeatSubmit annotation = method.getAnnotation(RepeatSubmit.class);
if (annotation != null)
{
if (this.isRepeatSubmit(request, annotation))
{
RestResult ajaxResult = RestResult.fail(annotation.message());
//todo
ServletUtils.renderString(response, JSONObject.toJSONString(ajaxResult));
return false;
}
}
return true;
}
else
{
return true;
}
}
/**
* 验证是否重复提交由子类实现具体的防重复提交的规则
*
* @param request
* @return
* @throws Exception
*/
public abstract boolean isRepeatSubmit(HttpServletRequest request, RepeatSubmit annotation);
}
5.SameUrlDataInterceptor
/**
* 判断请求url和数据是否和上一次相同,
* 如果和上次相同,则是重复提交表单。 有效时间为10秒内。
*/
@Component
public class SameUrlDataInterceptor extends RepeatSubmitInterceptor
{
public final String REPEAT_PARAMS = "repeatParams";
public final String REPEAT_TIME = "repeatTime";
public final String CACHE_REPEAT_KEY = "repeatData#";
/**
* 间隔时间,单位:秒 默认10秒
*
* 两次相同参数的请求,如果间隔时间大于该参数,系统不会认定为重复提交的数据
*/
private int intervalTime = 5;
public void setIntervalTime(int intervalTime)
{
this.intervalTime = intervalTime;
}
@SuppressWarnings("unchecked")
@Override
public boolean isRepeatSubmit(HttpServletRequest request, RepeatSubmit annotation)
{
//todo 可重复读取的流
String nowParams = "";
if (request instanceof RepeatedlyRequestWrapper) {
RepeatedlyRequestWrapper repeatedlyRequest = (RepeatedlyRequestWrapper) request;
nowParams = HttpHelper.getBodyString(repeatedlyRequest);
}
// body参数为空,获取Parameter的数据
if (StringUtils.isEmpty(nowParams))
{
nowParams = JSONObject.toJSONString(request.getParameterMap());
}
Map<String, Object> nowDataMap = new HashMap<String, Object>();
nowDataMap.put(REPEAT_PARAMS, nowParams);
nowDataMap.put(REPEAT_TIME, System.currentTimeMillis());
// 请求地址(作为存放cache的key值)
String url = request.getRequestURI();
//唯一标识(key + url)
String cacheRepeatKey = CACHE_REPEAT_KEY + url;
Object sessionObj = RedisUtil.get(cacheRepeatKey);
if (sessionObj != null)
{
Map<String, Object> sessionMap = (Map<String, Object>) sessionObj;
if (sessionMap.containsKey(url))
{
Map<String, Object> preDataMap = (Map<String, Object>) sessionMap.get(url);
if (compareParams(nowDataMap, preDataMap) && compareTime(nowDataMap, preDataMap))
{
//System.out.println("wx....重复提交测试");
return true;
}
}
}
Map<String, Object> cacheMap = new HashMap<String, Object>();
cacheMap.put(url, nowDataMap);
RedisUtil.set(cacheRepeatKey, cacheMap, annotation.interval() / 1000);
return false;
}
/**
* 判断参数是否相同
*/
private boolean compareParams(Map<String, Object> nowMap, Map<String, Object> preMap)
{
String nowParams = (String) nowMap.get(REPEAT_PARAMS);
String preParams = (String) preMap.get(REPEAT_PARAMS);
return nowParams.equals(preParams);
}
/**
* 判断两次间隔时间
*/
private boolean compareTime(Map<String, Object> nowMap, Map<String, Object> preMap)
{
long time1 = (Long) nowMap.get(REPEAT_TIME);
long time2 = (Long) preMap.get(REPEAT_TIME);
if ((time1 - time2) < (this.intervalTime * 1000))
{
return true;
}
return false;
}
}
6.从流里读数据
/**
* 通用http工具封装
*
* @author ruoyi
*/
public class HttpHelper
{
private static final Logger LOGGER = LoggerFactory.getLogger(HttpHelper.class);
public static String getBodyString(ServletRequest request)
{
StringBuilder sb = new StringBuilder();
BufferedReader reader = null;
try (InputStream inputStream = request.getInputStream())
{
reader = new BufferedReader(new InputStreamReader(inputStream, Charset.forName("UTF-8")));
String line = "";
while ((line = reader.readLine()) != null)
{
sb.append(line);
}
}
catch (IOException e)
{
LOGGER.warn("getBodyString出现问题!");
}
finally
{
if (reader != null)
{
try
{
reader.close();
}
catch (IOException e)
{
// LOGGER.error(ExceptionUtils.getFullStackTrace(e));
}
}
}
return sb.toString();
}
}