高并发下jmeter性能压测及性能提升解决方案（八）缓存库存+RocketMQ异步同步数据+一致性事务型消息

最新推荐文章于 2024-06-01 21:05:22 发布

xy294636185

最新推荐文章于 2024-06-01 21:05:22 发布

阅读量890

点赞数

分类专栏：后端开发（偏java向）经验交流文章标签： java

本文链接：https://blog.csdn.net/xy294636185/article/details/119332339

版权

后端开发（偏java向）经验交流专栏收录该内容

42 篇文章 3 订阅

订阅专栏

交易优化技术之缓存库存

jmeter压测：

在处理交易请求时，如果不做处理，进行jmeter压测。

填入http请求信息：

设置200线程请求20次，得到结果：

平均耗时500s，tps在300s左右。

使用top -H查看应用服务器资源消耗：

java进程cpu占用率已经70+%了。相比查询，交易带来的资源消耗是非常大的。

使用top -H查看数据库服务器资源消耗：

资源消耗几乎都在mysql上面，cpu占用率还好。

交易性能瓶颈：

交易验证完全依赖数据库：我们一般秒杀活动创建订单程序中，需要先查询下单商品是否存在，查询用户是否合法，查询活动是否适用当前商品，就需要发送3条查询sql。

库存行锁：然后我们再执行落单叫库存的操作（此为热点操作），对应删除时需要查询商品id进行删除加行锁。

后置处理逻辑：落单减库存之后就是真正的数据入库，会生成订单流水号并进行插入操作（插入订单信息+加销量）。

下单流程图：

通过上面分析，我们至少对数据库产生了6次的io操作，而且减库存操作需要行锁等待。

交易验证优化：

用户风控策略优化：策略缓存模型化

1.修改查询商品和用户登录，改为用Redis缓存：

    @Override
    public ItemModel getItemByIdInCache(Integer id) {
        ItemModel itemModel = (ItemModel)redisTemplate.opsForValue().get("item_validate_" + id);
        if(itemModel == null){
            itemModel=this.getItemById(id);
            redisTemplate.opsForValue().set("item_validate_"+id,itemModel);
            redisTemplate.expire("item_validate_"+id,10, TimeUnit.MINUTES);
        }
        return itemModel;
    }

    @Override
    public UserModel getUserByIdInCache(Integer id) {
        UserModel userModel = (UserModel) redisTemplate.opsForValue().get("user_validate_" + id);
        if(userModel == null){
            userModel = this.getUser(id);
            redisTemplate.opsForValue().set("user_validate_"+id,userModel);
            redisTemplate.expire("user_validate_"+id,10, TimeUnit.MINUTES);
        }
        return userModel;
    }

设置1000线程使用jmeter进行压测：

2.库存行锁优化

一、减库存语句分析，添加对应字段索引：

update item_stock set stock = stock - #{amount} where item_id = #{itemId} and stock >= #{amount}

数据库会在item_id = #{itemId}时加上行锁，前提条件是必须有索引，没有索引，则回去锁对应的表。附给item_stock字段加索引：alter table item_stock add unique index item_id_index(item_id)

活动校验策略优化：引入活动发布流程，模型缓存化。紧急下线能力。

一、方案一、将扣减库存行锁缓存化

（1）活动发布同步库存进缓存

    @RequestMapping(value = "/publishpromo",method = {RequestMethod.GET})
    @ResponseBody
    public CommonReturnType publishpromo(@RequestParam(name = "id")Integer id){
        promoService.publishPromo(id);
        return CommonReturnType.create(null);
    }

    @Override
    public void publishPromo(Integer promoId) {
        //通过活动Id获取活动
        PromoDO promoDO = promoDOMapper.selectByPrimaryKey(promoId);
        if(promoDO.getItemId() == null || promoDO.getItemId().intValue() == 0){
            return;
        }
        ItemModel itemModel = itemService.getItemById(promoDO.getItemId());
        //将库存同步到Redis内
        redisTemplate.opsForValue().set("promo_item_stock_"+itemModel.getId(),itemModel.getStock());
    }

（2）下单交易减缓存库存

    @Override
    @Transactional
    public boolean decreaseStock(Integer itemId, Integer amount) throws BusinessException {
        long result = redisTemplate.opsForValue().increment("promo_item_stock_"+itemId,amount.intValue()*-1); //改为从Redis中扣减库存
        if(result >= 0){
            return true;  //更新库存成功
        }
        return false;  //更新库存失败
    }

方案二、异步同步数据库

（1）活动发布同步库存进缓存

同上

（2）下单交易减缓存库存

同上

（3）异步消息扣减数据库内存库

使用异步消息队列rocketMQ中间件进行Redis更新异步同步到数据库。RocketMQ简介见下面。

异步同步数据库：库存数据库最终一致性保证

pom：
    <dependency>
      <groupId>org.apache.rocketmq</groupId>
      <artifactId>rocketmq-client</artifactId>
      <version>4.3.0</version>
    </dependency>

application.properties:

#设置Rocket连接
#注意：broker需要开放端口在19876以上
mq.nameserver.addr=localhost:9876
mp.topicname=stock

创建MQ文件夹以及生产者&消费者：

package com.miaoshaProject.mq;

import com.alibaba.fastjson.JSON;
import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.remoting.exception.RemotingException;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import java.nio.charset.Charset;
import java.util.HashMap;

/**
 * @author aric
 * @create 2021-08-09-10:29
 * @fun
 */
@Component  //代表spring的Bean，可以通过依赖关系注入进来
public class MqProducer {

    private DefaultMQProducer producer;

    @Value("${mq.nameserver.addr}")
    private String nameAddr;

    @Value("${mp.topicname}")
    private String topicName;

    @PostConstruct //在Bean的初始化完成之后被调用
    public void init() throws MQClientException {
        //做mq producer的初始化
        producer = new DefaultMQProducer("producer_group");
        producer.setNamesrvAddr(nameAddr);  //初始化地址
        producer.start();
    }

    //同步库存扣减消息
    public boolean asyncReducerStock(Integer itemId, Integer amount) {
        HashMap<Object, Object> bodyMap = new HashMap<>();
        bodyMap.put("itemId", itemId);
        bodyMap.put("amount", amount);
        Message message = new Message(topicName, "increase",
                JSON.toJSON(bodyMap).toString().getBytes(Charset.forName("UTF-8")));
        try {
            producer.send(message);
        } catch (MQClientException e) {
            e.printStackTrace();
            return false;
        } catch (RemotingException e) {
            e.printStackTrace();
            return false;
        } catch (MQBrokerException e) {
            e.printStackTrace();
            return false;
        } catch (InterruptedException e) {
            e.printStackTrace();
            return false;
        }
        return true;
    }
}

package com.miaoshaProject.mq;

import com.alibaba.fastjson.JSON;
import com.miaoshaProject.dao.ItemStockDOMapper;
import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.common.message.MessageExt;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import java.util.List;
import java.util.Map;

/**
 * @author aric
 * @create 2021-08-09-10:29
 * @fun
 */
@Component  //代表spring的Bean
public class MqConsumer {

    private DefaultMQPushConsumer consumer;

    @Value("${mq.nameserver.addr}")
    private String nameAddr;

    @Value("${mp.topicname}")
    private String topicName;

    @Autowired
    private ItemStockDOMapper itemStockDOMapper;

    @PostConstruct //在Bean的初始化完成之后被调用
    public void init() throws MQClientException {
        consumer = new DefaultMQPushConsumer("stock_consumer_group");
        consumer.setNamesrvAddr(nameAddr);
        consumer.subscribe(topicName,"*");  //指定订阅那个topic消息，订阅类型为所有
        //定义当有消息来时候怎么去处理
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            @Override
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext consumeConcurrentlyContext) {
                //实现库存真正到数据库内扣减的逻辑
                Message msg = msgs.get(0);
                String jsonString = new String(msg.getBody());
                Map<String,Object> map = JSON.parseObject(jsonString, Map.class);
                Integer itemId = (Integer)map.get("itemId");
                Integer amount = (Integer)map.get("amount");
                itemStockDOMapper.decreaseStock(itemId,amount);
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;  //默认成功，一旦有消息，rocketMQ就会认定被消费，不会在做对应投放
            }
        });
    }
}

修改下单时扣减库存逻辑：

    @Override
    @Transactional
    public boolean decreaseStock(Integer itemId, Integer amount) throws BusinessException {
        //使用select and update 实现会使用两条sql，但是这样只使用一条就可以达到原子操作
//        int affectedRow = itemStockDOMapper.decreaseStock(itemId, amount);
        long result = redisTemplate.opsForValue().increment("promo_item_stock_" + itemId, amount.intValue() * -1); //改为从Redis中扣减库存
        if (result >= 0) {
            //更新库存成功，发消息出去，让异步消息队列感知到，然后减数据库的库存
            boolean mqResult = mqProducer.asyncReducerStock(itemId, amount);
            if (!mqResult) {
                redisTemplate.opsForValue().increment("promo_item_stock_" + itemId, amount.intValue());
                return false;
            }
            return true;
        } else {
            //更新库存失败
            redisTemplate.opsForValue().increment("promo_item_stock_" + itemId, amount.intValue());
            return false;
        }
    }

问题：数据库记录不一致

1.异步消息发送失败

只能将对应的事务回滚，redis对应库存+1.

2.扣减操作执行失败

只能将对应的事务回滚，redis对应库存+1.

3.下单失败无法正确回补库存

见下文。

交易优化技术之事务性消息

分布式事务消息：

上述对应扣减库存的操作在外层大事务内若之后的操作发生错误进行回滚，但是将扣减库存操作放在Redis中，若扣减成功，异步消息也发送出去了，消费端也将消息扣减掉了，但是外层用户订单入库的时候产生了异常，返回给用户的结果肯定是落单失败，对应库存就损失掉了。不会发生超卖，但会少卖。商户会发现一段时间内库存少了，但是找不到对应的订单，仓库中就会积压货物，本质原因就在于分布式事务的问题。

改造：

将扣减消息模块拆分成异步发送消息单独开一个方法。

package com.miaoshaProject.service;

import com.miaoshaProject.error.BusinessException;
import com.miaoshaProject.service.model.ItemModel;

import java.util.List;

/**
 * @author aric
 * @create 2021-06-26-13:09
 * @fun
 */
public interface ItemService {

    //创建商品
    ItemModel createItem(ItemModel itemModel) throws BusinessException;

    //商品列表浏览
    List<ItemModel> listItem();

    //商品详情浏览
    ItemModel getItemById(Integer id);

    //验证item及promo缓存模型
    ItemModel getItemByIdInCache(Integer id);

    //库存扣减
    boolean decreaseStock(Integer itemId,Integer amount) throws BusinessException;

    //异步更新库存
    boolean asyncDecreaseStock(Integer itemId,Integer amount);

    //库存回滚
    boolean increaseStock(Integer itemId,Integer amount) throws BusinessException;

    //商品销量增加
    void increaseSales(Integer itemId,Integer amount);
}

@Service
public class ItemServiceImpl implements ItemService {

    @Override
    @Transactional
    public boolean decreaseStock(Integer itemId, Integer amount) throws BusinessException {
        //使用select and update 实现会使用两条sql，但是这样只使用一条就可以达到原子操作
//        int affectedRow = itemStockDOMapper.decreaseStock(itemId, amount);
        long result = redisTemplate.opsForValue().increment("promo_item_stock_" + itemId, amount.intValue() * -1); //改为从Redis中扣减库存
        if (result >= 0) {
            //更新库存成功，发消息出去，让异步消息队列感知到，然后减数据库的库存
//            boolean mqResult = mqProducer.asyncReducerStock(itemId, amount);
//            if (!mqResult) {
//                redisTemplate.opsForValue().increment("promo_item_stock_" + itemId, amount.intValue());
//                return false;
//            }
            return true;
        } else {
            //更新库存失败
            increaseSales(itemId, amount);
            return false;
        }
    }

    @Override
    public boolean asyncDecreaseStock(Integer itemId, Integer amount) {
        boolean mqResult = mqProducer.asyncReducerStock(itemId, amount);
        return mqResult;
    }

    @Override
    public boolean increaseStock(Integer itemId, Integer amount) throws BusinessException {
        redisTemplate.opsForValue().increment("promo_item_stock_" + itemId, amount.intValue());
        return true;
    }
}

@Service
public class OrderServiceImpl implements OrderService {

    @Override
    @Transactional
    public OrderModel createOrder(Integer userId, Integer itemId, Integer promoId, Integer amount) throws BusinessException {
        //1.检验下单状态，下单商品是否存在，用户是否合法，购买数量是否正确
//        ItemModel itemModel = itemService.getItemById(itemId);
        ItemModel itemModel = itemService.getItemByIdInCache(itemId);  //改为从Redis内存中获取，可以减少对数据库的依赖
        if (itemModel == null) {
            throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR, "商品信息不存在");
        }
//        UserModel userModel = userService.getUser(userId);
        UserModel userModel = userService.getUserByIdInCache(userId);
        if (userModel == null) {
            throw new BusinessException(EmBusinessError.USER_NOT_EXIST);
        }
        if (amount <= 0 || amount > 99) {
            throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR, "数量信息不正确");
        }
        //校验活动信息，promoId==null就是普通商品的下单
        if(promoId != null){
            //1.校验对应活动是否存在这个适用商品
            if(promoId.intValue() != itemModel.getPromoModel().getId()){
                throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR,"活动信息不正确");
            //2.校验活动是否进行中
            }else if(itemModel.getPromoModel().getStatus().intValue() != 2){
                throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR,"活动未开始");
            }
        }

        //2.落单减库存，支付减库存
        boolean result = itemService.decreaseStock(itemId, amount);
        if (!result) {
            throw new BusinessException(EmBusinessError.STOCK_NOT_ENOUGH);
        }
        //3.订单入库
        OrderModel orderModel = new OrderModel();
        orderModel.setUserId(userId);
        orderModel.setItemId(itemId);
        orderModel.setAmount(amount);
        if(promoId != null) {
            orderModel.setItemPrice(itemModel.getPromoModel().getPromoItemPrice());
        }else{
            orderModel.setItemPrice(itemModel.getPrice());
        }
        orderModel.setPromoId(promoId);
        orderModel.setOrderPrice(orderModel.getItemPrice().multiply(new BigDecimal(amount)));
        //生成交易流水号
        orderModel.setId(generateOrderNo());
        OrderDO orderDO = convertFromOrderModel(orderModel);
        orderDOMapper.insertSelective(orderDO);
        //加上商品的销量
        itemService.increaseSales(itemId,amount);

        //让异步消息滞后到下单成功（事务提交之后）后再发送
        TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
            @Override
            public void afterCommit() {
                //异步更新库存，就必须保证异步消息必须发送成功,一旦失败这条消息就永远丢失了，就需要RocketMQ的Transactional事务
                boolean mqResult = itemService.asyncDecreaseStock(itemId, amount);
//                if(!mqResult){
//                    itemService.increaseStock(itemId, amount);
//                    throw new BusinessException(EmBusinessError.MQ_SEND_FAIL);
//                }
            }
        });

        //4.返回前端
        return orderModel;
    }
}

事务型Producer：

由于之前发送异步消息事务提交之后，异步更新库存，就必须保证异步消息必须发送成功,一旦失败这条消息就永远丢失。所以就需要事务型Producer。

使用事务型Producer替换上述异步消息滞后发送：

注释相关代码：
//        //让异步消息滞后到下单成功（事务提交之后）后再发送
//        TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
//            @Override
//            public void afterCommit() {
//                //异步更新库存，就必须保证异步消息必须发送成功,一旦失败这条消息就永远丢失了，就需要RocketMQ的Transactional事务
//                boolean mqResult = itemService.asyncDecreaseStock(itemId, amount);
                if(!mqResult){
                    itemService.increaseStock(itemId, amount);
                    throw new BusinessException(EmBusinessError.MQ_SEND_FAIL);
                }
//            }
//        });

package com.miaoshaProject.controller;

import com.miaoshaProject.error.BusinessException;
import com.miaoshaProject.error.EmBusinessError;
import com.miaoshaProject.mq.MqProducer;
import com.miaoshaProject.response.CommonReturnType;
import com.miaoshaProject.service.OrderService;
import com.miaoshaProject.service.model.OrderModel;
import com.miaoshaProject.service.model.UserModel;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*;

import javax.servlet.http.HttpServletRequest;
import javax.validation.constraints.Email;

/**
 * @author aric
 * @create 2021-07-02-18:41
 * @fun
 */
@Controller("order")
@RequestMapping("/user")
@CrossOrigin(origins = {"*"},allowedHeaders = "true")
public class orderController extends BaseController{

    @Autowired
    private OrderService orderService;

    @Autowired
    private HttpServletRequest httpServletRequest;

    @Autowired
    private RedisTemplate redisTemplate;

    @Autowired
    private MqProducer mqProducer;

    //封装下单请求
    @RequestMapping(value = "/createorder",method = {RequestMethod.POST},consumes = {CONTENT_TYPE_FORMED})
    @ResponseBody
    public CommonReturnType createOrder(@RequestParam(name="itemId")Integer itemId,@RequestParam(name = "promoId",required = false) Integer promoId,@RequestParam(name = "amount")Integer amount) throws BusinessException {

        String token = httpServletRequest.getParameterMap().get("token")[0];  //也可以从参数中获取
        if(StringUtils.isEmpty(token)){
            throw new BusinessException(EmBusinessError.USER_NOT_LOGIN,"用户还未登录,不能下单");
        }
        //获取用户登录信息
        UserModel userModel = (UserModel)redisTemplate.opsForValue().get(token);
        if(userModel == null){  //以过期
            throw new BusinessException(EmBusinessError.USER_NOT_LOGIN,"用户还未登录,不能下单");
        }
        //调用事务型消息
        if(!mqProducer.transactionAsyncReduceStock(userModel.getId(), itemId, promoId, amount)){
            throw new BusinessException(EmBusinessError.UNKNOWN_ERROR,"下单失败");
        }
        return CommonReturnType.create(null);
    }
}

创建事务型消息：

发送事务型消息，事务型消息有二阶段提交的概念，这条消息发送后，Broker确实会收到消息，但是它的状态不是可被消费状态，而是准备状态，在准备状态下是不会被消费者看到的，他在准备状态下客户端会去执行executeLocalTransaction方法，也就是说这个消息做两件事：1.往消息队列里投递一个准备状态消息，被维护在broker中间件上面。2.在本地执行executeLocalTransaction，创建订单。

package com.miaoshaProject.mq;

import com.alibaba.fastjson.JSON;
import com.miaoshaProject.error.BusinessException;
import com.miaoshaProject.service.OrderService;
import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.*;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.common.message.MessageExt;
import org.apache.rocketmq.remoting.exception.RemotingException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.Map;

/**
 * @author aric
 * @create 2021-08-09-10:29
 * @fun
 */
@Component  //代表spring的Bean，可以通过依赖关系注入进来
public class MqProducer {

    private DefaultMQProducer producer;

    private TransactionMQProducer transactionMQProducer;

    @Value("${mq.nameserver.addr}")
    private String nameAddr;

    @Value("${mp.topicname}")
    private String topicName;

    @Autowired
    private OrderService orderService;

    @PostConstruct //在Bean的初始化完成之后被调用
    public void init() throws MQClientException {
        //做mq producer的初始化
        producer = new DefaultMQProducer("producer_group");
        producer.setNamesrvAddr(nameAddr);  //初始化地址
        producer.start();

        transactionMQProducer = new TransactionMQProducer("transaction_producer_group");
        transactionMQProducer.setNamesrvAddr(nameAddr);
        transactionMQProducer.start();

        transactionMQProducer.setTransactionListener(new TransactionListener() {
            @Override
            public LocalTransactionState executeLocalTransaction(Message message, Object arg) {
                //真正要做的事，创建订单
                Integer itemId = (Integer)((Map)arg).get("itemId");
                Integer promoId = (Integer)((Map)arg).get("promoId");
                Integer userId = (Integer)((Map)arg).get("userId");
                Integer amount = (Integer)((Map)arg).get("amount");
                try {
                    orderService.createOrder(userId,itemId,promoId,amount);
                } catch (BusinessException e) {
                    //发生异常就回滚
                    e.printStackTrace();
                    return LocalTransactionState.ROLLBACK_MESSAGE;
                }
                return LocalTransactionState.COMMIT_MESSAGE;
            }

            @Override
            public LocalTransactionState checkLocalTransaction(MessageExt msg) {
                //根据是否扣减库存成功，来判断要返回COMMIT，ROLLBACK还是继续UNKNOWN
                String jsonString = new String(msg.getBody());
                Map<String,Object> map = JSON.parseObject(jsonString, Map.class);
                Integer itemId = (Integer)map.get("itemId");
                Integer amount = (Integer)map.get("amount");
                return null;
            }
        });
    }

    //事务型同步库存扣减消息
    public boolean transactionAsyncReduceStock(Integer userId,Integer itemId,Integer promoId,Integer amount){
        HashMap<Object, Object> bodyMap = new HashMap<>();
        bodyMap.put("itemId", itemId);
        bodyMap.put("amount", amount);

        HashMap<Object, Object> argsMap = new HashMap<>();
        argsMap.put("itemId", itemId);
        argsMap.put("amount", amount);
        argsMap.put("userId", userId);
        argsMap.put("promoId", promoId);
        Message message = new Message(topicName, "increase",
                JSON.toJSON(bodyMap).toString().getBytes(Charset.forName("UTF-8")));
        TransactionSendResult sendResult = null;
        try {
            //发送事务型消息，事务型消息有二阶段提交的概念，这条消息发送后，Broker确实会收到消息，但是它的状态不是可悲消费状态，而是准备状态，在准备状态
            //下是不会被消费者看到的，他在准备状态下客户端会去执行executeLocalTransaction方法，也就是说这个消息做两件事：1.往消息队列里投递一个准备状
            //态消息，被维护在broker中间件上面。2.在本地执行executeLocalTransaction，创建订单。
            sendResult = transactionMQProducer.sendMessageInTransaction(message, argsMap);
        } catch (MQClientException e) {
            e.printStackTrace();
            return false;
        }
        if(sendResult.getLocalTransactionState() == LocalTransactionState.ROLLBACK_MESSAGE){
            return false;
        }else if(sendResult.getLocalTransactionState() == LocalTransactionState.COMMIT_MESSAGE) {
            return true;
        }else{
            return false;
        }
    }

    //同步库存扣减消息
    public boolean asyncReducerStock(Integer itemId, Integer amount) {
        HashMap<Object, Object> bodyMap = new HashMap<>();
        bodyMap.put("itemId", itemId);
        bodyMap.put("amount", amount);
        Message message = new Message(topicName, "increase",
                JSON.toJSON(bodyMap).toString().getBytes(Charset.forName("UTF-8")));
        try {
            producer.send(message);
        } catch (MQClientException e) {
            e.printStackTrace();
            return false;
        } catch (RemotingException e) {
            e.printStackTrace();
            return false;
        } catch (MQBrokerException e) {
            e.printStackTrace();
            return false;
        } catch (InterruptedException e) {
            e.printStackTrace();
            return false;
        }
        return true;
    }
}

操作流水：

当消息中间件发现对应准备消息一直没有被成功ROLLBACK_MESSAGE或者COMMIT_MESSAGE，就会发起checkLocalTransaction回调，来判断对应库存扣减和下单是否是成功的，但是仅凭itemId和amount不能判断对应的是哪一笔操作，因此需要引入库存流水。

创建对应的实体类：

public class StockLogDO {
    private String stockLogId;
    private Integer itemId;
    private Integer amount;
    private Integer status;// 1.表示初始状态2.表示下单扣减库存成功3.表示下单回滚
}

下单时加入流水号：

/**
 * @author aric
 * @create 2021-07-02-18:41
 * @fun
 */
public class orderController extends BaseController{

    @Autowired
    private OrderService orderService;

    @Autowired
    private HttpServletRequest httpServletRequest;

    @Autowired
    private RedisTemplate redisTemplate;

    @Autowired
    private MqProducer mqProducer;

    @Autowired
    private ItemService itemService;

    //封装下单请求
    @RequestMapping(value = "/createorder",method = {RequestMethod.POST},consumes = {CONTENT_TYPE_FORMED})
    @ResponseBody
    public CommonReturnType createOrder(@RequestParam(name="itemId")Integer itemId,@RequestParam(name = "promoId",required = false) Integer promoId,@RequestParam(name = "amount")Integer amount) throws BusinessException {

        String token = httpServletRequest.getParameterMap().get("token")[0];  //也可以从参数中获取
        if(StringUtils.isEmpty(token)){
            throw new BusinessException(EmBusinessError.USER_NOT_LOGIN,"用户还未登录,不能下单");
        }
        //获取用户登录信息
        UserModel userModel = (UserModel)redisTemplate.opsForValue().get(token);
        if(userModel == null){  //以过期
            throw new BusinessException(EmBusinessError.USER_NOT_LOGIN,"用户还未登录,不能下单");
        }

        //判断是否库存已经售罄，若对应的售罄key存在，则直接返回下单失败
        if(redisTemplate.hasKey("promo_item_stock_invalid_"+itemId)){
            throw new BusinessException(EmBusinessError.STOCK_NOT_ENOUGH);
        }
        //加入库存流水init状态
        String stockLogId = itemService.initStockLog(itemId,amount);

        //再去完成对应的下单事务型消息机制
        if(!mqProducer.transactionAsyncReduceStock(userModel.getId(), itemId, promoId, amount,stockLogId)){
            throw new BusinessException(EmBusinessError.UNKNOWN_ERROR,"下单失败");
        }
        return CommonReturnType.create(null);
    }
}

记录流水号入数据库：

    @Override
    @Transactional
    public String initStockLog(Integer itemId, Integer amount) {
        StockLogDO stockLogDO = new StockLogDO();
        stockLogDO.setItemId(itemId);
        stockLogDO.setAmount(amount);
        stockLogDO.setStockLogId(UUID.randomUUID().toString().replace("-",""));
        // 1.表示初始状态2.表示下单扣减库存成功3.表示下单回滚
        stockLogDO.setStatus(1);
        stockLogDOMapper.insertSelective(stockLogDO);
        return stockLogDO.getStockLogId();
    }

下单设置流水号实现：

@Service
public class OrderServiceImpl implements OrderService {
    @Override
    @Transactional
    public OrderModel createOrder(Integer userId, Integer itemId, Integer promoId, Integer amount,String stockLogId) throws BusinessException {
        //1.检验下单状态，下单商品是否存在，用户是否合法，购买数量是否正确
        ItemModel itemModel = itemService.getItemByIdInCache(itemId);  //改为从Redis内存中获取，可以减少对数据库的依赖
        if (itemModel == null) {
            throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR, "商品信息不存在");
        }

        UserModel userModel = userService.getUserByIdInCache(userId);
        if (userModel == null) {
            throw new BusinessException(EmBusinessError.USER_NOT_EXIST);
        }
        if (amount <= 0 || amount > 99) {
            throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR, "数量信息不正确");
        }
        //校验活动信息，promoId==null就是普通商品的下单
        if (promoId != null) {
            //1.校验对应活动是否存在这个适用商品
            if (promoId.intValue() != itemModel.getPromoModel().getId()) {
                throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR, "活动信息不正确");
                //2.校验活动是否进行中
            } else if (itemModel.getPromoModel().getStatus().intValue() != 2) {
                throw new BusinessException(EmBusinessError.PARAMETER_VALIDATION_ERROR, "活动未开始");
            }
        }

        //2.落单减库存，支付减库存
        boolean result = itemService.decreaseStock(itemId, amount);
        if (!result) {
            throw new BusinessException(EmBusinessError.STOCK_NOT_ENOUGH);
        }
        //3.订单入库
        OrderModel orderModel = new OrderModel();
        orderModel.setUserId(userId);
        orderModel.setItemId(itemId);
        orderModel.setAmount(amount);
        if (promoId != null) {
            orderModel.setItemPrice(itemModel.getPromoModel().getPromoItemPrice());
        } else {
            orderModel.setItemPrice(itemModel.getPrice());
        }
        orderModel.setPromoId(promoId);
        orderModel.setOrderPrice(orderModel.getItemPrice().multiply(new BigDecimal(amount)));
        //生成交易流水号
        orderModel.setId(generateOrderNo());
        OrderDO orderDO = convertFromOrderModel(orderModel);
        orderDOMapper.insertSelective(orderDO);
        //加上商品的销量
        itemService.increaseSales(itemId, amount);

        //设置库存流水状态为成功
        StockLogDO stockLogDO = stockLogDOMapper.selectByPrimaryKey(stockLogId);
        if(stockLogDO == null){
            throw new BusinessException(EmBusinessError.UNKNOWN_ERROR);
        }
        stockLogDO.setStatus(2);
        stockLogDOMapper.updateByPrimaryKeySelective(stockLogDO);

        //4.返回前端
        return orderModel;
    }
}

根据流水号判断订单状态（重要）：

package com.miaoshaProject.mq;

import com.alibaba.fastjson.JSON;
import com.miaoshaProject.dao.StockLogDOMapper;
import com.miaoshaProject.dataObject.StockLogDO;
import com.miaoshaProject.error.BusinessException;
import com.miaoshaProject.service.OrderService;
import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.*;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.common.message.MessageExt;
import org.apache.rocketmq.remoting.exception.RemotingException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import javax.annotation.PostConstruct;
import java.nio.charset.Charset;
import java.util.HashMap;
import java.util.Map;

/**
 * @author aric
 * @create 2021-08-09-10:29
 * @fun
 */
@Component  //代表spring的Bean，可以通过依赖关系注入进来
public class MqProducer {

    private DefaultMQProducer producer;

    private TransactionMQProducer transactionMQProducer;

    @Value("${mq.nameserver.addr}")
    private String nameAddr;

    @Value("${mp.topicname}")
    private String topicName;

    @Autowired
    private OrderService orderService;

    @Autowired
    private StockLogDOMapper stockLogDOMapper;

    @PostConstruct //在Bean的初始化完成之后被调用
    public void init() throws MQClientException {
        //做mq producer的初始化
        producer = new DefaultMQProducer("producer_group");
        producer.setNamesrvAddr(nameAddr);  //初始化地址
        producer.start();

        transactionMQProducer = new TransactionMQProducer("transaction_producer_group");
        transactionMQProducer.setNamesrvAddr(nameAddr);
        transactionMQProducer.start();

        transactionMQProducer.setTransactionListener(new TransactionListener() {
            @Override
            public LocalTransactionState executeLocalTransaction(Message message, Object arg) {
                //真正要做的事，创建订单
                Integer itemId = (Integer)((Map)arg).get("itemId");
                Integer promoId = (Integer)((Map)arg).get("promoId");
                Integer userId = (Integer)((Map)arg).get("userId");
                Integer amount = (Integer)((Map)arg).get("amount");
                String stockLogId = ((Map)arg).get("stockLogId").toString();
                try {
                    orderService.createOrder(userId,itemId,promoId,amount,stockLogId);
                } catch (BusinessException e) {
                    //发生异常就回滚
                    e.printStackTrace();
                    //设置对应的stockLog为回滚状态
                    StockLogDO stockLogDO = stockLogDOMapper.selectByPrimaryKey(stockLogId);
                    stockLogDO.setStatus(3);
                    stockLogDOMapper.updateByPrimaryKeySelective(stockLogDO);
                    return LocalTransactionState.ROLLBACK_MESSAGE;
                }
                return LocalTransactionState.COMMIT_MESSAGE;
            }

            //当消息中间件发现对应准备消息一直没有被成功ROLLBACK_MESSAGE或者COMMIT_MESSAGE，就会发起checkLocalTransaction回调，来判断对应库存扣减
            //和下单是否是成功的，但是仅凭itemId和amount不能判断对应的是哪一笔操作，因此需要引入库存流水
            @Override
            public LocalTransactionState checkLocalTransaction(MessageExt msg) {
                //根据是否扣减库存成功，来判断要返回COMMIT，ROLLBACK还是继续UNKNOWN
                String jsonString = new String(msg.getBody());
                Map<String,Object> map = JSON.parseObject(jsonString, Map.class);
                String stockLogId = map.get("stockLogId").toString();
                StockLogDO stockLogDO = stockLogDOMapper.selectByPrimaryKey(stockLogId);
                if(stockLogDO == null){
                    return LocalTransactionState.UNKNOW; //重试机制 1min-2min-4min。。。
                }
                if(stockLogDO.getStatus().intValue() == 2) {
                    return LocalTransactionState.COMMIT_MESSAGE;
                }else if(stockLogDO.getStatus().intValue() == 1){
                    return LocalTransactionState.UNKNOW;
                }
                return LocalTransactionState.ROLLBACK_MESSAGE;
            }
        });
    }

    //事务型同步库存扣减消息
    public boolean transactionAsyncReduceStock(Integer userId,Integer itemId,Integer promoId,Integer amount,String stockLogId){
        HashMap<Object, Object> bodyMap = new HashMap<>();
        bodyMap.put("itemId", itemId);
        bodyMap.put("amount", amount);
        bodyMap.put("stockLogId",stockLogId);

        HashMap<Object, Object> argsMap = new HashMap<>();
        argsMap.put("itemId", itemId);
        argsMap.put("amount", amount);
        argsMap.put("userId", userId);
        argsMap.put("promoId", promoId);
        argsMap.put("stockLogId",stockLogId);
        Message message = new Message(topicName, "increase",
                JSON.toJSON(bodyMap).toString().getBytes(Charset.forName("UTF-8")));
        TransactionSendResult sendResult = null;
        try {
            //发送事务型消息，事务型消息有二阶段提交的概念，这条消息发送后，Broker确实会收到消息，但是它的状态不是可被消费状态，而是准备状态，在准备状态
            //下是不会被消费者看到的，他在准备状态下客户端会去执行executeLocalTransaction方法，也就是说这个消息做两件事：1.往消息队列里投递一个准备状
            //态消息，被维护在broker中间件上面。2.在本地执行executeLocalTransaction，创建订单。
            sendResult = transactionMQProducer.sendMessageInTransaction(message, argsMap);
        } catch (MQClientException e) {
            e.printStackTrace();
            return false;
        }
        if(sendResult.getLocalTransactionState() == LocalTransactionState.ROLLBACK_MESSAGE){
            return false;
        }else if(sendResult.getLocalTransactionState() == LocalTransactionState.COMMIT_MESSAGE) {
            return true;
        }else{
            return false;
        }
    }

    //同步库存扣减消息
    public boolean asyncReducerStock(Integer itemId, Integer amount) {
        HashMap<Object, Object> bodyMap = new HashMap<>();
        bodyMap.put("itemId", itemId);
        bodyMap.put("amount", amount);
        Message message = new Message(topicName, "increase",
                JSON.toJSON(bodyMap).toString().getBytes(Charset.forName("UTF-8")));
        try {
            producer.send(message);
        } catch (MQClientException e) {
            e.printStackTrace();
            return false;
        } catch (RemotingException e) {
            e.printStackTrace();
            return false;
        } catch (MQBrokerException e) {
            e.printStackTrace();
            return false;
        } catch (InterruptedException e) {
            e.printStackTrace();
            return false;
        }
        return true;
    }
}

问题：

1.redis不可用时如何处理 2.扣减流水错误如何处理

业务场景决定高可用技术实现：

设计原则：宁可少卖，不能超卖。

方案：1.redis可以比实际数据库中少 2.超时释放（当下单超过15min以上，用户还没有操作，后台就需要将这个操作回滚释放掉）

库存售罄：

当抢购的商品只有几百件，瞬间的请求可能生成上亿条库存流水。所以需要修改对应的操作。

加入库存售罄标识。

售罄后不去操作后续流程。

售罄后通知各系统售罄。

回补上新。

修改更新库存逻辑：

    @Override
    @Transactional
    public boolean decreaseStock(Integer itemId, Integer amount) throws BusinessException {
        long result = redisTemplate.opsForValue().increment("promo_item_stock_" + itemId, amount.intValue() * -1); //改为从Redis中扣减库存
        if (result >= 0) {

            //更新库存成功
            return true;
        }else if(result == 0){
            //打上库存售罄的标识
            redisTemplate.opsForValue().set("promo_item_stock_invalid_"+itemId,"true");
            //更新库存成功
            return true;
        } else {
            //更新库存失败
            increaseSales(itemId, amount);
            return false;
        }
    }

在下单时候检查库存。

@Controller("order")
@RequestMapping("/user")
@CrossOrigin(origins = {"*"},allowedHeaders = "true")
public class orderController extends BaseController{
    //封装下单请求
    @RequestMapping(value = "/createorder",method = {RequestMethod.POST},consumes = {CONTENT_TYPE_FORMED})
    @ResponseBody
    public CommonReturnType createOrder(@RequestParam(name="itemId")Integer itemId,@RequestParam(name = "promoId",required = false) Integer promoId,@RequestParam(name = "amount")Integer amount) throws BusinessException {
        //获取用户的登录信息,之后替换从Redis读取
//        Boolean isLogin = (Boolean)httpServletRequest.getSession().getAttribute("IS_LOGIN");
        String token = httpServletRequest.getParameterMap().get("token")[0];  //也可以从参数中获取
        if(StringUtils.isEmpty(token)){
            throw new BusinessException(EmBusinessError.USER_NOT_LOGIN,"用户还未登录,不能下单");
        }
        //获取用户登录信息
        UserModel userModel = (UserModel)redisTemplate.opsForValue().get(token);
        if(userModel == null){  //以过期
            throw new BusinessException(EmBusinessError.USER_NOT_LOGIN,"用户还未登录,不能下单");
        }

        //判断是否库存已经售罄，若对应的售罄key存在，则直接返回下单失败
        if(redisTemplate.hasKey("promo_item_stock_invalid_"+itemId)){
            throw new BusinessException(EmBusinessError.STOCK_NOT_ENOUGH);
        }
        //加入库存流水init状态
        String stockLogId = itemService.initStockLog(itemId,amount);

        //再去完成对应的下单事务型消息机制
        if(!mqProducer.transactionAsyncReduceStock(userModel.getId(), itemId, promoId, amount,stockLogId)){
            throw new BusinessException(EmBusinessError.UNKNOWN_ERROR,"下单失败");
        }
        return CommonReturnType.create(null);
    }
}

附：

RocketMQ：

高性能，高并发，分布式消息中间件

典型应用场景：分布式事务，异步解耦

RocketMQ概念模型：

RocketMQ部署模型：

NameServer类似于Zookpeer，作为服务发现使用。所有的Peoducer，Broker，Consumer都需要去NamerServer上去做注册，Peoducer，Consumer是从NameServer上拉取对应的Broker信息，一个是为了SendMessage，一个是为了ReceiveMessge

RocketMQ架构图：

RocketMQ安装：

官网：rocketmq.apache.org/docs/quick-start/

创建rocketmq文件夹：

mkdir rocketmq

下载：

wget http://mirros.hust....../rocketmq-all-*.*.*-bin-release.zip //地址获取最新版

更改权限：

chmod -R 777 *

解压缩：

unzip rocketmq-all-*.*.*-bin-release.zip

启动NameServer：

sohup sh bin/mqnamesrv &

查看是否启动成功：

ps -ef | grep namesrv

netstat -anp | grep 9876

tail -f ~/logs/rocketmqlogs/namesrv.log

修改Broker.xml内存配置：

<-XX:NewSize>128M</-XX:NewSize>
<-XX:MaxNewSize>128M</-XX:MaxNewSize>
<-XX:PermSize>64M</-XX:PermSize>
<-XX:MaxPermSize>64M</-XX:MaxPermSize>

修改runbroker.sh内存配置：

JAVA_OPT="${JAVA_OPT} -server -Xms512m -Xmx512m -Xmn512m"

启动Broker

nohub sh bin/mqbroker - n localhost:9876 &

tail -f ~/logs/rocketmqlogs/broker.log

Send $ Receive Messages事例：

指定此NAMESRV的环境变量的地址：

export NAMESRV_ADDR=localhost:9876

设置Producer到broker内：

sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer

启动Consumer：

sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer

分布式事务：

xy294636185

关注

0
点赞
踩
2

收藏

觉得还不错? 一键收藏
0
评论
高并发下jmeter性能压测及性能提升解决方案（八）缓存库存+RocketMQ异步同步数据+一致性事务型消息

交易优化技术之缓存库存7-1
复制链接

扫一扫

专栏目录