最近的项目里要用到上传Excel,并且上传的Excel文件也很大,足足有27Mb,有大概27 w 的数据量,不过解析是放在前端了,虽然这样用户也需要比较久的时间用来上传。
但是短时间内,我能够想到的就是这样对于数据库的压力会比较大,所以我想到的就是对于数据库的垂直拆分(但是技术老大说是类微服务,我不是很懂,因为感觉这不是一会儿事唉),将上传的数据单独保存在一张表里,并且将这一张表放在另外一个数据库。(说到现在我还是感觉是数据库的垂直拆分。。)
问题来了,而我就是要写这样的一个方法:
@Transactional public void importer(String client, Transporter transporter, List<ShipmentAmendVM> reconciliations) { if (client.equals("client")) { List<Reconciliation> reconciliationList = new ArrayList<>(); for (ShipmentAmendVM shipmentAmendVM : reconciliations) { shipmentAmendVM.setId(StringUtils.trimAllWhitespace(shipmentAmendVM.getId())); Reconciliation reconciliation = new Reconciliation(); reconciliation.setId(shipmentAmendVM.getId()); reconciliation.setExtraFee(shipmentAmendVM.getExtraFee()); reconciliation.setWeight(shipmentAmendVM.getWeight()); reconciliation.setCost(shipmentAmendVM.getCost()); reconciliation.setReason(shipmentAmendVM.getReason()); reconciliation.setClient(client); reconciliation.setTransporter(transporter); reconciliation.setAmount(shipmentAmendVM.getAmount());shipmentRepository.findById(shipmentAmendVM.getId()).ifPresent(shipment -> { reconciliation.setDeclaredPrice(shipment.getPrice()); reconciliation.setDeclaredWeight(shipment.getWeight()); shipmentRepository.updateStatus(ShipmentStatus.CHECKED, shipmentAmendVM.getId()); });// 下面的是我之前写的,性能真的烂的一逼。。。
/*if (shipmentRepository.findById(shipmentAmendVM.getId()).isPresent()) { Shipment shipment = shipmentRepository.findById(shipmentAmendVM.getId()).get(); reconciliation.setDeclaredPrice(shipment.getPrice()); reconciliation.setDeclaredWeight(shipment.getWeight()); shipmentRepository.updateStatus(ShipmentStatus.CHECKED, shipmentAmendVM.getId()); }*/ reconciliationList.add(reconciliation); } reconciliationRepository.saveAll(reconciliationList); }}
因为我会重复查两次数据库,而且是在27W的数据量下。老大帮我改了一下,这个时候感觉还是lambda 很好用,以后应该要多学习一下了。