es批量修改数组动态增加并去重

最新推荐文章于 2024-07-03 22:18:00 发布

endless possibility

最新推荐文章于 2024-07-03 22:18:00 发布

阅读量944

点赞数

文章标签： elasticsearch es 批量修改数组去重

本文链接：https://blog.csdn.net/qielanyu_/article/details/116739763

版权

es创建测试的index和type(对应mysql的数据库和表)

PUT /uniq_test_idx1
{
  "mappings": {
    "uniq_test": {
      "properties": {
        "comId": {
          "type": "long"
        },
        "customerName": {
          "type": "keyword",
          "index": false
        }
      }
    }
  }
}

测试详情：

我会准备两条数据，comId和customerName均不一致。当我执行了批量修改后，要达到的效果，只入一条数据，comId为两条数据集合，customerName以后面的数据为准。

java中的测试代码如下：

@Data
    private static class SimpleEsModel {
        private String customerPhone;
        private String customerName;
        private Long[] comId;

        public SimpleEsModel(String customerPhone, String customerName, Long[] comId) {
            this.customerPhone = customerPhone;
            this.customerName = customerName;
            this.comId = comId;
        }
    }

    @Test
    public void test1() {
        List<SimpleEsModel> clueList = Lists.newArrayList();
        SimpleEsModel modal1 = new SimpleEsModel("133", "wang1", new Long[]{12L});
        SimpleEsModel modal2 = new SimpleEsModel("133", "wang2", new Long[]{13L});
        clueList.add(modal1);
        clueList.add(modal2);
        EsResponse esResponse = bulkUpsert(clueList);
        System.out.println(JacksonUtil.serialize(esResponse));
    }

    public EsResponse bulkUpsert(List<SimpleEsModel> vars) {
        if (CollectionUtils.isEmpty(vars)) {
            return EsResponse.success();
        }
        long start = System.currentTimeMillis();
        try {
            BulkRequest bulkRequest = new BulkRequest();
            for (SimpleEsModel var : vars) {
                String customerPhone = var.getCustomerPhone();

                String index = "uniq_test_idx1";
                String type = "uniq_test";
                Script script = generateScript(var);
                
                UpdateRequest updateRequest = new UpdateRequest(index, type, customerPhone)
                        .script(script)
                        .upsert(BeanMap.create(var));
                bulkRequest.add(updateRequest);
            }
            BulkResponse bulkResponse = esOperationService.bulk(bulkRequest);
            if (bulkResponse.hasFailures()) {
                String failureMessage = bulkResponse.buildFailureMessage();
                FkMonitor.recordOne("datacenter_es_bulkIndex_exception");
                return EsResponse.fail();
            }
            return EsResponse.success();
        } catch (Exception e) {
            return EsResponse.fail();
        } finally {
            FkMonitor.recordOne("datacenter_es_bulkIndex_total", System.currentTimeMillis() - start);
        }
    }

    public Script generateScript(SimpleEsModel simpleModel) {
        String scriptStr = "def comIdTargets = ctx._source.comId.findAll(oneComId -> oneComId == params.comId);\n" +
                "if(comIdTargets.size() == 0){\n" +
                "   ctx._source.comId.add(params.comId[0])\n" +
                "}" +
                "if(params.customerName!=null){ctx._source.customerName=params.customerName}\n";
        return new Script(ScriptType.INLINE, "painless", scriptStr, BeanMap.create(simpleModel));
    }

这段代码的核心是脚本的使用。

动态添加去重后comId的实现关键在于：传入的参数的comId类型为数组。

至于脚本使用的espainless脚本，关于这个脚本的语法规则我没有在网上找到过多的资料。

如果您有相关的资料，欢迎留言