Elasticsearch(037)：es中批量操作之bulk

一、bulk概述

1.1 bulk的基础概念

bulk是es提供的一种批量增删改的操作API。

1.2 bulk的语法

bulk对JSON串的有着严格的要求。每个JSON串不能换行，只能放在同一行，同时，相邻的JSON串之间必须要有换行（Linux下是\n；Window下是\r\n）。bulk的每个操作必须要一对JSON串（delete语法除外）。

{ action: { metadata }}
{ request body        }
{ action: { metadata }}
{ request body        }

例如，加入现在要给example的docs中新增一个文档。其表示如下：

POST _bulk
{"create": {"_index": "example", "_type": "docs", "_id": 11}}
{"name": "test_bulk", "counter":"100"}


#查询example所有数据,发现id为11的已经添加成功
GET example/docs/_search
{
  "query": {
    "match_all": {}
  }
}

1.3 bulk的操作类型

• create 如果文档不存在就创建，但如果文档存在就返回错误
• index 如果文档不存在就创建，如果文档存在就更新
• update 更新一个文档，如果文档不存在就返回错误
• delete 删除一个文档，如果要删除的文档id不存在，就返回错误

其实可以看得出来index是比较常用的。还有bulk的操作，某一个操作失败，是不会影响其他文档的操作的，它会在返回结果中告诉你失败的详细的原因。

二、bulk常用示例

2.1 测试数据准备

索引及映射结构准备

PUT example
PUT example/docs/_mapping
{
  "properties": {
    "id": {"type": "long"},
    "name": {"type": "text"},
    "counter": {"type": "integer"},
    "tags": {"type": "text"}
  }
}

2.1 批量插入

插入的action为index。

示例语法

POST example/docs/_bulk
{"index": {"_id": 1}}
{"id":1, "name": "admin", "counter":"10", "tags":["red", "black"]}
{"index": {"_id": 2}}
{"id":2, "name": "张三", "counter":"20", "tags":["green", "purple"]}
{"index": {"_id": 3}}
{"id":3, "name": "李四", "counter":"30", "tags":["red", "blue"]}
{"index": {"_id": 4}}
{"id":4, "name": "tom", "counter":"40", "tags":["orange"]}

返回结果

从下面的结果可以看出，四个文档均新建成功。

{
  "took": 18,
  "errors": false,
  "items": [
    {
      "index": {
        "_index": "example",
        "_type": "docs",
        "_id": "1",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "example",
        "_type": "docs",
        "_id": "2",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "example",
        "_type": "docs",
        "_id": "3",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    },
    {
      "index": {
        "_index": "example",
        "_type": "docs",
        "_id": "4",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 2,
          "failed": 0
        },
        "_seq_no": 1,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}

2.2 批量修改

修改的action为update。

示例语法

POST example/docs/_bulk
{"update": {"_id": 1}}
{"doc": {"id":1, "name": "admin-02", "counter":"11"}}
{"update": {"_id": 2}}
{"script":{"lang":"painless","source":"ctx._source.counter += params.num","params": {"num":2}}}
{"update":{"_id": 3}}
{"doc": {"name": "test3333name", "counter": 999}}
{"update":{"_id": 4}}
{"doc": {"name": "test444name", "counter": 888},  "doc_as_upsert" : true}

返回结果

{
  "took": 1,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 4,
    "max_score": 1,
    "hits": [
      {
        "_index": "example",
        "_type": "docs",
        "_id": "2",
        "_score": 1,
        "_source": {
          "id": 2,
          "name": "张三",
          "counter": "202",
          "tags": [
            "green",
            "purple"
          ]
        }
      },
      {
        "_index": "example",
        "_type": "docs",
        "_id": "4",
        "_score": 1,
        "_source": {
          "id": 4,
          "name": "test444name",
          "counter": 888,
          "tags": [
            "orange"
          ]
        }
      },
      {
        "_index": "example",
        "_type": "docs",
        "_id": "1",
        "_score": 1,
        "_source": {
          "id": 1,
          "name": "admin-02",
          "counter": "11",
          "tags": [
            "red",
            "black"
          ]
        }
      },
      {
        "_index": "example",
        "_type": "docs",
        "_id": "3",
        "_score": 1,
        "_source": {
          "id": 3,
          "name": "test3333name",
          "counter": 999,
          "tags": [
            "red",
            "blue"
          ]
        }
      }
    ]
  }
}

小贴士

由上面示例我们可以看出，批量更新支持参数选项：doc（部分文档），upsert，doc_as_upsert，脚本，``params（用于脚本），lang（用于脚本）和_source。

2.3 批量删除

修改的action为delete。

示例如下。

POST example/docs/_bulk
{"delete": {"_id": 1}}
{"delete": {"_id": 2}}
{"delete": {"_id": 3}}
{"delete": {"_id": 4}}

2.4 批量的混合操作

不过一般不推荐这种使用，项目中也用的极少。

POST _bulk
{ "index" : { "_index" : "example", "_type" : "docs", "_id" : "1" } }
{ "name" : "value11111" }
{ "delete" : { "_index" : "example", "_type" : "docs", "_id" : "2" } }
{ "create" : { "_index" : "example", "_type" : "docs", "_id" : "3" } }
{ "tags" : "value333" }
{ "update" : {"_id" : "4", "_type" : "docs", "_index" : "example"} }
{ "doc" : {"name" : "混合444"}