ElasticSearch ( 三 ) 基本操作

春哥的魔法书

已于 2023-04-03 09:15:56 修改

阅读量733

点赞数

分类专栏： ElasticSearch 文章标签： elasticsearch java 大数据

于 2022-11-09 11:19:37 首次发布

本文链接：https://blog.csdn.net/yuanchun05/article/details/127766246

版权

ElasticSearch 专栏收录该内容

10 篇文章 2 订阅

订阅专栏

4.Elasticsearch操作

4.1.准备

Elasticsearch提供了多种交互使用方式，包括Java API和RESTful API ，本文主要介绍RESTful API 。所有其他语言可以使用RESTful API 通过端口 9200 和 Elasticsearch 进行通信，你可以用你最喜爱的 web 客户端访问 Elasticsearch 。甚至，你还可以使用 curl 命令来和 Elasticsearch 交互。
一个Elasticsearch请求和任何 HTTP 请求一样，都由若干相同的部件组成：

curl -X<VERB> '<PROTOCOL>://<HOST>:<PORT>/<PATH>?<QUERY_STRING>' -d '<BODY>'

返回的数据格式为JSON，因为Elasticsearch中的文档以JSON格式储存。其中，被 < > 标记的部件：

部件	说明
VERB	适当的 HTTP 方法或谓词 : `GET`、 `POST`、 `PUT`、 `HEAD` 或者 `DELETE`。
PROTOCOL	`http` 或者 `https`（如果你在 Elasticsearch 前面有一个 `https` 代理）
HOST	Elasticsearch 集群中任意节点的主机名，或者用 `localhost` 代表本地机器上的节点。
PORT	运行 Elasticsearch HTTP 服务的端口号，默认是 `9200` 。
PATH	API 的终端路径（例如 `_count` 将返回集群中文档数量）。Path 可能包含多个组件，例如：`_cluster/stats` 和 `_nodes/stats/jvm` 。
QUERY_STRING	任意可选的查询字符串参数 (例如 `?pretty` 将格式化地输出 JSON 返回值，使其更容易阅读)
BODY	一个 JSON 格式的请求体 (如果请求需要的话)

对于HTTP方法，它们的具体作用为：

HTTP方法	说明
GET	获取请求对象的当前状态
POST	改变对象的当前状态
PUT	创建一个对象
DELETE	销毁对象
HEAD	请求获取对象的基础信息

我们以下面的数据为例，来展示Elasticsearch的用法。

[
    {
        "storyTitle": "欢乐英雄",
        "storyAuthor": "古龙",
        "storyContentEn": "This is a happy hero's story",
        "storyContentCn": "这是一个欢乐英雄的故事",
        "storyConut": 712312423
    },
    {
        "storyTitle": "大漠欢乐英雄",
        "storyAuthor": "金龙",
        "storyContentEn": "There are some happy heroes in the desert",
        "storyContentCn": "大沙漠里有一些欢乐的英雄",
        "storyConut": 54678
    },
    {
        "storyTitle": "欢乐无敌英雄",
        "storyAuthor": "不是真龙",
        "storyContentEn": "Happy heroes are invincible",
        "storyContentCn": "欢乐的英雄是无敌的",
        "storyConut": 1098
    },
    {
        "storyTitle": "大漠英雄",
        "storyAuthor": "古龙",
        "storyContentEn": "Heroes living in the desert",
        "storyContentCn": "住在沙漠里的英雄",
        "storyConut": 12
    },
    {
        "storyTitle": "欢乐无敌",
        "storyAuthor": "真龙",
        "storyContentEn": "Invincible happy",
        "storyContentCn": "欢乐无敌",
        "storyConut": 3256884
    },
    {
        "storyTitle": "英雄欢乐行动",
        "storyAuthor": "不是真龙",
        "storyContentEn": "One code name is hero happy action",
        "storyContentCn": "一个代号是英雄欢乐的行动",
        "storyConut": 232553
    },
    {
        "storyTitle": "英雄故事",
        "storyAuthor": "不是真龙",
        "storyContentEn": "Hero story",
        "storyContentCn": "英雄故事",
        "storyConut": 1234
    }
]

以下全部的操作都在Kibana中完成，创建的index为testdata, type为story.

4.2.基础操作

4.2.1.插入数据 PUT (insert into)

PUT 插入数据必须带id

第一次插入是添加 , 相同id 第二次就是修改

首先创建index为 testdata , 创建type为 story , 插入id为1的第一条数据，只需运行下面命令就行：

PUT /testdata/story/1
    
    {
        "storyTitle": "欢乐英雄",
        "storyAuthor": "古龙",
        "storyContentEn": "This is a happy hero's story",
        "storyContentCn": "这是一个欢乐英雄的故事",
        "storyConut": 712312423
    }

在上面的命令中，路径 /testdata/story/1 表示文档的index为testdata, type为story, id为1.

{
    "_index": "testdata",
    "_type": "story",
    "_id": "1",
    "_version": 1,
    "result": "created",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 0,
    "_primary_term": 1
}

如果再执行一次上面的指令, id 不变

{
    "_index": "testdata",
    "_type": "story",
    "_id": "1",
    "_version": 2,
    "result": "updated",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 7,
    "_primary_term": 1
}

注意 result 属性由每一次执行时的 created 变成了 updated, 第一次是添加, 第二次代表修改

类似于上面的操作，依次插入剩余的6条数据

4.2.1.1.乐观锁避免并发修改

每次修改操作 _version , _seq_no 属性会自增长, 利用这个特点以乐观锁的形式可以避免乐观锁

在发出修改url 是通过 ?if_seq_no=1 的判断来实现

PUT /testdata/story/4?if_seq_no=1&if_primary_term=1

4.2.2.修改数据 POST

给定 id , 如果有对应数据就修改, 没有对应数据就添加数据

不给id 就添加新数据, 并随机生成 id

修改数据的命令为POST, 比如我们想要将 testdata 中 story 里面 id 为 4 的文档的 storyTitle 改为 “大漠英雄传奇”，

那么需要运行命令如下：

POST /testdata/story/4
    
{
    "storyTitle": "大漠英雄传奇",
    "storyAuthor": "古龙",
    "storyContentEn": "Heroes living in the desert",
    "storyContentCn": "住在沙漠里的英雄",
    "storyConut": 120
}

返回的信息如下：（表示修改数据成功）

{
    "_index": "testdata",
    "_type": "story",
    "_id": "4",
    "_version": 2,
    "result": "updated",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 8,
    "_primary_term": 2
}

如果url 带 _update , 数据中要有 “doc”:

这种更新先比较与原始数据是否有变化 , 如果相同, 就什么也不做, 只有不同时, 才更新

POST /testdata/story/4/_update
{
    "doc":{
        "storyTitle": "大漠英雄传奇",
        "storyAuthor": "古龙",
        "storyContentEn": "Heroes living in the desert",
        "storyContentCn": "住在沙漠里的英雄",
        "storyConut": 120
    }
}

4.2.3.查询数据 GET

查询数据的命令为GET，查询命令也是Elasticsearch最为重要的功能之一。比如我们想查询 testdata 中 story 里面 id 为1的数据，运行命令如下：

4.2.3.1.根据 id 查找一个

GET /testdata/story/1

返回的结果如下：

{
    "_index": "testdata",
    "_type": "story",
    "_id": "1",
    "_version": 1,
    "_seq_no": 0,
    "_primary_term": 1,
    "found": true,
    "_source": {
        "storyTitle": "欢乐英雄",
        "storyAuthor": "古龙",
        "storyContentEn": "This is a happy hero's story",
        "storyContentCn": "这是一个欢乐英雄的故事",
        "storyConut": 712312423
    }
}

4.2.3.2.全查

如果想要搜索 testdata 中 story 里面所有的文档，运行命令如下：

GET /testdata/story/_search

默认只显示 10 条, 如果显示更多,加参数

GET /testdata/story/_search
{
  "size": 50
}

返回结果包括了所有文档，放在数组 hits 中。

{
    "took": 158,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 7,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "testdata",
                "_type": "story",
                "_id": "1",
                "_score": 1.0,
                "_source": {
                    "storyTitle": "欢乐英雄",
                    "storyAuthor": "古龙",
                    "storyContentEn": "This is a happy hero's story",
                    "storyContentCn": "这是一个欢乐英雄的故事",
                    "storyConut": 712312423
                }
            },
            {
                "_index": "testdata",
                "_type": "story",
                "_id": "2",
                "_score": 1.0,
                "_source": {
                    "storyTitle": "大漠欢乐英雄",
                    "storyAuthor": "金龙",
                    "storyContentEn": "There are some happy heroes in the desert",
                    "storyContentCn": "大沙漠里有一些欢乐的英雄",
                    "storyConut": 54678
                }
            },
            {
                "_index": "testdata",
                "_type": "story",
                "_id": "3",
                "_score": 1.0,
                "_source": {
                    "storyTitle": "欢乐无敌英雄",
                    "storyAuthor": "不是真龙",
                    "storyContentEn": "Happy heroes are invincible",
                    "storyContentCn": "欢乐的英雄是无敌的",
                    "storyConut": 1098
                }
            },
            {
                "_index": "testdata",
                "_type": "story",
                "_id": "4",
                "_score": 1.0,
                "_source": {
                    "storyTitle": "大漠英雄",
                    "storyAuthor": "古龙",
                    "storyContentEn": "Heroes living in the desert",
                    "storyContentCn": "住在沙漠里的英雄",
                    "storyConut": 12
                }
            },
            {
                "_index": "testdata",
                "_type": "story",
                "_id": "5",
                "_score": 1.0,
                "_source": {
                    "storyTitle": "欢乐无敌",
                    "storyAuthor": "真龙",
                    "storyContentEn": "Invincible happy",
                    "storyContentCn": "欢乐无敌",
                    "storyConut": 3256884
                }
            },
            {
                "_index": "testdata",
                "_type": "story",
                "_id": "6",
                "_score": 1.0,
                "_source": {
                    "storyTitle": "英雄欢乐行动",
                    "storyAuthor": "不是真龙",
                    "storyContentEn": "One code name is hero happy action",
                    "storyContentCn": "一个代号是英雄欢乐的行动",
                    "storyConut": 232553
                }
            },
            {
                "_index": "testdata",
                "_type": "story",
                "_id": "7",
                "_score": 1.0,
                "_source": {
                    "storyTitle": "英雄故事",
                    "storyAuthor": "不是真龙",
                    "storyContentEn": "Hero story",
                    "storyContentCn": "英雄故事",
                    "storyConut": 1234
                }
            }
        ]
    }
}

4.2.4.删除数据

4.2.4.1.删除文档

比如我们想要删除 testdata 中 story 里面 id 为 7 的数据，只需运行下面命令即可：

DELETE /testdata/story/7

返回结果如下：

{
    "_index": "testdata",
    "_type": "story",
    "_id": "7",
    "_version": 2,
    "result": "deleted",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 15,
    "_primary_term": 4
}

这时再查询 id 为 7 的

{
    "_index": "testdata",
    "_type": "story",
    "_id": "7",
    "found": false
}

4.2.4.2.删除索引

如果想删除整个 testdata 索引，可输入命令:

DELETE /testdata

返回结果

{
    "acknowledged": true
}

再进行查询返回

{
  "error" : {
    "root_cause" : [
      {
        "type" : "index_not_found_exception",
        "reason" : "no such index [testdata]",
        "resource.type" : "index_or_alias",
        "resource.id" : "testdata",
        "index_uuid" : "_na_",
        "index" : "testdata"
      }
    ],
    "type" : "index_not_found_exception",
    "reason" : "no such index [testdata]",
    "resource.type" : "index_or_alias",
    "resource.id" : "testdata",
    "index_uuid" : "_na_",
    "index" : "testdata"
  },
  "status" : 404
}

4.3.批量操作 _bulk

4.3.1.批量添加

POST /testdata/story/_bulk
{"index":{"_id":"1"}}
{ "storyTitle": "欢乐英雄","storyAuthor": "古龙","storyContentEn": "This is a happy hero's story","storyContentCn": "这是一个欢乐英雄的故事","storyConut": 712312423}
{"index":{"_id":"2"}}
{ "storyTitle": "大漠欢乐英雄","storyAuthor": "金龙","storyContentEn": "There are some happy heroes in the desert","storyContentCn": "大沙漠里有一些欢乐的英雄","storyConut": 54678}

特别注意 , 文档信息要写成一行

返回信息

{
  "took" : 118,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "testdata",
        "_type" : "story",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "testdata",
        "_type" : "story",
        "_id" : "2",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

4.3.2.批量操作

POST /_bulk
{"delete":{"_index":"testdata","_type":"story","_id":"3"}}
{"create":{"_index":"testdata","_type":"story","_id":"3"}}
{ "storyTitle": "欢乐无敌英雄","storyAuthor": "不是真龙","storyContentEn": "Happy heroes are invincible","storyContentCn":"欢乐的英雄是无敌的","storyConut": 1098}
{"index":{"_index":"testdata","_type":"story"}}
{ "storyTitle": "大漠英雄","storyAuthor": "古龙","storyContentEn": "Heroes living in the desert","storyContentCn": "住在沙漠里的英雄","storyConut": 12}
{"update":{"_index":"testdata","_type":"story","_id":"3"}}
{"doc":{ "storyTitle": "欢乐无敌","storyAuthor": "真龙","storyContentEn": "Invincible happy","storyContentCn": "欢乐无敌","storyConut": 3256884}}

{
  "took" : 20,
  "errors" : false,
  "items" : [
    {
      "delete" : {
        "_index" : "testdata",
        "_type" : "story",
        "_id" : "3",
        "_version" : 1,
        "result" : "not_found",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 2,
        "_primary_term" : 2,
        "status" : 404
      }
    },
    {
      "create" : {
        "_index" : "testdata",
        "_type" : "story",
        "_id" : "3",
        "_version" : 2,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 3,
        "_primary_term" : 2,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "testdata",
        "_type" : "story",
        "_id" : "5LS_nHwBSVU8aHB0zRTN",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 4,
        "_primary_term" : 2,
        "status" : 201
      }
    },
    {
      "update" : {
        "_index" : "testdata",
        "_type" : "story",
        "_id" : "3",
        "_version" : 3,
        "result" : "updated",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 5,
        "_primary_term" : 2,
        "status" : 200
      }
    }
  ]
}

4.3.3.批量导入

官网下载测试数据
https://download.elastic.co/demos/kibana/gettingstarted/logs.jsonl.gz
https://download.elastic.co/demos/kibana/gettingstarted/accounts.zip
https://download.elastic.co/demos/kibana/gettingstarted/shakespeare_6.0.json