elasticsearch

最新推荐文章于 2024-03-06 14:01:28 发布

qq_33808440

最新推荐文章于 2024-03-06 14:01:28 发布

阅读量212

点赞数

分类专栏： ES

本文链接：https://blog.csdn.net/qq_33808440/article/details/103461474

版权

ES 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

一. 基本概念

Node（节点）：安装ES的服务器
Cluster（集群）：一个或多个node组成的共同工作、分享数据、负载均衡的集群
Index（索引）：相似特征的文档的集合；相当于关系型数据库的库
Type（类型）：一个索引中可以定义一个多多个类型；相当于关系型数据库的表
Document（文档）：是一个可以被索引的基本信息单元；相当于关系型数据库的表中的一条数据
Field（列）：是ES的最小单位，相当于关系型数据库的字段
Shards（分片）：ES把一个索引分成若干份，每一份叫做一个分片，每个分片存放在不同node上，为了查询起来快
在这里插入图片描述
Replicas（复制）：是一个索引的一份或多份拷贝

1. shard和replica

一个index包含多个shard
每一个shard都是一个最小的工作单元，承载一部分数据，每一个shard都是一个lucene实例，都能完整的建立索引和处理请求
增减节点时shard会自动在node中移动做负载均衡
primary shard和replica shard：每个document只存在一个primary shard中，也存在这个primary shard的所有replica shard中；一个document不可能存在于多个primary shard中，也存在这个primary
replica shard是primary shard的副本，复杂容错，也承担查询请求
primary shard在创建索引时已经固定好了，之后不能修改；当replica shard的数量可以随时修改
创建索引时指定primary shard和replica shard个数的命令：
# 创建有7个primary shard和2个replicas shard的索引

curl -XPUT 'http://192.168.11.26:9200/zkwl' -d '{"settings":{"number_of_replicas":2,"number_of_shards":7}}' -H 'Content-Type:application/json'

primary shard不能和自己的replica shard放到同一个节点上，否则如果宕机，primary shard和副本都会丢失，起不到容错的作用；但可以和其他primary shard的副本放在一个节点上

二. ES安装、配置、log

配置文件：elasticsearch-7.2.1/config/elasticsearch.yml

三. 用curl命令操作ES数据

curl是用命令行方式向远程主机的http服务发送get、post请求，并返回响应。
curl常用参数：
-X：后接请求参数GET、POST等
-d: 向远程服务器发送数据，一般都是json格式的数据
-H：发送的请求头

1. ES集群操作

1.1 集群的增

1.2 集群的删

1.3 集群的改

1.4 集群的查

[root@slave122 config]# curl -XGET ‘http://192.168.11.28:9200/_cat/nodes?pretty&v’
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.11.28 58 65 1 0.26 0.37 0.37 mdi - ub-28
192.168.11.26 43 73 1 1.56 0.72 0.44 mdi - ub-26
192.168.11.27 39 62 1 0.27 0.26 0.29 mdi * ub-27

2. 索引操作

2.1 索引的增

# 新建一个叫new_index的索引
注意：如果只新建索引，不同时写document，只能用PUT方法，不能用POST方法

[root@slave122 config]# curl -XPUT 'http://192.168.11.28:9200/new_index?pretty'
{
  "acknowledged" : true,
  "shards_acknowledged" : true,
  "index" : "new_index"
}

# 新建一个叫test_index的索引，里面一个叫type1的type，并且添加一条document
注意：新建索引的同时写入document时只能用POST方法，不能用PUT方法

[root@slave122 config]# curl -XPOST 'http://192.168.11.28:9200/test_index/type1/1?pretty' -H 'Content-Type:application/json' -d '{"name":"andy","age":18,"work":"tester"}'
{
  "_index" : "test_index",
  "_type" : "type1",
  "_id" : "1",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 0,
  "_primary_term" : 1
}

2.2 索引的删

# 删除整个testindex索引

# curl -XDELETE 'http://192.168.11.26:9200/testindex/'

2.3 索引的改

2.4 索引的查

查看指定索引的详细信息，但不看具体document的内容

[root@bogon ~]# curl -XGET 'http://192.168.11.27:9200/testindex?pretty'
{
  "testindex" : {
    "aliases" : { },
    "mappings" : {
      "properties" : {
        "age" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        },
        "name" : {
          "type" : "text",
          "fields" : {
            "keyword" : {
              "type" : "keyword",
              "ignore_above" : 256
            }
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1575881963277",
        "number_of_shards" : "1",
        "number_of_replicas" : "1",
        "uuid" : "ZOFFaBDrRaaSyp6k8VSFPQ",
        "version" : {
          "created" : "7020099"
        },
        "provided_name" : "testindex"
      }
    }
  }
}

# 查看所有索引的信息
# 问号后面加额外参数
# 参数pretty为了显示好看
# 参数v为了显示filed名

[root@slave122 config]# curl -XGET 'http://192.168.11.28:9200/_cat/indices?pretty&v'
health status index                uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   event201912          DaQ3ryZhSpG006qrDxJ3AQ   5   1    1795136            0      2.6gb          1.3gb
green  open   .kibana_task_manager d855uxHeS0GM-rhk-ID96w   1   1          2            0     42.8kb         21.4kb
green  open   warn201912           t-LyMaS8QMSRfW2uH1jqYw   5   1      71435        14799     41.2mb         21.1mb
green  open   dblog201912          F4aDcrK0RwS3rqEvF6PM3g   5   1      33878            0     14.7mb          7.1mb

2.5 索引的打开和关闭

/index/_open

# curl -XPOST 'http://192.168.11.26:9200/new_index/_open'

/index/_close

# curl -XPOST 'http://192.168.11.26:9200/new_index/_close'

2.6 索引的刷新

/index/_refresh 刷新索引，使得新添加的内容对搜索可见，但不保证数据被写入磁盘
/index/_flush 刷新索引，会触发Lucene提交

3. 类型的操作

3.1 类型的增

3.2 类型的删

3.3 类型的改

3.4 类型的查

4. document操作

4.1 document的增

# 新建document时如果不指定id，会随机生成一个id (如果不指定id，只能用POST方法，不能用PUT方法)
# 自动生成的id是一个20个字符，URL安全，base64编码的，并且是GUID的（并行生成document时id不会冲突）

[root@slave122 config]# curl -XPOST 'http://192.168.11.28:9200/test_index/type1?pretty' -H 'Content-Type:application/json' -d '{"name":"andy","age":18,"work":"tester"}'
{
  "_index" : "test_index",
  "_type" : "type1",
  "_id" : "Jb71IW8BYa4AwTOQchiL",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 1,
  "_primary_term" : 1
}

# 新建document时如果指定id，即能用POST方法，又能用PUT方法

[root@bogon ~]# curl -XPUT 'http://192.168.11.26:9200/zkwa/product/11?pretty' -H 'Content-Type:application/json' -d '{"tage":"wo","name":"andy"}'
{
  "_index" : "zkwa",
  "_type" : "product",
  "_id" : "11",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 18,
  "_primary_term" : 1
}

# 用PUT或POST新建一个document时，如果id存在会全量替换，如果不想被替换，只想强制新建，如果id存在就报错，这时候要用_create

[root@bogon ~]# curl -XPUT 'http://192.168.11.26:9200/zkwa/product/1/_create?pretty' -H 'Content-Type:application/json' -d '{"tage":"wo","name":"andy"}'

4.2 document的删

删除某一个id的文件

curl -XDELETE 'http://192.168.11.27:9200/testindex/testtype/1?pretty'
{
  "_index" : "testindex",
  "_type" : "testtype",
  "_id" : "1",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 7,
  "_primary_term" : 1
}

2.清空某个type下所有doc

[root@localhost ~]# curl -XPOST 'http://192.168.11.26:9200/warn201911/_doc/_delete_by_query?conflicts=proceed&pretty' -H 'Content-Type:application/json' -d '{"query":{"match_all":{}}}'
{
  "took" : 164,
  "timed_out" : false,
  "total" : 20,
  "deleted" : 20,
  "batches" : 1,
  "version_conflicts" : 0,
  "noops" : 0,
  "retries" : {
    "bulk" : 0,
    "search" : 0
  },
  "throttled_millis" : 0,
  "requests_per_second" : -1.0,
  "throttled_until_millis" : 0,
  "failures" : [ ]
}

4.3 document的改

# 用PUT方法修改一个存在的document时是先删除，再建立一个同id的document

[root@slave122 config]# curl -XPUT 'http://192.168.11.27:9200/test_index/type1/1?pretty' -H 'Content-Type:application/json' -d'{"name1":"1","age1":'1'}'
{
  "_index" : "test_index",
  "_type" : "type1",
  "_id" : "1",
  "_version" : 2,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 2,
  "_primary_term" : 1
}

# 用POST方法加_update动作，并且内容要放在doc里面，这样是局部修改，只修改doc字典中的字段
[root@slave122 config]# curl -XPOST ‘http://192.168.11.27:9200/test_index/type1/1/_update?pretty’ -H ‘Content-Type:application/json’ -d’{“doc”:{“name2”:“andy”}}’
{
“_index” : “test_index”,
“_type” : “type1”,
“_id” : “1”,
“_version” : 4,
“result” : “updated”,
“_shards” : {
“total” : 2,
“successful” : 2,
“failed” : 0
},
“_seq_no” : 4,
“_primary_term” : 1
}

4.4 document的查

4.4.1 query string search

查看一个索引或索引/类型中的所有document

# curl -XGET 'http://192.168.11.27:9200/testindex/_search?pretty'

只查看name字段是fhq的document，并且按price字段的值降序排列

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?q=name:fhq&sort=price:desc&pretty'
{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 4,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "8",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 180
        },
        "sort" : [
          180
        ]
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "5",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 130
        },
        "sort" : [
          130
        ]
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "1",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 100
        },
        "sort" : [
          100
        ]
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "4",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 90
        },
        "sort" : [
          90
        ]
      }
    ]
  }
}

# 查name=tony & age=44

ub-26@ub-26:/opt$ curl -XGET 'http://localhost:9200/testindex/testtype/_search?q=name:tony&q=age:44&pretty'
{
  "took" : 922,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.3862944,
    "hits" : [
      {
        "_index" : "testindex",
        "_type" : "testtype",
        "_id" : "gusj6m4BX1toaSWY_bsN",
        "_score" : 1.3862944,
        "_source" : {
          "name" : "tony",
          "age" : "44"
        }
      }
    ]
  }
}

查询某个field不包含某个值的查询
q=-

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty&q=-tag:zkwa'

在所有field中某个匹配某个值的查询
q=值

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty&q=zkwa'

在这里插入图片描述
5. 根据document的id查询
例如：查索引testindex 类型testtype id为1的那条记录

curl -XGET 'http://localhost:9200/testindex/testtype/1?pretty'

结果：

{
  "_index" : "testindex",
  "_type" : "testtype",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "tony",
    "age" : "28"
  }
}

只查看document的各个字段，不看其他内容

[root@bogon ~]# curl -XGET 'http://192.168.11.27:9200/testindex/testtype/1/_source?pretty'
{
  "name" : "tony",
  "age" : "28"
}

只查看特定字段
如：只看name字段

[root@bogon ~]# curl -XGET 'http://192.168.11.27:9200/testindex/testtype/1/?_source=name&pretty'
{
  "_index" : "testindex",
  "_type" : "testtype",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "tony"
  }
}

如：查name字段和age字段

[root@bogon ~]# curl -XGET 'http://192.168.11.27:9200/testindex/testtype/1/?_source=name,age&pretty'
{
  "_index" : "testindex",
  "_type" : "testtype",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 2,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "name" : "tony",
    "age" : "28"
  }
}

查询name以外的字段

curl -XGET 'http://localhost:9200/testindex/testtype/1?_source_excludes=name,\&pretty'
{
  "_index" : "testindex",
  "_type" : "testtype",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "age" : "28"
  }
}

不显示_source中所有数据

ub-26@ub-26:/opt$ curl -XGET 'http://localhost:9200/testindex/testtype/1?_source=false\&pretty'
{
  "_index" : "testindex",
  "_type" : "testtype",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : { }
}
ub-26@ub-26:/opt$ curl -XGET 'http://localhost:9200/testindex/testtype/1?_source=false&pretty'
{
  "_index" : "testindex",
  "_type" : "testtype",
  "_id" : "1",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true
}

4.4.2 query DSL

搜索搜索\类型中的所有document

[root@bogon ~]# curl -H 'Content-Type:application/json'  -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -d '{"query":{"match_all":{}}}'
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 8,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "1",
        "_score" : 1.0,
        "_source" : {
          "name" : "fhq",
          "price" : 100
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "name" : "shenji",
          "price" : 110
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "name" : "anjian",
          "price" : 100
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "4",
        "_score" : 1.0,
        "_source" : {
          "name" : "fhq",
          "price" : 90
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "5",
        "_score" : 1.0,
        "_source" : {
          "name" : "fhq",
          "price" : 130
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "6",
        "_score" : 1.0,
        "_source" : {
          "name" : "anjian",
          "price" : 130
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "7",
        "_score" : 1.0,
        "_source" : {
          "name" : "shenji",
          "price" : 70
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "8",
        "_score" : 1.0,
        "_source" : {
          "name" : "fhq",
          "price" : 180
        }
      }
    ]
  }
}

值搜索name是fhq的document，并且降序排列结果

[root@bogon ~]# curl -H 'Content-Type:application/json'  -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -d '{
"query":{"match":{"name":"fhq"}},
> "sort":[
> {"price":"desc"}
> ]
> }'
{
  "took" : 4,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 4,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "8",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 180
        },
        "sort" : [
          180
        ]
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "5",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 130
        },
        "sort" : [
          130
        ]
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "1",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 100
        },
        "sort" : [
          100
        ]
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "4",
        "_score" : null,
        "_source" : {
          "name" : "fhq",
          "price" : 90
        },
        "sort" : [
          90
        ]
      }
    ]
  }
}

分页查询
"from":x,“size”:y
# 例如：第6个结果开始，看2个结果

[root@bogon ~]# curl -H 'Content-Type:application/json'  -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -d '{"query":{"match_all":{}},"from":5,"size":2}'
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 8,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "6",
        "_score" : 1.0,
        "_source" : {
          "name" : "anjian",
          "price" : 130
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "7",
        "_score" : 1.0,
        "_source" : {
          "name" : "shenji",
          "price" : 70
        }
      }
    ]
  }
}

指定只显示某些field
"_source":[field1,field2…]
# 例如：只显示price和name字段

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -H 'Content-Type:application/json' -d '{"query":{"match":{"name":"fhq"}},"_source":["price","name"]}'
{
  "took" : 5,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 4,
      "relation" : "eq"
    },
    "max_score" : 0.6931472,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "1",
        "_score" : 0.6931472,
        "_source" : {
          "price" : 100,
          "name" : "fhq"
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "4",
        "_score" : 0.6931472,
        "_source" : {
          "price" : 90,
          "name" : "fhq"
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "5",
        "_score" : 0.6931472,
        "_source" : {
          "price" : 130,
          "name" : "fhq"
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "8",
        "_score" : 0.6931472,
        "_source" : {
          "price" : 180,
          "name" : "fhq"
        }
      }
    ]
  }
}

4.4.3 query filter

用于复杂的、组合条件的搜索

搜索产品中name是fhq，price大于100的产品

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -H 'Content-Type:application/json' -d '{"query":{
> "bool":{
> 	"must":{
> 		"match":{
> 			"name":"fhq"
> }
> },
> 	"filter":{
> 		"range":{
> 			"price":{"gt":100}
> }}}}}
> '
{
  "took" : 7,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 0.6931472,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "5",
        "_score" : 0.6931472,
        "_source" : {
          "name" : "fhq",
          "price" : 130
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "8",
        "_score" : 0.6931472,
        "_source" : {
          "name" : "fhq",
          "price" : 180
        }
      }
    ]
  }
}

4.4.4 full-text search 全文检索

# 当搜索“wo zkwa pro"时会把命中"wo", “zkwa”,“pro”,"wo zkwa pro"的document都搜索出来，只是score分数不一样

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -H 'Content-Type:application/json' -d '{"query":{"match":{"tag":"wo zkwa pro"}}}'
{
  "took" : 597,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0071131,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "1",
        "_score" : 1.0071131,
        "_source" : {
          "tag" : "wo zkwa pro"
        }
      },
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "2",
        "_score" : 0.31473693,
        "_source" : {
          "tag" : "zkwa"
        }
      }
    ]
  }
}

4.4.5 phrase search 短语检索

# 与全文检索正好相反，全文检索会把输入的搜索词以空格拆分，逐个匹配检索；短语检索只会把整个搜索词不拆分整体匹配
match_phrase

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -H 'Content-Type:application/json' -d '{"query":{"match_phrase":{"tag":"wo zkwa pro"}}}'
{
  "took" : 10,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0071131,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "1",
        "_score" : 1.0071131,
        "_source" : {
          "tag" : "wo zkwa pro"
        }
      }
    ]
  }
}

4.4.6 highlight search 高亮搜索结果

# 高亮搜索，搜索出来的高亮词用<em>和</em>包围，在浏览器中就会高亮
"highlight":{“fields”:{“field名”:{}}}

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/product/_search?pretty' -H 'Content-Type:application/json' -d '{"query":{"match_phrase":{"tag":"wo zkwa pro"}},"highlight":{"fields":{"tag":{}}}}'
{
  "took" : 20,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0071131,
    "hits" : [
      {
        "_index" : "zkwa",
        "_type" : "product",
        "_id" : "1",
        "_score" : 1.0071131,
        "_source" : {
          "tag" : "wo zkwa pro"
        },
        "highlight" : {
          "tag" : [
            "<em>wo</em> <em>zkwa</em> <em>pro</em>"
          ]
        }
      }
    ]
  }
}

4.5 document的聚合查询

4.5.1 合并某个field，并计算这个field的不同值的document个数

# 用"aggs"分组聚合
# 按“etype”field分组聚合
# 如果不加"size":0 ,除了显示聚合后的count外，还显示具体各个document的数据

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/netlog201912/_search?pretty' -H 'Content-Type:application/json' -d '
{
"aggs":{
    "group_by_etype":{
        "terms":{"field":"etype"}
>         }
>       },
> "size":0}'
{
  "took" : 816,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "group_by_etype" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "ftp",
          "doc_count" : 3471878
        },
        {
          "key" : "http",
          "doc_count" : 304263
        },
        {
          "key" : "telnet",
          "doc_count" : 1
        }
      ]
    }
  }
}

4.5.2 有条件搜索并合并某个field，并计算这个field的不同值的document个数

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/netlog201912/_search?pretty' -H 'Content-Type:application/json' -d '
{
"aggs":{
    "group_by_etype":{
        "terms":{"field":"etype"}
        }
      },
"size":0,
"query":{
    "match":{"level":6}
>       }
>     }'
{
  "took" : 1029,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "group_by_etype" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "ftp",
          "doc_count" : 3471878
        },
        {
          "key" : "http",
          "doc_count" : 306326
        },
        {
          "key" : "telnet",
          "doc_count" : 1
        }
      ]
    }
  }
}

4.5.3 先分组，再计算每组某个field的平均值

GET /netlog201912/_search
{
  "size":0,
  "aggs":{
    "group_by_etype":{
      "terms":{"field":"etype"},
      "aggs":{
        "avg_level":{
          "avg":{"field":"price"}
        }
      }
    }
  }
}

4.6 查询超时时间的设置

ES的查询本身并没有超时时间，会一直查下去
可以人为的设置一个查询超时时间

4.7 多index和多type搜索模式

在这里插入图片描述

5. bulk批量操作

Bulk API可以在一次API调用中包含多个索引操作，例如更新索引，删除索引等,相当于批量操作
1、bulk相当于数据库里的bash操作。
2、引入批量操作bulk，提高工作效率，你想啊，一批一批添加与一条一条添加，谁快?
3、bulk API可以帮助我们同时执行多个请求
4、bulk文件的格式：
action：index/create/update/delete
metadata：_index,_type,_id
request body：_source (删除操作不需要加request body)
{ action: { metadata }}
{ request body }
5、bulk里为什么不支持get呢？
　　答：批量操作，里面放get操作，没啥用！所以，官方也不支持。
6、create（新建文档）：如果数据存在，使用create操作失败，会提示文档已经存在。
index（新建文档或全量替换文档）：文档不存在则新增；存在则全量替换。
update（局部修改文档）：
delete（删除文档）：
7、bulk一次最大处理多少数据量？
　　bulk会把将要处理的数据载入内存中，所以数据量是有限制的，最佳的数据量不是一个确定的数值，它取决于你的硬件，你的文档大小以及复杂性，你的索引以及搜索的负载。
一般建议是1000-5000个文档，如果你的文档很大，可以适当减少队列，大小建议是5-15MB，默认不能超过100M，可以在es的配置文件（即$ES_HOME下的config下的elasticsearch.yml）中。来修改这个值http.max_content_length: 100mb【不建议修改，太大的话bulk也会慢】。
8. 如果一个操作失败，不会影响其他操作
9. 每一个json不能换行
10. bulk操作一（url中不指明index和type）
(1). 在\root里，新建一文件，命名为request
输入如下内容：
{“index”:{"_index":“zhu”,"_type":“xiaozhu”,"_id":“1”}}
{“name”:“andy”,“age”:18}
{“index”:{"_index":“zhu”,"_type":“xiaozhu”,"_id":“2”}}
{“name”:“tom”,“age”:28}
{“index”:{"_index":“zhu”,"_type":“xiaozhu”,"_id":“1”}}
{“name”:“lili”,“age”:38}
（2). 执行下面命令

curl -PUT '192.168.11.27:9200/_bulk' --data-binary @\root\request -H 'Content-Type:application/json'
{"took":225,"errors":false,"items":[{"index":{"_index":"zhu","_type":"xiaozhu","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}},{"index":{"_index":"zhu","_type":"xiaozhu","_id":"2","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":1,"_primary_term":1,"status":201}},{"index":{"_index":"zhu","_type":"xiaozhu","_id":"1","_version":2,"result":"updated","_shards":{"total":2,"successfu

注释：四种action例子

{"create":{"_index":"zhu","_type":"xiaozhu","_id":"3"}}
{"name":"andy","age":18}
{"index":{"_index":"zhu","_type":"xiaozhu","_id":"2"}}
{"name":"tom","age":28}
{"index":{"_index":"zhu","_type":"xiaozhu","_id":"1"}}
{"name":"lili","age":38}
{"update":{"_index":"zhu","_type":"xiaozhu","_id":"1"}}
{"doc":{"name":"test","age":100}}
{"delete":{"_index":"zhu","_type":"xiaozhu","_id":"2"}}

bulk操作二 url中指明index
# bulk文件：

{"index":{"_type":"xiaozhu","_id":"1"}}
{"name":"andy","age":18}
{"index":{"_type":"xiaozhu","_id":"2"}}
{"name":"tom","age":28}
{"index":{"_type":"xiaozhu","_id":"3"}}
{"name":"lili","age":38}

# 命令：

[root@bogon ~]# curl -PUT '192.168.11.27:9200/my/_bulk?pretty' --data-binary @/root/test.json -H 'Content-Type:application/json'
{
  "took" : 414,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "my",
        "_type" : "xiaozhu",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 2,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "my",
        "_type" : "xiaozhu",
        "_id" : "2",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 2,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "my",
        "_type" : "xiaozhu",
        "_id" : "3",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 2,
          "failed" : 0
        },
        "_seq_no" : 2,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

bulk操作三 url中指明index和type
# bulk文件：

[root@bogon ~]# curl -PUT '192.168.11.27:9200/ni/ni/_bulk?pretty' --data-binary @/root/test.json -H 'Content-Type:application/json'
{
  "took" : 221,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "ni",
        "_type" : "ni",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "ni",
        "_type" : "ni",
        "_id" : "2",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "delete" : {
        "_index" : "ni",
        "_type" : "ni",
        "_id" : "3",
        "_version" : 1,
        "result" : "not_found",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 2,
        "_primary_term" : 1,
        "status" : 404
      }
    }
  ]
}

在命令行中直接输入数据

[root@bogon ~]# curl -XPOST 'http://192.168.11.27:9200/zkwa1/product1/_bulk?pretty' -H 'Content-Type:application/json' -d '{"create":{"_id":1}}
> {"name":"fhq","price":100}
> {"create":{"_id":2}}
> {"name":"shenji","price":110}
> '
{
  "took" : 239,
  "errors" : false,
  "items" : [
    {
      "create" : {
        "_index" : "zkwa1",
        "_type" : "product1",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "create" : {
        "_index" : "zkwa1",
        "_type" : "product1",
        "_id" : "2",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

6. mget批量查询

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/my/_mget?pretty' -d '{"docs":[{"_index":"my","_type":"xiaozhu","_id":2},{"_index":"my","_type":"xiaozhu","_id":1},{"_index":"my","_type":"xiaozhu","_id":3}]}' -H 'Content-Type:application/json'
{
  "docs" : [
    {
      "_index" : "my",
      "_type" : "xiaozhu",
      "_id" : "2",
      "_version" : 1,
      "_seq_no" : 1,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "tom",
        "age" : 28
      }
    },
    {
      "_index" : "my",
      "_type" : "xiaozhu",
      "_id" : "1",
      "_version" : 1,
      "_seq_no" : 0,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "andy",
        "age" : 18
      }
    },
    {
      "_index" : "my",
      "_type" : "xiaozhu",
      "_id" : "3",
      "_version" : 1,
      "_seq_no" : 2,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "lili",
        "age" : 38
      }
    }
  ]
}

# 如果需要查询的文档都在同一个index或type中，那么可以在url中指定index或type

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/my/xiaozhu/_mget?pretty' -d '{"docs":[{"_id":2},{"_id":1},{"_id":3}]}' -H 'Content-Type:application/json'
{
  "docs" : [
    {
      "_index" : "my",
      "_type" : "xiaozhu",
      "_id" : "2",
      "_version" : 1,
      "_seq_no" : 1,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "tom",
        "age" : 28
      }
    },
    {
      "_index" : "my",
      "_type" : "xiaozhu",
      "_id" : "1",
      "_version" : 1,
      "_seq_no" : 0,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "andy",
        "age" : 18
      }
    },
    {
      "_index" : "my",
      "_type" : "xiaozhu",
      "_id" : "3",
      "_version" : 1,
      "_seq_no" : 2,
      "_primary_term" : 1,
      "found" : true,
      "_source" : {
        "name" : "lili",
        "age" : 38
      }
    }
  ]
}

三. 全量更新中基于参数seq_no和primary_term的乐观锁并发控制

[root@bogon ~]# curl -XPOST 'http://192.168.11.26:9200/zkwa/product/1?pretty&if_seq_no=27&if_primary_term=2' -H 'Content-Type:application/json' -d '{"doc":{"name11222211":"andy"}}'
{
  "_index" : "zkwa",
  "_type" : "product",
  "_id" : "1",
  "_version" : 14,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "_seq_no" : 28,
  "_primary_term" : 2
}

四. partial更新中内置乐观锁并发控制

# 在partial更新时，ES内部自己执行乐观锁并发控制，但我们可以控制发生冲突时的重试次数
# 可以用retry_on_conflict来设置冲突时重试的次数
[root@bogon ~]# curl -XPOST ‘http://192.168.11.26:9200/zkwa/product/1/_update?pretty&retry_on_conflict=5’ -H ‘Content-Type:application/json’ -d ‘{“doc”:{“name11222211”:“andy”}}’
{
“_index” : “zkwa”,
“_type” : “product”,
“_id” : “1”,
“_version” : 15,
“result” : “noop”,
“_shards” : {
“total” : 0,
“successful” : 0,
“failed” : 0
}
}

五. mapping

往ES中直接插入document时，会自动建立索引和type，也会自动给type赋予一个mapping；
mapping是与type一一对应的
mapping有不同种类的mapping; mapping的作用是定义每个field的数据类型，以及如何分词等
ES中document的不同的field有不同的数据类型，比如text或date。不同的数据类型有的是exact value，有的是full text
exact value和full text在搜索时对field value的处理：
4.1 exact value在搜索时建立倒排索引时，分词时，是将整个值作为一个关键字建立到倒排索引中的
4.2 full text在搜索时建立倒排索引时，分词时，是会经历各种各样的处理的：分词、normaliztion（时态转换、同义词转换、大小写转换），处理后才建立倒排索引的
exact value和full text在搜索时对要搜索的关键字的处理
5.1 exact value会将要搜索的整个值作为一个关键字去倒排索引中搜索
5.2 full text会将要搜索的值，分词、normaliztion后再去倒排索引中搜索

1. 查询type的mapping

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/zkwa/_mapping/product?pretty&include_type_name=true'

2. 分词器

测试分词器
测试一种分词器(standard)如何将一个数据类型(text)的值(“I liked a dogs”)分词的

[root@bogon ~]# curl -XGET 'http://192.168.11.26:9200/_analyze?pretty' -H 'Content-Type:application/json' -d '{"analyzer":"standard","text":"I liked a dogs"}'
{
  "tokens" : [
    {
      "token" : "i",
      "start_offset" : 0,
      "end_offset" : 1,
      "type" : "<ALPHANUM>",
      "position" : 0
    },
    {
      "token" : "liked",
      "start_offset" : 2,
      "end_offset" : 7,
      "type" : "<ALPHANUM>",
      "position" : 1
    },
    {
      "token" : "a",
      "start_offset" : 8,
      "end_offset" : 9,
      "type" : "<ALPHANUM>",
      "position" : 2
    },
    {
      "token" : "dogs",
      "start_offset" : 10,
      "end_offset" : 14,
      "type" : "<ALPHANUM>",
      "position" : 3
    }
  ]
}

2.分词器的作用：切分词语、normalization
2. 几种内置分词器
在这里插入图片描述

qq_33808440

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
elasticsearch

新建索引为testindex 类型为testtype的索引，并插入一条id为1的记录curl -XPUT 'http://localhost:9200/testindex/testtype/1?pretty' -H 'Content-Type:application/json' -d '{"name":"tony","age":"28"}‘{ "_index" : "testinde...
复制链接

扫一扫

专栏目录