Elasticsearch 操作总结

目录

1 安装部署

2 API操作

2.1 更新操作

2.2 批量操作

2.3 _upsert 创建或者更新操作

2.4 批量_upsert操作

2.5 聚合操作

2.6查询操作

3 问题解决

3.1 index readonly问题

---

1 安装部署

1.报错 max file descriptors [4096] for elasticsearch process is too low,increase to at least [65536]

vim /etc/security/limits.conf

es         hard    nofile   65536
es         soft    nofile   65536
es         soft    nproc    4096
es         hard    nproc    4096

es         soft    memlock  unlimited
es         hard    memlock  unlimited

2.max virtual memory areas vm.max_map_count [65530] is too low,increase to at least [262144]

vim /etc/sysctl.conf

vm.max_map_count=655360

sysctl -p

3. 该项错误已经包含到（1中）memory locking requested for elasticsearch process but memory is not locked

echo "es hard memlock unlimited">>/etc/security/limits.conf
echo "es soft memlock unlimited">>/etc/security/limits.conf

4.依然报错

关闭当前shell窗口，重新打开shell窗口

2 API操作

2.1 更新操作

update 请求最简单的一种形式是接收文档的一部分作为 doc 的参数， 它只是与现有的文档进行合并。对象被合并到一起，覆盖现有的字段，增加新的字段。

POST /website/blog/1/
{
  "title": "My first blog entry",
  "text":  "Just trying this out..."
}

例如，我们增加字段 tags 和 views 到我们的博客文章，如下所示：

POST /website/blog/1/_update
{
   "doc" : {
      "tags" : [ "testing" ],
      "views": 0
   }
}

2.2 批量操作

action 必须是以下选项之一:

create:如果文档不存在，那么就创建它。

index:创建一个新文档或者替换一个现有的文档。

update:部分更新一个文档。

delete:删除一个文档。

metadata 应该指定被索引、创建、更新或者删除的文档的 _index 、 _type 和 _id 。

完整批量更新示例

POST /_bulk
{ "delete": { "_index": "website", "_type": "blog", "_id": "123" }} 
{ "create": { "_index": "website", "_type": "blog", "_id": "123" }}
{ "title":    "My first blog post" }
{ "index":  { "_index": "website", "_type": "blog" }}
{ "title":    "My second blog post" }
{ "update": { "_index": "website", "_type": "blog", "_id": "123", "_retry_on_conflict" : 3} }
{ "doc" : {"title" : "My updated blog post"} } 

返回结果示例

{
   "took": 4,
   "errors": false, 
   "items": [
      {  "delete": {
            "_index":   "website",
            "_type":    "blog",
            "_id":      "123",
            "_version": 2,
            "status":   200,
            "found":    true
      }},
      {  "create": {
            "_index":   "website",
            "_type":    "blog",
            "_id":      "123",
            "_version": 3,
            "status":   201
      }},
      {  "create": {
            "_index":   "website",
            "_type":    "blog",
            "_id":      "EiwfApScQiiy7TIKFxRCTw",
            "_version": 1,
            "status":   201
      }},
      {  "update": {
            "_index":   "website",
            "_type":    "blog",
            "_id":      "123",
            "_version": 4,
            "status":   200
      }}
   ]
}

2.3 _upsert 创建或者更新操作

如果文档存在就使用doc更新（如果更新时，字段不存在，就会创建字段），如果文档不存在就用upsert初始化

XPOST 'localhost:9200/test/type1/1/_update' -d '{
    "doc" : {
        "name" : "new_name"
    },
    "upsert" : {
        "counter" : 1
    }
}'

2.4 批量_upsert操作

POST http://172.16.78.18:9200/node_31/_bulk

{ "update": { "_index": "node_31", "_type": "node", "_id": "a83a1e1f0a567f694babc83d1560d52a1"}}
{ "doc" : {"key1" : "aaa","key2":"bbb"}, "doc_as_upsert" : true }

2.5 聚合操作

cardinality 去重查询

GET /company/_search?search_type=count

{
    "size" : 0,
    "aggs" : {
        "distinct_w" : {
            "cardinality" : {
              "field" : "companyUid"
            }
        }
    }
}

date_histogram 时间聚合查询

//聚合单位 year, quarter, month, week, day, hour, minute, second

GET /cars/transactions/_search?search_type=count

{
  "size": 0,
    "query": {
        "bool": {
            "must": [{
                "range": {
                    "sendTime": {
                        "gt": "{}",
                        "lt": "{}"
                    }
                }
            }]
        }
    },	
  "aggregations": {
    "group_message_time": {
      "date_histogram": {
        "field": "send_time",
        "interval": "hour",
        "format": "yyyy-MM-dd HH:mm:ss",
        "min_doc_count" : 0
      }
    }
  }
}

双层聚合查询

{
	"size": 0,
	"aggregations": {
		"appID": {
			"terms": {
				"field": "appId",
				"size": 100
			},
			"aggregations": {
				"memberuId": {
					"terms": {
						"field": "spokesmanUid",
						"size": 100
					}
				}
			}
		}
	}
}

查询不返回文档

 GET /_search
{
    "_source": false,
    "query" : {
        "term" : { "user" : "kimchy" }
    }
}

查询返回指定字段

GET /_search
{
    "_source": [ "obj1.*", "obj2.*" ],
    "query" : {
        "term" : { "user" : "kimchy" }
    }
}

字段不匹配某些值

GET order/list/_search
{
  "size": 10, 
  "query": {
    "bool": {
      "must_not": [
        {
          "bool": {
            "should": [
              {
                "terms": {
                  "order": [
                      "iphone","","iphone4","iphone5","iphone5c"
                  ]
                }
              }
            ]
          }
        }
      ],
      "must": [
        {
          "range": {
            "date": {
              "gte": "2014-01-02",
              "lte": "2015-12-01"
            }
          }
        }
      ]
   }
  },
  "sort": [
    {
      "date": {
        "order": "desc"
      }
    }
  ]
}

2.6查询操作

POST /my_test1/my_doc/_msearch
{}
{"query":{"bool":{"must":[{"match":{"contentMd5":"b0c4c49a2deb60df5d86c51815507a26"}}]}},"from":0,"size":1,"sort":[{"sendTime":{"order":"desc"}}]}
{}
{"query":{"bool":{"must":[{"match":{"contentMd5":"b0c4c49a2deb60df5d86c51815507a26"}}]}},"from":0,"size":1,"sort":[{"sendTime":{"order":"asc"}}]}
{}
{"size":0,"query":{"bool":{"must":[{"match":{"contentMd5":"b0c4c49a2deb60df5d86c51815507a26"}}]}},"aggs":{"distinct":{"cardinality":{"field":"gUid"}}}}
{}
{"size":0,"query":{"bool":{"must":[{"match":{"contentMd5":"b0c4c49a2deb60df5d86c51815507a26"}}]}},"aggs":{"distinct":{"cardinality":{"field":"manUid"}}}}

3 问题解决

3.1 index readonly问题

PUT _settings
{
    "index": {
       "blocks": {
          "read_only_allow_delete": "false"
        }
    }
}