elaselasticsearch2.3 python search超时bug

2019独角兽企业重金招聘Python工程师标准>>>

本地安装elasticsearch python包

pip install elasticsearch==2.3.0

注意指明版本号

建立连接

from elasticsearch import Elasticsearch
es = Elasticsearch()
或
es = Elasticsearch(
    [
        'ip1:9200','ip2:9200'
    ]
)

不指明ip和端口号，默认连接本地9200端口，连接多台可以保证高可用

执行操作

特别说明search操作

官网函数说明

search(*args, **kwargs)
Execute a search query and get back search hits that match the query. http://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html

Parameters:	
index – A comma-separated list of index names to search; use _all or empty string to perform the operation on all indices

doc_type – A comma-separated list of document types to search; leave empty to perform the operation on all types

body – The search definition using the Query DSL

_source – True or false to return the _source field or not, or a list of fields to return

_source_exclude – A list of fields to exclude from the returned _source field

_source_include – A list of fields to extract and return from the _source field

allow_no_indices – Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

analyze_wildcard – Specify whether wildcard and prefix queries should be analyzed (default: false)

analyzer – The analyzer to use for the query string

default_operator – The default operator for query string query (AND or OR), default ‘OR’, valid choices are: ‘AND’, ‘OR’

df – The field to use as default where no field prefix is given in the query string

expand_wildcards – Whether to expand wildcard expression to concrete indices that are open, closed or both., default ‘open’, valid choices are: ‘open’, ‘closed’, ‘none’, ‘all’

explain – Specify whether to return detailed information about score computation as part of a hit

fielddata_fields – A comma-separated list of fields to return as the field data representation of a field for each hit

fields – A comma-separated list of fields to return as part of a hit

from_ – Starting offset (default: 0)

ignore_unavailable – Whether specified concrete indices should be ignored when unavailable (missing or closed)

lenient – Specify whether format-based query failures (such as providing text to a numeric field) should be ignored

lowercase_expanded_terms – Specify whether query terms should be lowercased

preference – Specify the node or shard the operation should be performed on (default: random)

q – Query in the Lucene query string syntax

request_cache – Specify if request cache should be used for this request or not, defaults to index level setting

routing – A comma-separated list of specific routing values

scroll – Specify how long a consistent view of the index should be maintained for scrolled search

search_type – Search operation type, valid choices are: ‘query_then_fetch’, ‘dfs_query_then_fetch’, ‘count’, ‘scan’

size – Number of hits to return (default: 10)

sort – A comma-separated list of <field>:<direction> pairs

stats – Specific ‘tag’ of the request for logging and statistical purposes

suggest_field – Specify which field to use for suggestions

suggest_mode – Specify suggest mode, default ‘missing’, valid choices are: ‘missing’, ‘popular’, ‘always’

suggest_size – How many suggestions to return in response

suggest_text – The source text for which the suggestions should be returned

terminate_after – The maximum number of documents to collect for each shard, upon reaching which the query execution will terminate early.

timeout – Explicit operation timeout

track_scores – Whether to calculate and return scores even if they are not used for sorting

version – Specify whether to return document version as part of a hit

代码

rs = es.search(index=index,body=body,size=0,timeout="60s")

当我们执行search的时候会报read timeout的错误

Traceback (most recent call last):
  File "es_test.py", line 39, in <module>
    rs = es.search(index=index,body=body,size=0,timeout="60s")
  File "/home/xiaoju/weijian/anaconda2/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/xiaoju/weijian/anaconda2/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 548, in search
    doc_type, '_search'), params=params, body=body)
  File "/home/xiaoju/weijian/anaconda2/lib/python2.7/site-packages/elasticsearch/transport.py", line 329, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/home/xiaoju/weijian/anaconda2/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 102, in perform_request
    raise ConnectionTimeout('TIMEOUT', str(e), e)
elasticsearch.exceptions.ConnectionTimeout: ConnectionTimeout caused by - ReadTimeoutError(HTTPConnectionPool(host='bigdata-ara-ser200.gz01', port=8200): Read timed out. (read timeout=10))

我们明明设置了timeout为60s，为什么报错显示为10s，timeout为什么没有生效，我百思不得其解，后来我按照报错在源码包里的位置找到对应的函数

如图所示，它取得是request_timeout这个参数，但是search这个函数的参数里却只有timeout这个参数，估计是官网没有更新这个bug，然后我们修改代码

rs = es.search(index=index,body=body,size=0,request_timeout=60)

这时候我们发现运行成功。

更多api请查看

http://elasticsearch-py.readthedocs.io/en/2.3.0/api.html#elasticsearch

转载于:https://my.oschina.net/u/2000675/blog/1512007