Python bulk方式写入ElasticSearch全流程，设置用户名密码，ImportError: No module named elasticsearch

最新推荐文章于 2024-09-04 19:00:19 发布

ITZHIHONH

最新推荐文章于 2024-09-04 19:00:19 发布

阅读量5.6k

点赞数 1

分类专栏： ElasticSearch python 文章标签： elasticsearch python

本文链接：https://blog.csdn.net/wzh8108/article/details/106058021

版权

ElasticSearch 同时被 2 个专栏收录

4 篇文章 0 订阅

订阅专栏

python

1 篇文章 0 订阅

订阅专栏

测试需要写入一堆数据，找到一个例子，但是只有中间部分，补了一些，但是一步步都是问题。先看几个解决的问题：

1、ImportError: No module named elasticsearch

Traceback (most recent call last):
File "pyimportjson.py", line 2, in <module>
import elasticsearch
ImportError: No module named elasticsearch

看代码没问题，书写争取，需要pip安装elasticsearch模块，根据python版本选择

pip install elasticsearch 或者 pip3 install elasticsearch

2、TypeError: 'encoding' is an invalid keyword argument for this function

input_file = open('300.json', "r", encoding='utf-8')
TypeError: 'encoding' is an invalid keyword argument for this function

修改为 input_file = open('300.json')后正常。

完整代码，读取文件内容后通过bulk方式写入ES

from datetime import datetime
from elasticsearch import Elasticsearch
from elasticsearch import helpers

import json

client = Elasticsearch(["192.168.99.110"],port="19200",http_auth=('elastic', 'passmy123'), timeout=20)




def read_and_write_index():
    # define an empty list for the Elasticsearch docs
    doc_list = []

    # use Python's enumerate() function to iterate over list of doc strings
    input_file=open('filedata.json')

    json_array = json.load(input_file)

    for item in json_array:
        try:
            # convert the string to a dict object
            # add a new field to the Elasticsearch doc
            dict_doc = {}
            # add a dict key called "_id" if you'd like to specify an ID for the doc
            dict_doc["_id"] = item['id']
            dict_doc["contents"] = item['contents']
            dict_doc["type"] = item['type']
            dict_doc["author"] = item['author']
            dict_doc["title"] = item['title']

            # append the dict object to the list []
            doc_list += [dict_doc]

        except json.decoder.JSONDecodeError as err:
            # print the errors
            print("ERROR for num:", item['id'], "-- JSONDecodeError:", err, "for doc:", dict_doc)
            print("Dict docs length:", len(doc_list))



    try:
        print ("\nAttempting to index the list of docs using helpers.bulk()")

        # use the helpers library's Bulk API to index list of Elasticsearch docs
        resp = helpers.bulk(
            client,
            doc_list,
            index = "some_index",
            doc_type = "_doc"
            )

        # print the response returned by Elasticsearch
        print ("helpers.bulk() RESPONSE:", resp)
        print ("helpers.bulk() RESPONSE:", json.dumps(resp, indent=4))
    except Exception as err:
        # print any errors returned w
        ## Prerequisiteshile making the helpers.bulk() API call
        print("Elasticsearch helpers.bulk() ERROR:", err)
        quit()

read_and_write_index()