mongo_connector.oplog_manager:670 - Exception during collection dump

今天再整合mongodb和elasticsearch时，执行最后一步命令

“mongo-connector -m -m localhost:8090 -t  -t -t localhost:9200 -d  -d elastic2_doc_manager -n -n admin.users ers ”的时候mongo-connector一直报这个错

2018-08-31 16:59:09,191 [ALWAYS] mongo_connector.connector:51 - Starting mongo-connector version: 2.5.1
2018-08-31 16:59:09,191 [ALWAYS] mongo_connector.connector:51 - Python version: 2.7.5 (default, Jul 13 2018, 13:06:57)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
2018-08-31 16:59:09,191 [ALWAYS] mongo_connector.connector:51 - Platform: Linux-3.10.0-862.11.6.el7.x86_64-x86_64-with-centos-7.5.1804-Core
2018-08-31 16:59:09,191 [ALWAYS] mongo_connector.connector:51 - pymongo version: 3.7.1
2018-08-31 16:59:09,198 [ALWAYS] mongo_connector.connector:51 - Source MongoDB version: 4.0.1
2018-08-31 16:59:09,198 [ALWAYS] mongo_connector.connector:51 - Target DocManager: mongo_connector.doc_managers.elastic2_doc_manager version: 0.3.0
2018-08-31 16:59:09,228 [WARNING] elasticsearch:82 - POST /_bulk [status:406 request:0.004s]
2018-08-31 16:59:09,228 [CRITICAL] mongo_connector.oplog_manager:670 - Exception during collection dump
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mongo_connector/oplog_manager.py", line 625, in do_dump
upsert_all(dm)
File "/usr/lib/python2.7/site-packages/mongo_connector/oplog_manager.py", line 611, in upsert_all
mapped_ns, long_ts)
File "/usr/lib/python2.7/site-packages/mongo_connector/util.py", line 46, in wrapped
reraise(new_type, exc_value, exc_tb)
File "/usr/lib/python2.7/site-packages/mongo_connector/util.py", line 35, in wrapped
return f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/mongo_connector/doc_managers/elastic2_doc_manager.py", line 366, in bulk_upsert
for ok, resp in responses:
File "/usr/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 159, in streaming_bulk
for result in _process_bulk_chunk(client, bulk_actions, raise_on_exception, raise_on_error, **kwargs):
File "/usr/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 89, in _process_bulk_chunk
raise e
OperationFailed: TransportError(406, u'Content-Type header [] is not supported')
2018-08-31 16:59:09,230 [ERROR] mongo_connector.oplog_manager:678 - OplogThread: Failed during dump collection cannot recover! Collection(Database(MongoClient(host=['localhost:8091'], document_class=dict, tz_aware=False, connect=True, replicaset=u'sand'), u'local'), u'oplog.rs')
2018-08-31 16:59:10,221 [ERROR] mongo_connector.connector:398 - MongoConnector: OplogThread <OplogThread(Thread-3, started 139947591845632)> unexpectedly stopped! Shutting down

折腾了两天最终解决，原因是新版的elasticsearch必须要有请求头，然而自己又不知道mongo-contronner内部怎么实现的，解决办法，降版本，我把elasticsearch的版本降到5.0后这个问题解决了

 

正确的启动信息

2018-09-03 10:14:55,497 [ALWAYS] mongo_connector.connector:51 - Starting mongo-connector version: 2.5.1
2018-09-03 10:14:55,498 [ALWAYS] mongo_connector.connector:51 - Python version: 2.7.5 (default, Jul 13 2018, 13:06:57)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
2018-09-03 10:14:55,498 [ALWAYS] mongo_connector.connector:51 - Platform: Linux-3.10.0-862.11.6.el7.x86_64-x86_64-with-centos-7.5.1804-Core
2018-09-03 10:14:55,498 [ALWAYS] mongo_connector.connector:51 - pymongo version: 3.7.1
2018-09-03 10:14:55,506 [ALWAYS] mongo_connector.connector:51 - Source MongoDB version: 4.0.1
2018-09-03 10:14:55,507 [ALWAYS] mongo_connector.connector:51 - Target DocManager: mongo_connector.doc_managers.elastic2_doc_manager version: 0.3.0

 

转载于:https://www.cnblogs.com/monkSand/p/9579412.html