最近刚开始接触MongDB,学习爬虫,需要把爬取到的数据存到Mongo中去,刚开始连接单机,后来连接不上了,了解到mongdb是集群,是有主从,连接单机是不行的,请教了下大神,特此记录下:
附上代码
mongdb类
import pymongo
'''
mongdb 操作工具类
'''
class mongodb:
def __init__(self, host, port, dbname, username, pwd):
self.client = pymongo.MongoClient(host=host, port=port)
self.db = self.client[dbname]
self.db.authenticate(username, pwd)
def get_collection(self, collectionname):
# 选择集合(mongo中collection和database都是延时创建的)
coll = self.db[collectionname]
return coll
def close(self):
self.client.close()
self.client = None
连接mongdb集群
def open_spider(self, spider):
self.db = mongodb('mongodb://节点1,节点2,节点3', port, 'dbname', "username", "pwd")
self.collection = self.db.get_collection("数据库名")
结果没问题了,又能连接了