概述
对于mongodb的相关介绍,在此不多累赘,如需了解请见:
http://www.mongodb.org/display/DOCS/Home
本文目的是说明如下几个功能:
1.在测试环境构建Replica Set方案
2.在服务器端和客户端测试failover
3.测试天然的读写分离,减轻服务器压力
4.添加权限认证功能
部署Replica Sets 方案
Relica Sets使用的是n个mongod节点,构建具备自动的容错功能(auto-failover),自动恢复的(auto-recovery)的高可用方案,理论上需要三个mongodb实例,在这个测试环境中,我采用的方案是2个mongod+1个arbiter。
(0)基础环境
mongod1 : deploy ip : 10.12.7.107 port : 27031 $ mkdir -p /data/db/0 ./mongod --dbpath /data/db/0 --port 27031 --replSet myset
mongod2 : doc ip : 10.12.7.108 port : 27032 $ mkdir -p /data/db/1 ./mongod --dbpath /data/db/1 --port 27032 --replSet myset
mongod3 : deploy ip : 10.12.7.107 port : 27033 $ mkdir -p /data/db/2 ./mongod --dbpath /data/db/2 --port 27033 --replSet myset
(1)实例化Replica Sets
任选一个mongod节点,mongo shell 登陆进去,执行如下内容:
> config = {_id: 'myset', members: [ {_id: 0, host: '10.12.7.107:27031'}, {_id: 1, host: '10.12.7.108:27032'}, {_id: 2, host: '10.12.7.107:27033', arbiterOnly: true}]} > rs.initiate(config) > rs.conf() #查看配置信息 > rs.staus()
到此,Replica Sets配置就算完成了,相关的配置信息保存在local数据库中。
(2)执行流程说明
在同一时刻,每组 Replica Sets 只有一个 Primary,用于接受写操作。而后会异步复制到其他成员数据库中。一旦 primary 死掉,会自动投票选出接任的 primary 来,原服务器恢复后成为普通成员。如果数据尚未从先前的 primary 复制到成员服务器,有可能会丢失数据。
在服务器端和客户端测试failover
(1)服务器端
在 mongo 中向 primary (27031) 插入数据:
> use test switched to db test > db.users.insert({name:"terrylc"})
会在mongodb的输出信息中,看到如下信息:
#27031 Tue Oct 18 08:44:07 [FileAllocator] allocating new datafile /data/db/0/test.ns, filling with zeroes... Tue Oct 18 08:44:07 [FileAllocator] done allocating datafile /data/db/0/test.ns, size: 16MB, took 0.054 secs Tue Oct 18 08:44:07 [FileAllocator] allocating new datafile /data/db/0/test.0, filling with zeroes... Tue Oct 18 08:44:07 [FileAllocator] done allocating datafile /data/db/0/test.0, size: 64MB, took 0.153 secs Tue Oct 18 08:44:07 [FileAllocator] allocating new datafile /data/db/0/test.1, filling with zeroes... Tue Oct 18 08:44:07 [conn3] building new index on { _id: 1 } for test.users Tue Oct 18 08:44:07 [conn3] done for 0 records 0secs Tue Oct 18 08:44:07 [conn3] insert test.users 213ms Tue Oct 18 08:44:09 [FileAllocator] done allocating datafile /data/db/0/test.1, size: 128MB, took 1.527 secs #27032 Tue Oct 18 23:43:02 [FileAllocator] allocating new datafile /data/db/1/test.ns, filling with zeroes... Tue Oct 18 23:43:02 [FileAllocator] done allocating datafile /data/db/1/test.ns, size: 16MB, took 0.054 secs Tue Oct 18 23:43:02 [FileAllocator] allocating new datafile /data/db/1/test.0, filling with zeroes... Tue Oct 18 23:43:03 [FileAllocator] done allocating datafile /data/db/1/test.0, size: 64MB, took 0.556 secs Tue Oct 18 23:43:03 [FileAllocator] allocating new datafile /data/db/1/test.1, filling with zeroes... Tue Oct 18 23:43:03 [replica set sync] building new index on { _id: 1 } for test.users Tue Oct 18 23:43:03 [replica set sync] done for 0 records 0secs Tue Oct 18 23:43:03 [FileAllocator] done allocating datafile /data/db/1/test.1, size: 128MB, took 0.166 secs #27033 Tue Oct 18 08:42:22 [ReplSetHealthPollTask] replSet info 10.12.7.107:27031 is up Tue Oct 18 08:42:22 [ReplSetHealthPollTask] replSet member 10.12.7.107:27031 PRIMARY Tue Oct 18 08:42:22 [ReplSetHealthPollTask] replSet info 10.12.7.108:27032 is up Tue Oct 18 08:42:22 [ReplSetHealthPollTask] replSet member 10.12.7.108:27032 RECOVERING Tue Oct 18 08:42:34 [ReplSetHealthPollTask] replSet member 10.12.7.108:27032 SECONDARY
停止27031服务,观察现象:
#27032 Tue Oct 18 23:47:47 [conn2] end connection 10.12.7.107:58587 Tue Oct 18 23:47:47 [replica set sync] replSet syncThread: 10278 dbclient error communicating with server: 10.12.7.107:27031 Tue Oct 18 23:47:48 [ReplSetHealthPollTask] DBClientCursor::init call() failed Tue Oct 18 23:47:48 [ReplSetHealthPollTask] replSet info 10.12.7.107:27031 is down (or slow to respond): DBClientBase::findOne: transport error: 10.12.7.107:27031 query: { replSetHeartbeat: "myset", v: 1, pv: 1, checkEmpty: false, from: "10.12.7.108:27032" } Tue Oct 18 23:47:48 [rs Manager] replSet info electSelf 1 Tue Oct 18 23:47:48 [rs Manager] replSet couldn't elect self, only received -9999 votes Tue Oct 18 23:47:54 [rs Manager] replSet info electSelf 1 Tue Oct 18 23:47:54 [rs Manager] replSet PRIMARY #27033 Tue Oct 18 08:48:52 [conn2] end connection 10.12.7.107:43768 Tue Oct 18 08:48:53 [conn3] 10.12.7.108:27032 is trying to elect itself but 10.12.7.107:27031 is already primary and more up-to-date Tue Oct 18 08:48:54 [ReplSetHealthPollTask] DBClientCursor::init call() failed Tue Oct 18 08:48:54 [ReplSetHealthPollTask] replSet info 10.12.7.107:27031 is down (or slow to respond): DBClientBase::findOne: transport error: 10.12.7.107:27031 query: { replSetHeartbeat: "myset", v: 1, pv: 1, checkEmpty: false, from: "10.12.7.107:27033" } Tue Oct 18 08:48:59 [conn3] replSet info voting yea for 1 Tue Oct 18 08:49:00 [ReplSetHealthPollTask] replSet member 10.12.7.108:27032 PRIMARY }}}
Mongod 27032 被选为 Primary,在对应的终端下,执行shell mongo
$ ./mongo localhost:27032 > rs.status() { "set" : "myset", "date" : ISODate("2011-10-18T15:53:01Z"), "myState" : 1, "members" : [ { "_id" : 0, "name" : "10.12.7.107:27031", "health" : 0, "state" : 1, "stateStr" : "(not reachable/healthy)", "uptime" : 0, "optime" : { "t" : 1318952647000, "i" : 1 }, "optimeDate" : ISODate("2011-10-18T15:44:07Z"), "lastHeartbeat" : ISODate("2011-10-18T15:47:46Z"), "errmsg" : "socket exception" }, { "_id" : 1, "name" : "10.12.7.108:27032", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "optime" : { "t" : 1318952647000, "i" : 1 }, "optimeDate" : ISODate("2011-10-18T15:44:07Z"), "self" : true }, { "_id" : 2, "name" : "10.12.7.107:27033", "health" : 1, "state" : 7, "stateStr" : "ARBITER", "uptime" : 705, "optime" : { "t" : 0, "i" : 0 }, "optimeDate" : ISODate("1970-01-01T00:00:00Z"), "lastHeartbeat" : ISODate("2011-10-18T15:53:00Z") } ], "ok" : 1 } }}}
查询先前插入的记录正常:
myset:PRIMARY> db.users.findOne() { "_id" : ObjectId("4e9d9ec76df8f2f72bd9c45a"), "name" : "terrylc" }
如果重新启动27031对应的mongod,此实例将会变成secondary
Tue Oct 18 23:56:46 [ReplSetHealthPollTask] replSet info 10.12.7.107:27031 is up Tue Oct 18 23:56:46 [ReplSetHealthPollTask] replSet member 10.12.7.107:27031 SECONDARY Tue Oct 18 23:56:47 [conn6] I am already primary, 10.12.7.107:27031 can try again once I've stepped down Tue Oct 18 23:56:48 [initandlisten] connection accepted from 10.12.7.107:44777 #7 Tue Oct 18 23:56:49 [slaveTracking] building new index on { _id: 1 } for local.slaves Tue Oct 18 23:56:49 [slaveTracking] done for 0 records 0secs Tue Oct 18 23:57:03 [conn5] end connection 127.0.0.1:58555
(2)客户端测试
从客户端连接 Replica Sets,需要 drivers 支持,本例中,我采用pymongo2.0.1
* 测试功能可用性
目标:当一个主节点出现故障时,备用节点是否可以转化为主节点。
import pymongo import sys def testInsert(): flag = 0 while True: if flag == 4: print "try to connect server 4 times but failed,so give up" break try: conn = pymongo.Connection(host=["10.12.7.107:27031", "10.12.7.108:27032", "10.12.7.107:27033"]) db = conn.test post = {"name1": "terrylc"} db.users.insert(post) conn.disconnect() return sys.exit(0) except Exception, e: print e flag += 1 if __name__ == '__main__': testInsert() }}}
结论:
在服务器端的某个实例当掉后,客户端依然能够正常工作。
* 测试大规模写数据完整性
需求:当进行大规模写的时候,如果几个节点在特定时间交叉宕机后,能否确保数据完成性
目标:用python客户端向db中连续插入5000条记录,期望在插入完成后db中增加5000条记录:
def testInsert2(index): try: conn = pymongo.Connection(host=["10.12.7.107:27031", "10.12.7.108:27032", "10.12.7.107:27033"]) db = conn.test while index <= 5000: post = {"testTAG": index} time.sleep(1) objectId = db.users.save(post, safe=True,w=2) if objectId is not None: index += 1 conn.disconnect() except Exception, e: print e print "you index is :" + str(index) time.sleep(2) testInsert2(index) if __name__ == '__main__': testInsert2(1) }}}
结论:
在测试过程,分别模拟几台mongodb节点宕机现象,客户端在进行重新连接后能够继续进行写操作,并且最后数据一致且都能查询到。
测试天然的读写分离,减轻服务器压力
其实很简单的方式是在客户端的请求中添加如下请求字段即可:
conn = pymongo.Connection(host=["10.12.7.107:27031", "10.12.7.108:27032", "10.12.7.107:27033"], slave_okay=True) slave_okay=True # 这个是关键,可以将读写分离
添加权限认证功能
Authentication was added in 1.7.5(mongodb)
Replica Sets 的认证跟原本的单台的认证是不同的,最直接的表现在使用replica Sets时,启动实例方式如下:
$ echo "this is my super secret key" > mykey $ chmod 600 mykey $ mongod --keyFile mykey # other options... $ echo "this is my super secret key" > mykey $ chmod 600 mykey $ mongod --keyFile mykey # other options... $ echo "this is my super secret key" > mykey $ chmod 600 mykey $ mongod --keyFile mykey # other options...
然后在对应的数据库中添加用户即可。
启动方式不再使用--auth
补充
如果后续需要考虑写服务器的负载问题,可以在现在的组件外层加上Sharding层。