1. MongoDB主从复制
主从复制是Mongodb最常用的复制方式,可用于备份、故障恢复、读扩展等
规划
这里资源有限,在一台机器上不同dbpath和port启动多个mongodb实例来实现主从
| conf_file | dbpath | logpath | port |
主 | /etc/mongodb/master.conf | /data/mongodb/master | /data/mongodb/logs/master.log | 10000 |
从 | /etc/mongodb/slave.conf | /data/mongodb/slave | /data/mongodb/logs/slave.log | 20000 |
创建[db|log]path
~]# mkdir /data/mongodb/master
~]# mkdir /data/mongodb/slave
~]# mkdir /data/mongodb/logs
创建配置文件
~]# mkdir /etc/mongodb/
#Master配置文件
~]# vim /etc/mongodb/master.conf
# mongodb config file
dbpath = /data/mongodb/master
logpath = /data/mongodb/logs/master.log
logappend = true
fork = true
oplogSize = 100
port = 10000
master = true
#Slave配置文件
~]# vim /etc/mongodb/slave.conf
# mongodb config file
dbpath = /data/mongodb/slave
logpath = /data/mongodb/logs/slave.log
logappend = true
fork = true
port = 20000
slave = true
source = 192.168.211.217:10000
启动master
~]# /usr/local/mongodb/bin/mongod -f /etc/mongodb/master.conf
about to fork child process, waiting until server is ready for connections.
forked process: 30160
child process started successfully, parent exiting
~]# netstat -ntpl|grep mong
tcp 0 0 0.0.0.0:10000 0.0.0.0:* LISTEN 30160/mongod
~]# ps -ef |grep mong
root 30160 1 0 10:06 ? 00:00:00 /usr/local/mongodb/bin/mongod -f /etc/mongodb/master.conf
#监控master的log
~]# tail –f /data/mongodb/logs/master.log
2015-01-14T10:06:45.147+0800 [initandlisten] MongoDB starting : pid=30160 port=10000 dbpath=/data/mongodb/master master=1 64-bit host=test-whole-db
2015-01-14T10:06:45.147+0800 [initandlisten] db version v2.6.6
2015-01-14T10:06:45.147+0800 [initandlisten] git version: 608e8bc319627693b04cc7da29ecc300a5f45a1f
2015-01-14T10:06:45.147+0800 [initandlisten] build info: Linux build10.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2015-01-14T10:06:45.147+0800 [initandlisten] allocator: tcmalloc
2015-01-14T10:06:45.147+0800 [initandlisten] options: { config: "/etc/mongodb/master.conf", master: true, net: { port: 10000 }, processManagement: { fork: true }, replication: { oplogSizeMB: 100 }, storage: { dbPath: "/data/mongodb/master" }, systemLog: { destination: "file", logAppend: true, path: "/data/mongodb/logs/master.log" } }
2015-01-14T10:06:45.214+0800 [initandlisten] journal dir=/data/mongodb/master/journal
2015-01-14T10:06:45.214+0800 [initandlisten] recover : no journal files present, no recovery needed
2015-01-14T10:06:45.476+0800 [initandlisten] ******
2015-01-14T10:06:45.476+0800 [initandlisten] creating replication oplog of size: 100MB...
2015-01-14T10:06:45.476+0800 [FileAllocator] allocating new datafile /data/mongodb/master/local.1, filling with zeroes...
2015-01-14T10:06:45.476+0800 [FileAllocator] creating directory /data/mongodb/master/_tmp
2015-01-14T10:06:45.525+0800 [FileAllocator] done allocating datafile /data/mongodb/master/local.1, size: 128MB, took 0.024 secs
2015-01-14T10:06:45.526+0800 [initandlisten] ******
2015-01-14T10:06:45.526+0800 [initandlisten] waiting for connections on port 10000
2015-01-14T10:07:24.089+0800 [initandlisten] connection accepted from 127.0.0.1:61489 #1 (1 connection now open)
#确认是否为master
~]# /usr/local/mongodb/bin/mongo --port 10000
MongoDB shell version: 2.6.6
connecting to: 127.0.0.1:10000/test
> db.isMaster()
{
"ismaster" : true,
"maxBsonObjectSize" : 16777216,
"maxMessageSizeBytes" : 48000000,
"maxWriteBatchSize" : 1000,
"localTime" : ISODate("2015-01-14T02:50:13.810Z"),
"maxWireVersion" : 2,
"minWireVersion" : 0,
"ok" : 1
}
启动slave
~]# /usr/local/mongodb/bin/mongod -f /etc/mongodb/slave.conf
about to fork child process, waiting until server is ready for connections.
forked process: 31056
child process started successfully, parent exiting
~]# tail -f /data/mongodb/logs/slave.log
2015-01-14T10:55:26.719+0800 [initandlisten] MongoDB starting : pid=31056 port=20000 dbpath=/data/mongodb/slave slave=1 64-bit host=test-whole-db
2015-01-14T10:55:26.719+0800 [initandlisten] db version v2.6.6
2015-01-14T10:55:26.719+0800 [initandlisten] git version: 608e8bc319627693b04cc7da29ecc300a5f45a1f
2015-01-14T10:55:26.719+0800 [initandlisten] build info: Linux build10.nj1.10gen.cc 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 BOOST_LIB_VERSION=1_49
2015-01-14T10:55:26.719+0800 [initandlisten] allocator: tcmalloc
2015-01-14T10:55:26.719+0800 [initandlisten] options: { config: "/etc/mongodb/slave.conf", net: { port: 20000 }, processManagement: { fork: true }, slave: true, source: "192.168.211.217:10000", storage: { dbPath: "/data/mongodb/slave" }, systemLog: { destination: "file", logAppend: true, path: "/data/mongodb/logs/slave.log" } }
2015-01-14T10:55:26.774+0800 [initandlisten] journal dir=/data/mongodb/slave/journal
2015-01-14T10:55:26.774+0800 [initandlisten] recover : no journal files present, no recovery needed
2015-01-14T10:55:28.453+0800 [initandlisten] preallocateIsFaster=true 20.96
2015-01-14T10:55:30.548+0800 [initandlisten] preallocateIsFaster=true 13.2
2015-01-14T10:55:33.139+0800 [initandlisten] preallocateIsFaster=true 19.04
2015-01-14T10:55:33.139+0800 [initandlisten] preallocateIsFaster check took 6.364 secs
2015-01-14T10:55:33.139+0800 [initandlisten] preallocating a journal file /data/mongodb/slave/journal/prealloc.0
2015-01-14T10:56:08.323+0800 [initandlisten] preallocating a journal file /data/mongodb/slave/journal/prealloc.1
2015-01-14T10:56:36.587+0800 [initandlisten] preallocating a journal file /data/mongodb/slave/journal/prealloc.2
2015-01-14T10:57:17.838+0800 [initandlisten] allocating new ns file /data/mongodb/slave/local.ns, filling with zeroes...
2015-01-14T10:57:18.417+0800 [FileAllocator] allocating new datafile /data/mongodb/slave/local.0, filling with zeroes...
2015-01-14T10:57:18.417+0800 [FileAllocator] creating directory /data/mongodb/slave/_tmp
2015-01-14T10:57:18.484+0800 [FileAllocator] done allocating datafile /data/mongodb/slave/local.0, size: 64MB, took 0.024 secs
2015-01-14T10:57:18.505+0800 [initandlisten] build index on: local.startup_log properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.startup_log" }
2015-01-14T10:57:18.505+0800 [initandlisten] added index to empty collection
2015-01-14T10:57:18.509+0800 [initandlisten] command local.$cmd command: create { create: "startup_log", size: 10485760, capped: true } ntoreturn:1 keyUpdates:0 numYields:0 reslen:37 671ms
2015-01-14T10:57:18.519+0800 [initandlisten] waiting for connections on port 20000
2015-01-14T10:57:19.521+0800 [replslave] build index on: local.me properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.me" }
2015-01-14T10:57:19.521+0800 [replslave] added index to empty collection
2015-01-14T10:57:19.523+0800 [replslave] build index on: local.sources properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "local.sources" }
2015-01-14T10:57:19.523+0800 [replslave] added index to empty collection
2015-01-14T10:57:19.523+0800 [replslave] repl: syncing from host:192.168.211.217:10000
2015-01-14T10:57:19.530+0800 [replslave] resync: dropping database admin
2015-01-14T10:57:19.690+0800 [replslave] removeJournalFiles
2015-01-14T10:57:19.750+0800 [replslave] resync: cloning database admin to get an initial copy
2015-01-14T10:57:19.760+0800 [replslave] allocating new ns file /data/mongodb/slave/admin.ns, filling with zeroes...
2015-01-14T10:57:20.182+0800 [FileAllocator] allocating new datafile /data/mongodb/slave/admin.0, filling with zeroes...
2015-01-14T10:57:20.223+0800 [FileAllocator] done allocating datafile /data/mongodb/slave/admin.0, size: 64MB, took 0.041 secs
2015-01-14T10:57:20.320+0800 [replslave] build index on: admin.system.version properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "admin.system.version" }
2015-01-14T10:57:20.320+0800 [replslave] building index using bulk method
2015-01-14T10:57:20.320+0800 [replslave] build index done. scanned 1 total records. 0 secs
2015-01-14T10:57:20.335+0800 [replslave] build index on: admin.system.users properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "admin.system.users" }
2015-01-14T10:57:20.335+0800 [replslave] building index using bulk method
2015-01-14T10:57:20.335+0800 [replslave] build index done. scanned 3 total records. 0 secs
2015-01-14T10:57:20.336+0800 [replslave] build index on: admin.system.users properties: { v: 1, unique: true, key: { user: 1, db: 1 }, name: "user_1_db_1", ns: "admin.system.users" }
2015-01-14T10:57:20.336+0800 [replslave] building index using bulk method
2015-01-14T10:57:20.336+0800 [replslave] build index done. scanned 3 total records. 0 secs
2015-01-14T10:57:20.336+0800 [replslave] resync: done with initial clone for db: admin
2015-01-14T10:57:28.379+0800 [replslave] repl: applied 1 operations
2015-01-14T10:57:28.379+0800 [replslave] repl: end sync_pullOpLog syncedTo: Jan 14 10:57:28 54b5db18:1
2015-01-14T10:57:28.379+0800 [replslave] repl: syncing from host:192.168.211.217:10000
2015-01-14T10:57:28.379+0800 [replslave] resync: dropping database test
2015-01-14T10:57:29.073+0800 [replslave] removeJournalFiles
2015-01-14T10:57:29.171+0800 [replslave] resync: cloning database test to get an initial copy
2015-01-14T10:57:29.172+0800 [replslave] allocating new ns file /data/mongodb/slave/test.ns, filling with zeroes...
2015-01-14T10:57:29.663+0800 [FileAllocator] allocating new datafile /data/mongodb/slave/test.0, filling with zeroes...
2015-01-14T10:57:29.689+0800 [FileAllocator] done allocating datafile /data/mongodb/slave/test.0, size: 64MB, took 0.026 secs
2015-01-14T10:57:29.776+0800 [replslave] build index on: test.emp properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test.emp" }
2015-01-14T10:57:29.776+0800 [replslave] building index using bulk method
2015-01-14T10:57:29.776+0800 [replslave] build index done. scanned 15 total records. 0 secs
2015-01-14T10:57:29.792+0800 [replslave] build index on: test.my_collection properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test.my_collection" }
2015-01-14T10:57:29.792+0800 [replslave] added index to empty collection
2015-01-14T10:57:29.793+0800 [replslave] build index on: test.yu_collection properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "test.yu_collection" }
2015-01-14T10:57:29.793+0800 [replslave] added index to empty collection
2015-01-14T10:57:29.794+0800 [replslave] resync: done with initial clone for db: test
2015-01-14T10:57:34.806+0800 [replslave] repl: syncing from host:192.168.211.217:10000
测试主从
#slave上当前没有masterslave集合
#master创建masterslave,并且插入文档
#再次查看slave上,也同步了master的操作
local数据库
Local(本地)数据库用来存放所有内部复制状态,主从节点都有。Local数据库不会被复制,每个节点独自拥有自己的local数据库
#master上查看master的信息
> use local
switched to db local
> db.slaves.find()
{ "_id" : ObjectId("54b5db0fddeb3d1dd59a5aa1"), "config" : { "host" : "192.168.211.217:46756", "upgradeNeeded" : true }, "ns" : "local.oplog.$main", "syncedTo" : Timestamp(1421216379, 1) }
#slave上查看master的信息
> use local
switched to db local
> db.sources.find()
{ "_id" : ObjectId("54b5db0fddeb3d1dd59a5aa2"), "host" : "192.168.211.217:10000", "source" : "main", "syncedTo" : Timestamp(1421213869, 1) }
主从复制原理
在主从结构中,主节点的操作记录成为oplog(operation log)。
oplog存储在一个系统数据库local的集合oplog.$main中,这个集合的每个文档都代表主节点上执行的一个操作。
#master上查看oplog内容
> db
local
> db.oplog.$main.find()
{ "ts" : Timestamp(1421201205, 11), "op" : "n", "ns" : "", "o" : { } }
{ "ts" : Timestamp(1421201219, 1), "op" : "n", "ns" : "", "o" : { } }
{ "ts" : Timestamp(1421201229, 1), "op" : "n", "ns" : "", "o" : { } }
{ "ts" : Timestamp(1421201239, 1), "op" : "n", "ns" : "", "o" : { } }
{ "ts" : Timestamp(1421201249, 1), "op" : "n", "ns" : "", "o" : { } }
。。。。。。
键“ts”表示时间戳,键“op”表示操作类型,“ns”表示操作的集合名,“o”表示具体操作的文档
注意:存储在oplog中的操作有时可能和主节点操作并不一致,因为存储操作前要进行等幂变换,也就是这些操作必须可以再从服务器上多次执行,只要顺序一致,结果就一致。
因此,比如更新器$inc的操作,最后会被变换为$set操作!并且这个集合中只会存储影响数据的操作,查询这类不影响数据的操作不会以oplog的形式进行存储!从服务器会定期从主服务器中获取oplog记录,然后在本机上执行!
对于存储oplog的集合,MongoDB采用的是固定集合,也就是说随着操作过多,新的操作会覆盖旧的操作!这样做也是有道理的,不然,这个集合占用的空间就无法估算了!我们在启动服务时,可以通过选项--oplogSize来指定这个集合的大小,单位是MB,在Windows平台下,默认MongoDB会使用数据库安装分区可用空间的5%作为这个集合的大小!
常用选项
--only
在从节点上指定只复制特定的某个数据库(默认是复制所有数据库)
--slavedelay
用在从节点上,当应用主节点的操作时,从节点增加延时复制(单位秒).这样就能轻松设置延时从节点,这种节点对用户无意中删除重要文档或者插入垃圾数据等有防护作用,这些不良操作都会被复制到所有的从节点上,通过延时执行操作,可以有个恢复的时间差.
--fastsync
以主节点的数据快照为基础启动从节点.如果数据目录一开始是主节点的数据快照,从节点用这个选项启动要比做完整的同步快的多.
--autoresync
如果从节点与主节点不同步了,则自动重新同步
--oplogsize
主节点oplog的大小(单位MB)
添加删除源
来源于《mongodb权威指南》
来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/27000195/viewspace-1401941/,如需转载,请注明出处,否则将追究法律责任。
转载于:http://blog.itpub.net/27000195/viewspace-1401941/