Mongodb副本集：
NoSQL的产生就是为了解决大数据量、高扩展性、高性能、灵活数据模型、高可用性。但是光通过主从模式的架构远远达不到上面几点，由此MongoDB设计了副本集和分片的功能，先来用用副本集。
Mongodb副本集的同步机制：
数据复制的目的是使数据得到最大的可用性，避免单点故障引起的整站不能访问的情况的发生，Mongodb的副本集在同一时刻只有一台服务器是可以写的，副本集的主从复制也是一个异步同步的过程，是slave端从primary端获取日志，然后在自己身上完全顺序的执行日志所记录的各种操作（该日志是不记录查询操作的），这个日志就是local数据库中的oplog.rs表，默认在64位机器上这个表是比较大的，占磁盘大小的5%，oplog.rs的大小可以在启动参数中设定：--oplogSize 1000,单位是M。
1，设置yum源安装
创建/etc/yum.repos.d/mongodb-enterprise.repo文件，这样就可以使用yum来安装mongodb了。
|
每一个release版本的.repo文件可以在repository找到，记住奇数镜像版本（比如e.g.2.5）是开发版本，放在生成环境是不太合适的。
2，源码安装
2.1 下载安装包
在官方网站可以下载到比较新的版本，下载地址：
https://www.mongodb.org/dr/fastdl.mongodb.org/linux/mongodb-linux-x86_64-3.2.3.tgz/download；
我这里下载的是稳定版本：mongodb-linux-x86_64-3.0.3.tgz，安装也是用的3.0.3； 3.0.3的下载地址为：http://download.csdn.net/detail/mchdba/9546256
2.2 解压安装
| tar -xvf mongodb-linux-x86_64-3.0.3.tgz -C /usr/local/ |
3 启动3个mongodb服务
安装mongodbreplica sets，至少需要3台linux服务器，而初始化副本集合，本来应该是3台服务器，但是这里只是测试环境，启用了3个mongodb端口进程来来模拟3个mongodb的服务器达到一样的效果，在实际中，可以部署在多台mongodb服务器上，如下表所示：
| Mongodb 主机 | 服务器端口地址 | 默认角色 |
| mongodb主机 1 | localhost:27017 | primary |
| mongodb主机 2 | localhost:27018 | secondary |
| mongodb主机 3 | localhost:27019 | secondary |
开始启动3个mongodb服务：
|
# 启动第一个mongodb 27017，默认为primary服务器 mkdir -p /data/mongodb/log27017 mkdir -p /data/mongodb/data27017 /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod --replSet repset --port 27017 --dbpath /data/mongodb/data27017 --oplogSize 2048 --logpath /data/mongodb/log27017/mongod.log &
# 启动第二个mongodb 27018 mkdir -p /data/mongodb/log27018 mkdir -p /data/mongodb/data27018 /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod --replSet repset --port 27018 --dbpath /data/mongodb/data27018 --oplogSize 2048 --logpath /data/mongodb/log27018/mongod.log &
# 启动第三个 mongodb 27019 mkdir -p /data/mongodb/log27019 mkdir -p /data/mongodb/data27019 /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongod --replSet repset --port 27019 --dbpath /data/mongodb/data27019 --oplogSize 2048 --logpath /data/mongodb/log27019/mongod.log & |
4 添加replica sets配置
在任何一台mongodb实例上登录，进入admin库，执行config命令：
| > use admin switched to db admin > config = { _id:"repset", members:[ ... {_id:0,host:"mongodb_server:27017"}, ... {_id:1,host:"mongodb_server:27018"}, ... {_id:2,host:"mongodb_server:27019"}] ... };
> config = { _id:"repset", members:[ ... ... {_id:0,host:"mongodb_server:27017"}, ... ... {_id:1,host:"mongodb_server:27018"}, ... ... {_id:2,host:"mongodb_server:27019"}] ... ... }; { "_id" : "repset", "members" : [ { "_id" : 0, "host" : "mongodb_server:27017" }, { "_id" : 1, "host" : "mongodb_server:27018" }, { "_id" : 2, "host" : "mongodb_server:27019" } ] } > |
5 初始化副本集
# 初始化副本集配置，使用rs.initiate(config);命令
rs.initiate(config);
> rs.initiate(config);
{ "ok" : 1 }
repset:SECONDARY>
6 查看replica sets状态
看到config配置里面的第一个默认为primary，后面的都为secondary，然后查看"optime": Timestamp(1465641737, 1)，如果集合中的mongodb的optime一致，表是集群中各个mongodb数据是保持一致性的：
| # 随意登录一个mongodb服务 [root@oracle_master ~]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27017 MongoDB shell version: 3.0.3 connecting to: localhost:27017/test Server has startup warnings: 2016-06-10T07:56:33.333-0700 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-06-10T07:56:33.333-0700 I CONTROL [initandlisten] repset:PRIMARY> use admin switched to db admin repset:PRIMARY>
# 查看状态如下： repset:PRIMARY> rs.status(); { "set" : "repset", "date" : ISODate("2016-06-10T15:04:54.045Z"), "myState" : 1, "members" : [ { "_id" : 0, "name" : "mongodb_server:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 501, "optime" : Timestamp(1465571046, 1), "optimeDate" : ISODate("2016-06-10T15:04:06Z"), "electionTime" : Timestamp(1465571050, 1), "electionDate" : ISODate("2016-06-10T15:04:10Z"), "configVersion" : 1, "self" : true }, { "_id" : 1, "name" : "mongodb_server:27018", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 47, "optime" : Timestamp(1465571046, 1), "optimeDate" : ISODate("2016-06-10T15:04:06Z"), "lastHeartbeat" : ISODate("2016-06-10T15:04:52.493Z"), "lastHeartbeatRecv" : ISODate("2016-06-10T15:04:52.612Z"), "pingMs" : 1, "configVersion" : 1 }, { "_id" : 2, "name" : "mongodb_server:27019", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 47, "optime" : Timestamp(1465571046, 1), "optimeDate" : ISODate("2016-06-10T15:04:06Z"), "lastHeartbeat" : ISODate("2016-06-10T15:04:52.642Z"), "lastHeartbeatRecv" : ISODate("2016-06-10T15:04:52.642Z"), "pingMs" : 0, "lastHeartbeatMessage" : "could not find member to sync from", "configVersion" : 1 } ], "ok" : 1 } repset:PRIMARY> |
7，查看后台日志
可以看到27107的mongodb默认成为了primary，而27018和27019的成为了secondary了：
| [root@oracle_master ~]# tail -f /data/mongodb/log27017/mongod.log 2016-06-11T03:42:19.664-0700 I REPL [ReplicationExecutor] replSet election succeeded, assuming primary role 2016-06-11T03:42:19.664-0700 I REPL [ReplicationExecutor] transition to PRIMARY 2016-06-11T03:42:19.686-0700 I REPL [ReplicationExecutor] Member mongodb_server:27019 is now in state STARTUP2 2016-06-11T03:42:19.738-0700 I REPL [rsSync] transition to primary complete; database writes are now permitted 2016-06-11T03:42:22.798-0700 I NETWORK [initandlisten] connection accepted from 192.168.136.128:18197 #6 (4 connections now open) 2016-06-11T03:42:22.812-0700 I NETWORK [initandlisten] connection accepted from 192.168.136.128:18198 #7 (5 connections now open) 2016-06-11T03:42:22.849-0700 I NETWORK [conn6] end connection 192.168.136.128:18197 (4 connections now open) 2016-06-11T03:42:22.906-0700 I NETWORK [conn7] end connection 192.168.136.128:18198 (3 connections now open) 2016-06-11T03:42:23.668-0700 I REPL [ReplicationExecutor] Member mongodb_server:27018 is now in state SECONDARY 2016-06-11T03:42:23.689-0700 I REPL [ReplicationExecutor] Member mongodb_server:27019 is now in state SECONDARY |
8，验证replica sets 数据一致性
（1）先进去主库primary的mongodb上，录入数据：
| # 登录primary主库 [root@oracle_master mongodb]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27017 MongoDB shell version: 3.0.3 connecting to: localhost:27017/test Server has startup warnings: 2016-06-11T03:41:07.938-0700 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-06-11T03:41:07.938-0700 I CONTROL [initandlisten] repset:PRIMARY> # 录入数据 repset:PRIMARY> use yes_db; switched to db yes_db repset:PRIMARY> db.yes_users.insert({"uid":"20160611_0001"},{"uname":"tim.man"}); WriteResult({ "nInserted" : 1 }) repset:PRIMARY> |
（2）去第一个备库27018上，验证数据，有了yes_users表的数据，如下：
| [root@oracle_master mongodb]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27018 MongoDB shell version: 3.0.3 connecting to: localhost:27018/test Server has startup warnings: 2016-06-11T03:41:14.305-0700 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-06-11T03:41:14.305-0700 I CONTROL [initandlisten] repset:SECONDARY> use yes_db; switched to db yes_db repset:SECONDARY> show tables; 2016-06-11T04:03:02.088-0700 E QUERY Error: listCollections failed: { "note" : "from execCommand", "ok" : 0, "errmsg" : "not master" } at Error (<anonymous>) at DB._getCollectionInfosCommand (src/mongo/shell/db.js:646:15) at DB.getCollectionInfos (src/mongo/shell/db.js:658:20) at DB.getCollectionNames (src/mongo/shell/db.js:669:17) at shellHelper.show (src/mongo/shell/utils.js:625:12) at shellHelper (src/mongo/shell/utils.js:524:36) at (shellhelp2):1:1 at src/mongo/shell/db.js:646 # mongodb默认是从主节点读写数据的，副本节点上不允许读，需要设置副本节点可以读。 repset:SECONDARY> db.getMongo().setSlaveOk(); repset:SECONDARY> show tables; system.indexes yes_users repset:SECONDARY> db.yes_users.find(); # 看到有数据了 { "_id" : ObjectId("575bef8126150b8de92029eb"), "uid" : "20160611_0001" } repset:SECONDARY> |
（3）去第二个secondary备库27019上，检查数据是否已经复制过来：
| [root@oracle_master mongodb]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27019 MongoDB shell version: 3.0.3 connecting to: localhost:27019/test Server has startup warnings: 2016-06-11T03:41:20.363-0700 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-06-11T03:41:20.363-0700 I CONTROL [initandlisten] repset:SECONDARY> use yes_db switched to db yes_db repset:SECONDARY> db.getMongo().setSlaveOk();# mongodb默认是从主节点读写数据的，副本节点上不允许读，需要设置副本节点可以读。 repset:SECONDARY> db.yes_users.find();# 看到有数据了，数据已经复制过来。 { "_id" : ObjectId("575bef8126150b8de92029eb"), "uid" : "20160611_0001" } repset:SECONDARY> |
9，验证primary、secondary自动切换
9.1 自动切换示意图
官方介绍：https://docs.mongodb.com/master/core/replica-set-high-availability/
| Replica Set High Availability
Replica sets provide high availability using automatic failover. Failover allows a secondary member to become primary if the current primary becomes unavailable. |
9.2 验证自动切换操作
当primary不可用，则自动将合适的secondary成员切换成新的primary，接下来的(1)-> (5)就来验证这个：
（1） 关闭27017的mongod服务，即停止primary主库：
| [root@oracle_master mongodb]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27017 MongoDB shell version: 3.0.3 connecting to: localhost:27017/test Server has startup warnings: 2016-06-11T03:41:07.938-0700 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-06-11T03:41:07.938-0700 I CONTROL [initandlisten] repset:PRIMARY> use admin switched to db admin repset:PRIMARY> db.shutdownServer(); 2016-06-11T04:18:24.148-0700 I NETWORK DBClientCursor::init call() failed server should be down... 2016-06-11T04:18:24.151-0700 I NETWORK trying reconnect to localhost:27017 (127.0.0.1) failed 2016-06-11T04:18:24.152-0700 I NETWORK reconnect localhost:27017 (127.0.0.1) ok 2016-06-11T04:18:24.437-0700 I NETWORK Socket recv() errno:104 Connection reset by peer 127.0.0.1:27017 2016-06-11T04:18:24.437-0700 I NETWORK SocketException: remote: 127.0.0.1:27017 error: 9001 socket exception [RECV_ERROR] server [127.0.0.1:27017] 2016-06-11T04:18:24.437-0700 I NETWORK DBClientCursor::init call() failed 2016-06-11T04:18:24.440-0700 I NETWORK trying reconnect to localhost:27017 (127.0.0.1) failed 2016-06-11T04:18:24.441-0700 W NETWORK Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused 2016-06-11T04:18:24.441-0700 I NETWORK reconnect localhost:27017 (127.0.0.1) failed failed couldn't connect to server localhost:27017 (127.0.0.1), connection attempt failed > |
（2） 登录原来的主库primary库的mongodb，就会报错：
| [root@oracle_master mongodb]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27017 MongoDB shell version: 3.0.3 connecting to: localhost:27017/test 2016-06-11T04:20:40.597-0700 W NETWORK Failed to connect to 127.0.0.1:27017, reason: errno:111 Connection refused 2016-06-11T04:20:40.614-0700 E QUERY Error: couldn't connect to server localhost:27017 (127.0.0.1), connection attempt failed at connect (src/mongo/shell/mongo.js:181:14) at (connect):1:6 at src/mongo/shell/mongo.js:181 exception: connect failed [root@oracle_master mongodb]# |
（3） primary自动切换到下一个mongodb服务，即27018端口的mongodb为primary服务：
| [root@oracle_master mongodb]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27018 MongoDB shell version: 3.0.3 connecting to: localhost:27018/test Server has startup warnings: 2016-06-11T03:41:14.305-0700 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-06-11T03:41:14.305-0700 I CONTROL [initandlisten] repset:PRIMARY> rs.status(); { "set" : "repset", "date" : ISODate("2016-06-11T11:20:50.730Z"), "myState" : 1, "members" : [ { "_id" : 0, "name" : "mongodb_server:27017", "health" : 0, "state" : 8, "stateStr" : "(not reachable/healthy)", "uptime" : 0, "optime" : Timestamp(0, 0), "optimeDate" : ISODate("1970-01-01T00:00:00Z"), "lastHeartbeat" : ISODate("2016-06-11T11:20:50.467Z"), "lastHeartbeatRecv" : ISODate("2016-06-11T11:18:22.616Z"), "pingMs" : 6, "lastHeartbeatMessage" : "Failed attempt to connect to mongodb_server:27017; couldn't connect to server mongodb_server:27017 (192.168.136.128), connection attempt failed", "configVersion" : -1 }, { "_id" : 1, "name" : "mongodb_server:27018", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 2377, "optime" : Timestamp(1465642881, 2), "optimeDate" : ISODate("2016-06-11T11:01:21Z"), "electionTime" : Timestamp(1465643906, 1), "electionDate" : ISODate("2016-06-11T11:18:26Z"), "configVersion" : 1, "self" : true }, { "_id" : 2, "name" : "mongodb_server:27019", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 2312, "optime" : Timestamp(1465642881, 2), "optimeDate" : ISODate("2016-06-11T11:01:21Z"), "lastHeartbeat" : ISODate("2016-06-11T11:20:50.177Z"), "lastHeartbeatRecv" : ISODate("2016-06-11T11:20:50.718Z"), "pingMs" : 0, "configVersion" : 1 } ], "ok" : 1 } repset:PRIMARY> |
（4）在27018的后台log里面，也可以看到这一切换记录：
| 2016-06-11T04:18:25.647-0700 I REPL [ReplicationExecutor] Standing for election 2016-06-11T04:18:25.649-0700 W NETWORK [ReplExecNetThread-6] Failed to connect to 192.168.136.128:27017, reason: errno:111 Connection refused 2016-06-11T04:18:25.649-0700 I REPL [ReplicationExecutor] replSet possible election tie; sleeping 848ms until 2016-06-11T04:18:26.497-0700 2016-06-11T04:18:25.650-0700 I REPL [ReplicationExecutor] Error in heartbeat request to mongodb_server:27017; Location18915 Failed attempt to connect to mongodb_server:27017; couldn't connect to server mongodb_server:27017 (192.168.136.128), connection attempt faile d 2016-06-11T04:18:25.650-0700 W NETWORK [ReplExecNetThread-6] Failed to connect to 192.168.136.128:27017, reason: errno:111 Connection refused 2016-06-11T04:18:25.650-0700 I REPL [ReplicationExecutor] Error in heartbeat request to mongodb_server:27017; Location18915 Failed attempt to connect to mongodb_server:27017; couldn't connect to server mongodb_server:27017 (192.168.136.128), connection attempt faile d 2016-06-11T04:18:26.497-0700 I REPL [ReplicationExecutor] Standing for election 2016-06-11T04:18:26.500-0700 I REPL [ReplicationExecutor] replSet info electSelf 2016-06-11T04:18:26.502-0700 I REPL [ReplicationExecutor] replSet election succeeded, assuming primary role 2016-06-11T04:18:26.502-0700 I REPL [ReplicationExecutor] transition to PRIMARY 2016-06-11T04:18:27.333-0700 I REPL [rsSync] transition to primary complete; database writes are now permitted |
（5）再启动原来旧的primary库27017端口的mongodb（如果是实际上的坏了就是修复后启动），27017就会自动变成secondary状态。
看到后台mongodb日志：
| 2016-06-11T04:28:33.979-0700 I NETWORK [initandlisten] connection accepted from 192.168.136.128:11453 #170 (2 connections now open) 2016-06-11T04:28:33.979-0700 I NETWORK [conn170] end connection 192.168.136.128:11453 (1 connection now open) 2016-06-11T04:28:33.992-0700 I NETWORK [initandlisten] connection accepted from 192.168.136.128:11454 #171 (2 connections now open) 2016-06-11T04:28:35.422-0700 I REPL [ReplicationExecutor] Member mongodb_server:27017 is now in state SECONDARY 2016-06-11T04:28:55.672-0700 I NETWORK [conn169] end connection 192.168.136.128:11424 (1 connection now open) 2016-06-11T04:28:55.673-0700 I NETWORK [initandlisten] connection accepted from 192.168.136.128:11459 #172 (2 connections now open) |
再查看replica sets状态：
| [root@oracle_master mongodb]# /usr/local/mongodb-linux-x86_64-3.0.3/bin/mongo localhost:27018 MongoDB shell version: 3.0.3 connecting to: localhost:27018/test Server has startup warnings: 2016-06-11T03:41:14.305-0700 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended. 2016-06-11T03:41:14.305-0700 I CONTROL [initandlisten] repset:PRIMARY> rs.status(); { "set" : "repset", "date" : ISODate("2016-06-11T11:30:00.192Z"), "myState" : 1, "members" : [ { "_id" : 0, "name" : "mongodb_server:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 85, "optime" : Timestamp(1465642881, 2), "optimeDate" : ISODate("2016-06-11T11:01:21Z"), "lastHeartbeat" : ISODate("2016-06-11T11:29:58.752Z"), "lastHeartbeatRecv" : ISODate("2016-06-11T11:29:58.263Z"), "pingMs" : 0, "configVersion" : 1 }, { "_id" : 1, "name" : "mongodb_server:27018", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 2927, "optime" : Timestamp(1465642881, 2), "optimeDate" : ISODate("2016-06-11T11:01:21Z"), "electionTime" : Timestamp(1465643906, 1), "electionDate" : ISODate("2016-06-11T11:18:26Z"), "configVersion" : 1, "self" : true }, { "_id" : 2, "name" : "mongodb_server:27019", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 2862, "optime" : Timestamp(1465642881, 2), "optimeDate" : ISODate("2016-06-11T11:01:21Z"), "lastHeartbeat" : ISODate("2016-06-11T11:29:59.858Z"), "lastHeartbeatRecv" : ISODate("2016-06-11T11:29:59.804Z"), "pingMs" : 0, "configVersion" : 1 } ], "ok" : 1 } repset:PRIMARY> |
至此，验证完成，和理论上示意图上面说的效果是一致的，mongodb的replica sets可以实现高可用解决方案。
10，启动报错
Secondary库启动报错，如下所示：
| [root@oracle_master log27018]# more /data/mongodb/log27018/mongod.log 2016-06-11T02:44:32.326-0700 I JOURNAL [initandlisten] journal dir=/data/mongodb/data27018/journal 2016-06-11T02:44:32.326-0700 I JOURNAL [initandlisten] recover : no journal files present, no recovery needed 2016-06-11T02:44:32.326-0700 I JOURNAL [initandlisten] 2016-06-11T02:44:32.326-0700 E JOURNAL [initandlisten] Insufficient free space for journal files 2016-06-11T02:44:32.326-0700 I JOURNAL [initandlisten] Please make at least 3379MB available in /data/mongodb/data27018/journal or use --smallfiles 2016-06-11T02:44:32.326-0700 I JOURNAL [initandlisten] 2016-06-11T02:44:32.327-0700 I STORAGE [initandlisten] exception in initAndListen: 15926 Insufficient free space for journals, terminating 2016-06-11T02:44:32.327-0700 W REPL [initandlisten] ReplicationCoordinatorImpl::shutdown() called before startReplication() finished. Shutting down without cleaning up the replication system 2016-06-11T02:44:32.327-0700 I CONTROL [initandlisten] now exiting 2016-06-11T02:44:32.327-0700 I NETWORK [initandlisten] shutdown: going to close listening sockets... 2016-06-11T02:44:32.327-0700 I NETWORK [initandlisten] removing socket file: /tmp/mongodb-27018.sock 2016-06-11T02:44:32.327-0700 I NETWORK [initandlisten] shutdown: going to flush diaglog... 2016-06-11T02:44:32.327-0700 I NETWORK [initandlisten] shutdown: going to close sockets... 2016-06-11T02:44:32.327-0700 I STORAGE [initandlisten] shutdown: waiting for fs preallocator... 2016-06-11T02:44:32.327-0700 I STORAGE [initandlisten] shutdown: final commit... 2016-06-11T02:44:32.327-0700 I STORAGE [initandlisten] shutdown: closing all files... 2016-06-11T02:44:32.327-0700 I STORAGE [initandlisten] closeAllFiles() finished 2016-06-11T02:44:32.327-0700 I CONTROL [initandlisten] dbexit: rc: 100 [root@oracle_master log27018]# |
解决：
看到Please make at least3379MB available in /data/mongodb/data27018/journal or use –smallfiles，所以清理下磁盘空间，磁盘至少要保证有30G以上的空间才行。
网友段书峰blog地址：blog.itpub.net/15498
352
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
