Mongodb集群——分片和集群2高级实例

前面我们介绍了简单的集群配置实例。在简单实例中，虽然MongoDB auto-Sharding解决了海量存储问题，和动态扩容问题，但是离我们在真实环境下面所需要的高可靠性和高可用性还有一定的距离。

下面我们就介绍一个接近实际的解决方案：

• Shard：使用Replica Sets，来确保数据的可靠性。通过这个方案，可以在每个节点有数据的备份、实现自动控制容错转移和自动恢复
• Config：使用3个配置服务器，确保元数据完整性
• Route：配合LVS，实现负载均衡，提高接入性能。

1、配置集群目录

数据存储目录：

$ mkdir -p /home/scotte.ye/data/10001
$ mkdir -p /home/scotte.ye/data/10002
$ mkdir -p /home/scotte.ye/data/10003

$ mkdir -p /home/scotte.ye/data/10011
$ mkdir -p /home/scotte.ye/data/10012
$ mkdir -p /home/scotte.ye/data/10013

$ mkdir -p /home/scotte.ye/data/config1
$ mkdir -p /home/scotte.ye/data/config2
$ mkdir -p /home/scotte.ye/data/config3

日志目录：

$ mkdir -p /home/scotte.ye/data/logs

2、配置 Shard Replica Sets

配置第一组Sets

//10001
$ ./mongod --shardsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/10001/ -port=10001 --nohttpinterface --replSet set1

//10002
$ ./mongod --shardsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/10002/ -port=10002 --nohttpinterface --replSet set1

//10003
$ ./mongod --shardsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/10003/ -port=10003 --nohttpinterface --replSet set1

配置集群

$ ./mongo -port 10001
MongoDB shell version: 2.0.2
connecting to: 127.0.0.1:10001/test
>#配置集合1
>#构建参数
>cfg={_id:'set1',
members:[{_id:1,host:'192.168.35.106:10001'},
                {_id:1,host:'192.168.35.106:10002'},
                {_id:1,host:'192.168.35.106:10003'}]
};
{
     "_id":"set1",
     "members":[
          {
               "_id":1,
               "host":"192.168.35.106:10001"
          },
          {
               "_id":2,
               "host":"192.168.35.106:10002"
          },
          {
               "_id":3,
               "host":"192.168.35.106:10003"
          }
     ]
}
>#让参数生效
>rs.initiate(cfg);
{
     "info":"Config now saved locally. Should come online in about aminute.",
     "OK":1
}
>#查看运行状态
>rs.status();
{
        "set" : "set1",
        "date" : ISODate("2012-02-29T12:02:46Z"),
        "myState" : 2,
        "syncingTo" : "192.168.35.106:10003",
        "members" : [
                {
                        "_id" : 1,
                        "name" : "192.168.35.106:10001",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "PRIMARY",
                        "optime" : {
                                "t" : 1330435339000,
                                "i" : 2413
                        },
                        "optimeDate" : ISODate("2012-02-28T13:22:19Z"),
                        "self" : true
                },
                {
                        "_id" : 2,
                        "name" : "192.168.35.106:10002",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 1112736,
                        "optime" : {
                                "t" : 1330435339000,
                                "i" : 2413
                        },
                        "optimeDate" : ISODate("2012-02-28T13:22:19Z"),
                        "lastHeartbeat" : ISODate("2012-02-29T12:02:45Z"),
                        "pingMs" : 0
                },
                {
                        "_id" : 3,
                        "name" : "192.168.35.106:10003",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "SECONDARY",
                        "uptime" : 1112736,
                        "optime" : {
                                "t" : 1330435339000,
                                "i" : 2413
                        },
                        "optimeDate" : ISODate("2012-02-28T13:22:19Z"),
                        "lastHeartbeat" : ISODate("2012-02-29T12:02:46Z"),
                        "pingMs" : 0
                }
        ],
        "ok" : 1
}

配置第二组sets

//10011
$ ./mongod --shardsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/10011/ -port=10011 --nohttpinterface --replSet set2

//10012
$ ./mongod --shardsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/10012/ -port=10012 --nohttpinterface --replSet set2

//10013
$ ./mongod --shardsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/10013/ -port=10013 --nohttpinterface --replSet set2

配置集群

$ ./mongo -port 10011
MongoDB shell version: 2.0.2
connecting to: 127.0.0.1:10011/test
>#配置集合1
>#构建参数
>cfg={_id:'set2',
members:[{_id:1,host:'192.168.35.106:10011'},
                {_id:1,host:'192.168.35.106:10012'},
                {_id:1,host:'192.168.35.106:10013'}]
};
{
     "_id":"set2",
     "members":[
          {
               "_id":1,
               "host":"192.168.35.106:10011"
          },
          {
               "_id":2,
               "host":"192.168.35.106:10012"
          },
          {
               "_id":3,
               "host":"192.168.35.106:10013"
          }
     ]
}
>#让参数生效
>rs.initiate(cfg);
{
     "info":"Config now saved locally. Should come online in about aminute.",
     "OK":1
}
>#查看运行状态
>rs.status();
{
        "set" : "set2",
        "date" : ISODate("2012-02-29T12:12:46Z"),
        "myState" : 2,
        "syncingTo" : "192.168.35.106:10011",
        "members" : [
                {
                        "_id" : 1,
                        "name" : "192.168.35.106:10011",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "PRIMARY",
                        "optime" : {
                                "t" : 1330435339000,
                                "i" : 2413
                        },
                        "optimeDate" : ISODate("2012-02-28T13:22:19Z"),
                        "self" : true
                },
                {
                        "_id" : 2,
                        "name" : "192.168.35.106:10012",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 1112736,
                        "optime" : {
                                "t" : 1330435339000,
                                "i" : 2413
                        },
                        "optimeDate" : ISODate("2012-02-28T13:22:19Z"),
                        "lastHeartbeat" : ISODate("2012-02-29T12:02:45Z"),
                        "pingMs" : 0
                },
                {
                        "_id" : 3,
                        "name" : "192.168.35.106:10013",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "SECONDARY",
                        "uptime" : 1112736,
                        "optime" : {
                                "t" : 1330435339000,
                                "i" : 2413
                        },
                        "optimeDate" : ISODate("2012-02-28T13:22:19Z"),
                        "lastHeartbeat" : ISODate("2012-02-29T12:02:46Z"),
                        "pingMs" : 0
                }
        ],
        "ok" : 1
}

启用config servier

$ ./mongod --configsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/config1/ -port=20000 --nohttpinterface
$ ./mongod --configsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/config2/ -port=20001 --nohttpinterface
$ ./mongod --configsvr --fork -logpath=/home/data/logs/null -dbpath=/home/data/config3/ -port=20002 --nohttpinterface

启用Route server

$ ./mongos -configdb="192.168.35.106:20000,192.168.35.106:20001,192.168.35.106:20002" --fork -logpath=/home/data/logs/null
>#查看是否正常运行
>ps aux|grep mongos|grep -v grep
root      2726  0.0  0.1 174192  3392 ?        Sl   Feb16   3:20 ./mongos -configdb=192.168.35.106:20000,192.168.35.106:20001,192.168.35.106:20002 --fork -logpath=/home/data/logs/null

开始配置Sharding

$ ./mongo -port 10001
MongoDB shell version: 2.0.2
connecting to: 127.0.0.1:10001/test
>#进入管理数据库
>use admin
switched to db admin
>#添加Sharding
> db.runcommand({addshard:'set1/192.168.35.106:10001,192.168.35.106:10002,192.168.35.106:10003'})
{"shardAdded":"set1","ok":1}
> db.runCommand({addshard:'set2/192.168.35.106:10011,192.168.35.106:10012,192.168.35.106:10013'})
{"shardAdded":"set2","ok":1}
>#让test数据库支持Sharding
>db.runCommand({enablesharding:'test'})
{"ok":1}
>#让数据库中的一个集合生效，且根据ID来进行分片
>db.runCommand({shardcollection:'test.user',key:{_id:1}})
{"collectionshrded":"test.user","ok":1}

注：另外可以通过下面两个命令查看sharding情况:
>db.runCommand({listshards:1}
>printShardingStatus();