mongodb 分布式集群部署记录

资源有限，所有shard和repl在同一台机器，且只有一个repl
目录结构

.
├── cfg
│?? ├── c1
│?? └── c2
├── mos
│?? ├── m1
│?? └── m2
├── repl
│?? ├── re1
│?? ├── re2
│?? ├── re3
│?? └── re4
└── sd
    ├── chunk1
    ├── chunk2
    ├── chunk3
    └── chunk4

shards:

shard1:192.168.13.131:27100,192.168.13.131:27200(repl)
shard2:192.168.13.131:27101,192.168.13.131:27201(repl)
shard3:192.168.13.131:27102,192.168.13.131:27202(repl)
shard4:192.168.13.131:27103,192.168.13.131:27203(repl)

configserver:

cfg:192.168.13.131:27000,192.168.13.131:27001

mongos 路由节点

mos:192.168.13.130:27300,192.168.13.130:27301

步骤

1 以shard1为例配置shard的repl组

节点a： 192.168.13.131:27100

# mongod.conf
# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
  dbPath: /root/mdb/sd/chunk1
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:
# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /root/mdb/sd/chunk1/mongod.log
# network interfaces
net:
  port: 27100
  bindIp: 192.168.13.130
#processManagement:
#security:
#operationProfiling:
replication:
  oplogSizeMB: 500
  replSetName: shard1
sharding:
  clusterRole: shardsvr
## Enterprise-Only Options:
#auditLog:
#snmp:

节点b： 192.168.13.131:27100

# mongod.conf
# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
  dbPath: /root/mdb/repl/re1
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:
# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /root/mdb/repl/re1/mongod.log
# network interfaces
net:
  port: 27200
  bindIp: 192.168.13.130
#processManagement:
#security:
#operationProfiling:
replication:
  oplogSizeMB: 500
  replSetName: shard1
sharding:
  clusterRole: shardsvr
## Enterprise-Only Options:
#
#auditLog:
#snmp:

分别启动节点a和b

mongod -f sd/chunk1/mongo.conf&
mongod -f repl/re1/mongo.conf&

连接任意一台，如a，配置repl

>rs.initiate()
>rs.add("192.168.13.131:27100")

2配置configserver

节点a：192.168.13.131:27000

# mongod.conf
# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
  dbPath: /root/mdb/cfg/c1
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:
# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /root/mdb/cfg/c1/mongod.log
# network interfaces
net:
  port: 27000
  bindIp: 192.168.13.130
#processManagement:
#security:
#operationProfiling:
replication:
  oplogSizeMB: 500
  replSetName: cfg
sharding:
  clusterRole: configsvr
## Enterprise-Only Options:
#auditLog:
#snmp:

节点b：192.168.13.131:27001

# mongod.conf
# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
storage:
  dbPath: /root/mdb/cfg/c2
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:
# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /root/mdb/cfg/c2/mongod.log
# network interfaces
net:
  port: 27001
  bindIp: 192.168.13.130
#processManagement:
#security:
#operationProfiling:
replication:
  oplogSizeMB: 500
  replSetName: cfg
sharding:
  clusterRole: configsvr
## Enterprise-Only Options:
#auditLog:
#snmp:

分别启动节点a和b

mongod -f cfg/c1/mongo.conf&
mongod -f cfg/c2/mongo.conf&

配置repl，主节点随机

rs.initiate(
  {
    _id: "cfg",//replSetName
    configsvr: true,
    members: [
      { _id : 0, host : "192.168.13.131:27000" },
      { _id : 1, host : "192.168.13.131:27001" }
    ]
  }
)

3 配置mogos路由

节点a：

# mongod.conf
# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
#storage:
#  dbPath: /root/mdb/mos/m1
#  journal:
#    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:
# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /root/mdb/mos/m1/mongod.log
# network interfaces
net:
  port: 27300
  bindIp: 192.168.13.130
#processManagement:
#security:
#operationProfiling:
#replication:
#  oplogSizeMB: 500
#  replSetName: shard1
sharding:
  configDB: cfg/192.168.13.130:27000,192.168.13.130:27001
## Enterprise-Only Options:
#auditLog:
#snmp:

节点b：192.168.13.131:27301

# mongod.conf
# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/
# Where and how to store data.
#storage:
#  dbPath: /root/mdb/mos/m2
#  journal:
#    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:
# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /root/mdb/mos/m2/mongod.log
# network interfaces
net:
  port: 27301
  bindIp: 192.168.13.130
#processManagement:
#security:
#operationProfiling:
#replication:
#  oplogSizeMB: 500
#  replSetName: shard1
sharding:
  configDB: cfg/192.168.13.130:27000,192.168.13.130:27001
## Enterprise-Only Options:
#auditLog:
#snmp:

启动路由节点

mongos -f mos/m1/mongo.conf&
mongos -f mos/m2/mongo.conf&

连接任意一台，如a，配置repl，方式同shard相同

>rs.initiate()
>rs.add("192.168.13.131:27301")

4 添加集群

有repl的节点，只添加其中一个

sh.addShard("shard1/192.168.13.131:27100")
sh.addShard("shard2/192.168.13.131:27101")
sh.addShard("shard3/192.168.13.131:27102")
sh.addShard("shard4/192.168.13.131:27103")

创建数据库shardtest，集群的数据库和collection需要特别声明

sh.enableSharding("shardtest")
sh.shardCollection( "shardtest.coll", { "_id" : 1,"user_id":1 } )

上面这两个命令可以在数据库和collection创建后执行

附，以下命令可以指定repl的主节点

cfg = rs.conf()
cfg.members[0].priority = 3
rs.reconfig(cfg)

插入数据，测试集群

var bulk = db.coll.initializeUnorderedBulkOp();
people = ["Mawr2wc", "Bieel3l", "Geo22rege", "El3e3iot", "Me3watt", "Trw3grey", "Trtrawc3y", "Gtrrwe3g", "Ste3wvre", "Kriwrs3tina", "Kwatyt3ie", "Je5wf3f"];
for(var i=0; i<1000000; i++){
   b = i;
   a = people[Math.floor(Math.random()*people.length)] + String(i) ;
   c = Math.floor(Math.random()*10001);
   bulk.insert( { "a":a, "b":b, "c":c });
}
bulk.execute();