mongodb 分片集群配置

最近使用MongoDB,对分片功能服务搭建,再此 记录下一些流程。该文档对应着MongoDB4.0.0版。 希望能对初搭者有些帮助。

一、下载MongoDB

  1. 下载linux版的MongoDB https://www.mongodb.com/download-center?jmp=nav#community、
  2. 解压到/guaoran/mongodb/mongodb-4.0.0目录中(本人目录)

二、复制集

  1. 复制集实现了数据库的亢余备份、故障转移的功能。为了保证数据库的高可用性,在生成环境中一个复制集至少需要包含三个节点,其中一个是主节点(primary),一个为仲裁节点(arbiter),其他是second节点。   (对于primary、second、arbiter是什么不清楚的,可以参考下MongoDB中文社区 的解释或其他网站。
  2. 搭建复制集
    1. 创建复制集目录(192.168.20.56[某一个台]),其他两台相同
      1. cd /guaoran/mongodb/shard
      2. mkdir -p rs0/data
      3. mkdir -p rs0/logs
      4. mkdir -p rs1/data
      5. mkdir -p rs1/logs
      6. mkdir -p rs2/data
      7. mkdir -p rs2/logs
    2. 配置文件
      1. cd rs0
        1. vi mongo.conf
          dbpath=/guaoran/mongodb/shard/rs0/data
          logpath=/guaoran/mongodb/shard/rs0/logs/mongo.log
          logappend=true
          port=40000
          journal=true
          replSet=shard0 #####该处的命名每台服务名称不一样。其他两台对应shard1、shard2
          shardsvr=true  #############做分片时必须要加的###################
          bind_ip=0.0.0.0

            

      2. cd rs1
        1. vi mongo.conf
          dbpath=/guaoran/mongodb/shard/rs1/data
          logpath=/guaoran/mongodb/shard/rs1/logs/mongo.log
          logappend=true
          port=40001
          journal=true
          replSet=shard0 #####该处的命名每台服务名称不一样。其他两台对应shard1、shard2
          shardsvr=true  #############做分片时必须要加的###################
          bind_ip=0.0.0.0
      3. cd rs2
        1. vi mongo.conf
          dbpath=/guaoran/mongodb/shard/rs2/data
          logpath=/guaoran/mongodb/shard/rs2/logs/mongo.log
          logappend=true
          port=40002
          journal=true
          replSet=shard0 #####该处的命名每台服务名称不一样。其他两台对应shard1、shard2
          shardsvr=true  #############做分片时必须要加的###################
          bind_ip=0.0.0.0
      4. 其他两台基本相同
    3. 启动复制集服务(其他两台基本相同,对应配置中的ip地址修改下即可)
      1. cd /guaoran/mongodb/mongodb-4.0.0/bin/
        
        ./mongod --config /guaoran/mongodb/shard/rs0/mongo.conf &
        ./mongod --config /guaoran/mongodb/shard/rs1/mongo.conf &
        ./mongod --config /guaoran/mongodb/shard/rs2/mongo.conf &
        
      2. 在某一台执行如下配置
        ./mongo --port 40000
        
        var rs_conf={"_id" : "shard0","members" : [{"_id" : 0,"host" : "192.168.20.56:40000",priority:1000},{"_id" : 1,"host" : "192.168.20.56:40001",priority:999},{"_id" : 2,"host" : "192.168.20.56:40002",priority:1,arbiterOnly: true}]}
        
        rs.initiate(rs_conf);
        
        rs.status()

         

      3. 执行rs.status()如下所示代表成功
      4. shard1:PRIMARY> rs.status()
        {
                "set" : "shard1",
                "date" : ISODate("2018-07-24T05:28:20.502Z"),
                "myState" : 1,
                "term" : NumberLong(1),
                "syncingTo" : "",
                "syncSourceHost" : "",
                "syncSourceId" : -1,
                "heartbeatIntervalMillis" : NumberLong(2000),
                "optimes" : {
                        "lastCommittedOpTime" : {
                                "ts" : Timestamp(1532410099, 1),
                                "t" : NumberLong(1)
                        },
                        "readConcernMajorityOpTime" : {
                                "ts" : Timestamp(1532410099, 1),
                                "t" : NumberLong(1)
                        },
                        "appliedOpTime" : {
                                "ts" : Timestamp(1532410099, 1),
                                "t" : NumberLong(1)
                        },
                        "durableOpTime" : {
                                "ts" : Timestamp(1532410099, 1),
                                "t" : NumberLong(1)
                        }
                },
                "lastStableCheckpointTimestamp" : Timestamp(1532410049, 1),
                "members" : [
                        {
                                "_id" : 0,
                                "name" : "192.168.20.57:40000",
                                "health" : 1,
                                "state" : 1,
                                "stateStr" : "PRIMARY",
                                "uptime" : 7037,
                                "optime" : {
                                        "ts" : Timestamp(1532410099, 1),
                                        "t" : NumberLong(1)
                                },
                                "optimeDate" : ISODate("2018-07-24T05:28:19Z"),
                                "syncingTo" : "",
                                "syncSourceHost" : "",
                                "syncSourceId" : -1,
                                "infoMessage" : "",
                                "electionTime" : Timestamp(1532406267, 1),
                                "electionDate" : ISODate("2018-07-24T04:24:27Z"),
                                "configVersion" : 2,
                                "self" : true,
                                "lastHeartbeatMessage" : ""
                        },
                        {
                                "_id" : 1,
                                "name" : "192.168.20.57:40001",
                                "health" : 1,
                                "state" : 2,
                                "stateStr" : "SECONDARY",
                                "uptime" : 3844,
                                "optime" : {
                                        "ts" : Timestamp(1532410089, 1),
                                        "t" : NumberLong(1)
                                },
                                "optimeDurable" : {
                                        "ts" : Timestamp(1532410089, 1),
                                        "t" : NumberLong(1)
                                },
                                "optimeDate" : ISODate("2018-07-24T05:28:09Z"),
                                "optimeDurableDate" : ISODate("2018-07-24T05:28:09Z"),
                                "lastHeartbeat" : ISODate("2018-07-24T05:28:19.475Z"),
                                "lastHeartbeatRecv" : ISODate("2018-07-24T05:28:20.415Z"),
                                "pingMs" : NumberLong(0),
                                "lastHeartbeatMessage" : "",
                                "syncingTo" : "192.168.20.57:40000",
                                "syncSourceHost" : "192.168.20.57:40000",
                                "syncSourceId" : 0,
                                "infoMessage" : "",
                                "configVersion" : 2
                        },
                        {
                                "_id" : 2,
                                "name" : "192.168.20.57:40002",
                                "health" : 1,
                                "state" : 7,
                                "stateStr" : "ARBITER",
                                "uptime" : 3822,
                                "lastHeartbeat" : ISODate("2018-07-24T05:28:19.475Z"),
                                "lastHeartbeatRecv" : ISODate("2018-07-24T05:28:19.477Z"),
                                "pingMs" : NumberLong(0),
                                "lastHeartbeatMessage" : "",
                                "syncingTo" : "",
                                "syncSourceHost" : "",
                                "syncSourceId" : -1,
                                "infoMessage" : "",
                                "configVersion" : 2
                        }
                ],
                "ok" : 1,
                "operationTime" : Timestamp(1532410099, 1),
                "$gleStats" : {
                        "lastOpTime" : Timestamp(0, 0),
                        "electionId" : ObjectId("7fffffff0000000000000001")
                },
                "lastCommittedOpTime" : Timestamp(1532410099, 1),
                "$configServerState" : {
                        "opTime" : {
                                "ts" : Timestamp(1532410094, 2),
                                "t" : NumberLong(1)
                        }
                },
                "$clusterTime" : {
                        "clusterTime" : Timestamp(1532410099, 1),
                        "signature" : {
                                "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                "keyId" : NumberLong(0)
                        }
                }
        }
        shard1:PRIMARY> 

三、分片

  1. 配置集群服务
    1. 创建目录、配置信息(其他两台一模一样配置)
      1. cd /guaoran/mongodb/shard
      2. mkdir configs/data/
      3. mkdir configs/logs/
      4. cd configs
      5. vi cfgServer.conf
        dbpath=/guaoran/mongodb/shard/configs/data
        logpath=/guaoran/mongodb/shard/configs/logs/config.log
        logappend=true
        port=40006
        journal=true
        bind_ip=0.0.0.0
        configsvr=true
        replSet=cfgset
        fork = true

         

    2.  启动服务
      1. cd /guaoran/mongodb/mongodb-4.0.0/bin/
        ./mongod --config /guaoran/mongodb/shard/configs/cfgServer.conf &
        
        

         

      2. 在某一台执行如下命令进行配置

        
        ./mongo --port 40006
        
        rs.initiate({_id:"cfgset",configsvr:true,members:[{_id:1,host:"192.168.20.56:40006"},{_id:2,host:"192.168.20.57:40006"},{_id:3,host:"192.168.20.58:40006"}]})
        
        
        

         

      3. 执行rs.status()查看结果

  2. 配置路由(该处只配置了一台,可以配置多台,配置方式同上面的差不多)
    1. 在192.168.20.56服务器上继续创建目录
    2. cd /guaoran/mongodb/shard
    3. mkdir -p mongos/logs
    4. cd mongos
    5. vi cfg_mongos.conf
      logpath=/guaoran/mongodb/shard/mongos/logs/mongos.log
      logappend=true
      port=40009
      bind_ip=0.0.0.0
      ### 该处的cfgset 为服务的复制集名称
      configdb=cfgset/192.168.20.56:40006,192.168.20.57:40006,192.168.20.58:40006

       

    6. 启动路由服务

      1. cd /guaoran/mongodb/mongodb-4.0.0/bin/
        
        ./mongos --config /guaoran/mongodb/shard/mongos/cfg_mongos.conf &
        
        ./mongo --port 40009
        
        --添加分片
        sh.addShard("shard0/192.168.20.56:40000")
        sh.addShard("shard1/192.168.20.57:40000")

         

      2. 进行测试

        1. 此时使用preson数据库,用student作为collection
        2.  
          mongos> use presonuse preson
          switched to db preson
          mongos> show collections
          mongos> sh.enableSharding("preson");
          {
                  "ok" : 1,
                  "operationTime" : Timestamp(1532411733, 4),
                  "$clusterTime" : {
                          "clusterTime" : Timestamp(1532411733, 4),
                          "signature" : {
                                  "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                  "keyId" : NumberLong(0)
                          }
                  }
          }
          mongos> sh.shardCollection("preson.teacher",{name:"hashed"});sh.shardCollection("preson.teacher",{name:"hashed"});
          {
                  "collectionsharded" : "preson.teacher",
                  "collectionUUID" : UUID("eaa495c4-f104-4d9c-92fa-21f3f0c7fbf7"),
                  "ok" : 1,
                  "operationTime" : Timestamp(1532411741, 54),
                  "$clusterTime" : {
                          "clusterTime" : Timestamp(1532411741, 62),
                          "signature" : {
                                  "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                                  "keyId" : NumberLong(0)
                          }
                  }
          }
          mongos> for (var i=0;i<100;i++) db.teacher.insert({name:"zs"+i,age:i});for (var i=0;i<100;i++) db.teacher.insert({name:"zs"+i,age:i});
          WriteResult({ "nInserted" : 1 })
          mongos> 
          show collectionsshow collections
          teacher
          mongos> db.teacher.find()
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c1"), "name" : "zs5", "age" : 5 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c2"), "name" : "zs6", "age" : 6 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c3"), "name" : "zs7", "age" : 7 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c4"), "name" : "zs8", "age" : 8 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c6"), "name" : "zs10", "age" : 10 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c7"), "name" : "zs11", "age" : 11 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c8"), "name" : "zs12", "age" : 12 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2cc"), "name" : "zs16", "age" : 16 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d1"), "name" : "zs21", "age" : 21 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d2"), "name" : "zs22", "age" : 22 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d3"), "name" : "zs23", "age" : 23 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d5"), "name" : "zs25", "age" : 25 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d6"), "name" : "zs26", "age" : 26 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d8"), "name" : "zs28", "age" : 28 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d9"), "name" : "zs29", "age" : 29 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2db"), "name" : "zs31", "age" : 31 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2dd"), "name" : "zs33", "age" : 33 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2de"), "name" : "zs34", "age" : 34 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2e1"), "name" : "zs37", "age" : 37 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2e2"), "name" : "zs38", "age" : 38 }
          Type "it" for more
          mongos> 

           

        3. 此时发现数据是出来了, 不过,数据顺序好像是有点乱,不过没事,这是因为分片的机制是通过name是hash进行分片的原因,
        4. 于是我们分别连上192.168.20.56:40000和192.168.20.57:40000  进行测试下,
        5. 56:
          ./mongo 192.168.20.56:40000
          
          
          shard0:PRIMARY> show dbs
          admin    0.000GB
          config   0.000GB
          local    0.000GB
          preson   0.000GB
          shard0:PRIMARY> use preson
          switched to db preson
          shard0:PRIMARY> show collections
          teacher
          shard0:PRIMARY> db.teacher.find()
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c1"), "name" : "zs5", "age" : 5 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c2"), "name" : "zs6", "age" : 6 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c3"), "name" : "zs7", "age" : 7 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c4"), "name" : "zs8", "age" : 8 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c6"), "name" : "zs10", "age" : 10 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c7"), "name" : "zs11", "age" : 11 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c8"), "name" : "zs12", "age" : 12 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2cc"), "name" : "zs16", "age" : 16 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d1"), "name" : "zs21", "age" : 21 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d2"), "name" : "zs22", "age" : 22 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d3"), "name" : "zs23", "age" : 23 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d5"), "name" : "zs25", "age" : 25 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d6"), "name" : "zs26", "age" : 26 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d8"), "name" : "zs28", "age" : 28 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d9"), "name" : "zs29", "age" : 29 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2db"), "name" : "zs31", "age" : 31 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2dd"), "name" : "zs33", "age" : 33 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2de"), "name" : "zs34", "age" : 34 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2e1"), "name" : "zs37", "age" : 37 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2e2"), "name" : "zs38", "age" : 38 }
          Type "it" for more
          shard0:PRIMARY> 

          发现此时的数据是从5开始的。

        6. 57:
          ./mongo 192.168.20.57:40000
          
          shard1:PRIMARY> show dbsshow dbs
          admin    0.000GB
          config   0.000GB
          local    0.000GB
          preson   0.000GB
          shard1:PRIMARY> use preson
          switched to db preson
          shard1:PRIMARY> show collections
          teacher
          shard1:PRIMARY> db.teacher.find()
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2bc"), "name" : "zs0", "age" : 0 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2bd"), "name" : "zs1", "age" : 1 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2be"), "name" : "zs2", "age" : 2 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2bf"), "name" : "zs3", "age" : 3 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c0"), "name" : "zs4", "age" : 4 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c5"), "name" : "zs9", "age" : 9 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2c9"), "name" : "zs13", "age" : 13 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2ca"), "name" : "zs14", "age" : 14 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2cb"), "name" : "zs15", "age" : 15 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2cd"), "name" : "zs17", "age" : 17 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2ce"), "name" : "zs18", "age" : 18 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2cf"), "name" : "zs19", "age" : 19 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d0"), "name" : "zs20", "age" : 20 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d4"), "name" : "zs24", "age" : 24 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2d7"), "name" : "zs27", "age" : 27 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2da"), "name" : "zs30", "age" : 30 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2dc"), "name" : "zs32", "age" : 32 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2df"), "name" : "zs35", "age" : 35 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2e0"), "name" : "zs36", "age" : 36 }
          { "_id" : ObjectId("5b56bf6a05fb8df7a5b7e2e5"), "name" : "zs41", "age" : 41 }
          Type "it" for more
          shard1:PRIMARY> 

          此时数据是从0开始的

        7. 通过上面两个测试,会发现,shard0中缺少的数据正好在shard1中,shard1中存在的数据也正好是shard0中缺少的。至此实现了分片的机制。服务搭建完成。

 

 

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值