mongodb的3台服务器shard集群搭建

最新推荐文章于 2021-11-30 13:21:47 发布

justdoit180

最新推荐文章于 2021-11-30 13:21:47 发布

阅读量637

点赞数 1

分类专栏： mongodb 文章标签： mongodb的shard集群的搭建

本文链接：https://blog.csdn.net/weixin_43865381/article/details/103513579

版权

mongodb 专栏收录该内容

3 篇文章 0 订阅

订阅专栏

1.硬件条件

53~55 共3台服务器，操作系统均是centos 7.5，
服务器相关配置修改：

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag
echo 0 > /proc/sys/vm/zone_reclaim_mode

1.1搭建规划说明

可部署成3个shard（每个shard采用2个副本集+1个仲裁），
3个config server和1个mongos，具体结构如下：
198.16.8.53 mongod(shard1_1 27101)
mongod(shard3_2 27103)
mongod(shard2_3 arbiter)
mongod(config server 30000)
mongos(27001)
198.16.8.54 mongod(shard1_2 27101)
mongod(shard2_1 27102)
mongod(shard3_3 27103 arbiter)
mongod(config server 30000)
198.16.8.55 mongod(shard3_1 27103)
mongod(shard2_2 27102)
mongod(shard1_3 27101 arbiter)
mongod(config server 30000)

shard1: 53(shard1_1)、54(shard1_2)、55(shard1_3 arbiter)
shard2: 54(shard2_1)、55(shard2_2)、53(shard2_3 arbiter)
shard3: 55(shard3_1)、53(shard3_2)、54(shard3_3 arbiter)
config server: 53(30000)、54(30000)、55(30000) //config server的个数只能是奇数个（目前根据服务器台数只能选择1个或者3个）
mongos: 53(40000)

2.软件准备

下载mongodb 3.4.4,然后解压后如下：
[test@localhost mongodb-linux-x86_64-3.4.4]$ ls
bin  GNU-AGPL-3.0  README  THIRD-PARTY-NOTICES
在目录下新建data、conf、log三个目录，如下：
[test@localhost mongodb-linux-x86_64-3.4.4]$ ls
bin  conf  data  GNU-AGPL-3.0  log  README  THIRD-PARTY-NOTICES
在data目录下新建configsvr、sd1、sd2、sd3、sd4 5个文件夹，如下
configsvr  sd1  sd2  sd3  sd4
在conf目录下新建5个配置文件，configsvr.conf,mongos.conf，sd1.conf,sd2.conf,sd3.conf
[test@localhost conf]$ ls
configsvr.conf  mongos.conf  sd1.conf  sd2.conf  sd3.conf

配置文件内容如下：
sd1.conf
dbpath=/data/mongodb-linux-x86_64-3.4.4/data/sd1/
#where to log
logpath=/data/mongodb-linux-x86_64-3.4.4/log/sd1.log
logappend=true
fork=true
port=27101
replSet=sd1
shardsvr=true
oplogSize=2028
rest=true
httpinterface=true
#Add 2014-12-15, use wiredTiger as storageEngine(default is MMAP v1)
storageEngine=wiredTiger
#wiredTigerDirectoryForIndexes=true
wiredTigerCacheSizeGB=30

sd2.conf
dbpath=/data/mongodb-linux-x86_64-3.4.4/data/sd2/
#where to log
logpath=/data/mongodb-linux-x86_64-3.4.4/log/sd2.log
logappend=true
fork=true
port=27102
replSet=sd2
shardsvr=true
oplogSize=2028
rest=true
httpinterface=true
#Add 2014-12-15, use wiredTiger as storageEngine(default is MMAP v1)
storageEngine=wiredTiger
#wiredTigerDirectoryForIndexes=true
wiredTigerCacheSizeGB=30

sd3.conf
dbpath=/data/mongodb-linux-x86_64-3.4.4/data/sd3/
#where to log
logpath=/data/mongodb-linux-x86_64-3.4.4/log/sd3.log
logappend=true
fork=true
port=27103
replSet=sd3
shardsvr=true
oplogSize=2028
rest=true
httpinterface=true
#Add 2014-12-15, use wiredTiger as storageEngine(default is MMAP v1)
storageEngine=wiredTiger
#wiredTigerDirectoryForIndexes=true
wiredTigerCacheSizeGB=30


configsvr.conf
dbpath=/data/mongodb-linux-x86_64-3.4.4/data/configsvr/
logpath=/data/mongodb-linux-x86_64-3.4.4/log/configsvr.log
logappend=true
fork=true
port=30000
configsvr=true
replSet=cs
#Add 2014-12-15, use wiredTiger as storageEngine(default is MMAP v1)
storageEngine=wiredTiger
#wiredTigerDirectoryForIndexes=true
wiredTigerCacheSizeGB=10

mongos.conf
# mongos file
configdb=cs/198.16.8.53:30000,198.16.8.54:30000,198.16.8.55:30000
port=40000
logpath=/data/mongodb-linux-x86_64-3.4.4/log/mongos.log
logappend=true
fork=true

至此软件就准备好了，将整个mongodb-linux-x86_64-3.4.4文件夹都拷贝到53~55 3台机器上。

3.mongodb的配置启动

启动各个shard的mongod服务
进入mongodb-linux-x86_64-3.4.4文件夹，
198.16.8.53、54、55上分别运行如下命令：
numactl --interleave=all ./bin/mongod -f conf/sd1.conf
numactl --interleave=all ./bin/mongod -f conf/sd2.conf
numactl --interleave=all ./bin/mongod -f conf/sd3.conf

副本集配置
配置副本集sd1：
在53/54/55中任意一台机器均可，运行如下命令：

./bin/mongo --port 27101
>use admin			//切到admin
>config = {_id:"sd1", members: [{_id: 0, host:"198.16.8.53:27101"},{_id: 1, host:"198.16.8.54:27101"},{_id: 2, host:"198.16.8.55:27101","arbiterOnly":true}]};
执行完这条命令的返回结果如下：
{
        "_id" : "sd1",
        "members" : [
                {
                        "_id" : 0,
                        "host" : "198.16.8.53:27101"
                },
                {
                        "_id" : 1,
                        "host" : "198.16.8.54:27101"
                },
                {
                        "_id" : 2,
                        "host" : "198.16.8.55:27101",
                        "arbiterOnly" : true
                }
        ]
}

上面配置语句只是指定了仲裁节点，而没有指定主节点，至于谁作为主节点就交由选举决定。当然我们也可以通过指定优先级的方式来指定主节点，如下：
config = {_id:"sd1", members: [{_id: 0, host:"198.16.8.53:27101",priority:2},{_id: 1, host:"198.16.8.54:27101",priority:1},{_id: 2, host:"198.16.8.55:27101","arbiterOnly":true}]};
优先级高的节点将优先作为主节点。

>rs.initiate(config)
{ "ok" : 1 }	//此返回结果代表初始化成功

配置完之后可以用 rs.config() / rs.status() 查看副本集配置信息和状态
sd1:OTHER> rs.status()
{
        "set" : "sd1",
        "date" : ISODate("2019-09-22T09:55:47.469Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "name" : "198.16.8.53:27101",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 780,
                        "optime" : Timestamp(1442915733, 1),
                        "optimeDate" : ISODate("2019-09-22T09:55:33Z"),
                        "electionTime" : Timestamp(1442915734, 1),
                        "electionDate" : ISODate("2019-09-22T09:55:34Z"),
                        "configVersion" : 1,
                        "self" : true
                },
                {
                        "_id" : 1,
                        "name" : "198.16.8.54:27101",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 14,
                        "optime" : Timestamp(1442915733, 1),
                        "optimeDate" : ISODate("2019-09-22T09:55:33Z"),
                        "lastHeartbeat" : ISODate("2019-09-22T09:55:46.585Z"),
                        "lastHeartbeatRecv" : ISODate("2019-09-22T09:55:46.556Z"),
                        "pingMs" : 0,
                        "lastHeartbeatMessage" : "could not find member to sync from",
                        "configVersion" : 1
                },
                {
                        "_id" : 2,
                        "name" : "198.16.8.55:27101",
                        "health" : 1,
                        "state" : 7,
                        "stateStr" : "ARBITER",
                        "uptime" : 14,
                        "lastHeartbeat" : ISODate("2019-09-22T09:55:46.585Z"),
                        "lastHeartbeatRecv" : ISODate("2019-09-22T09:55:46.353Z"),
                        "pingMs" : 0,
                        "configVersion" : 1
                }
        ],
        "ok" : 1
}


sd1:OTHER> rs.config()

{
        "_id" : "sd1",
        "version" : 1,
        "members" : [
                {
                        "_id" : 0,
                        "host" : "198.16.8.53:27101",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : 0,
                        "votes" : 1
                },
                {
                        "_id" : 1,
                        "host" : "198.16.8.54:27101",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : 0,
                        "votes" : 1
                },
                {
                        "_id" : 2,
                        "host" : "198.16.8.55:27101",
                        "arbiterOnly" : true,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : 0,
                        "votes" : 1
                }
        ],
        "settings" : {
                "chainingAllowed" : true,
                "heartbeatTimeoutSecs" : 10,
                "getLastErrorModes" : {

                },
                "getLastErrorDefaults" : {
                        "w" : 1,
                        "wtimeout" : 0
                }
        }
}

配置副本集sd2：
在53/54/55中任意一台机器上运行如下命令：

./bin/mongo --port 27102
>use admin			//切到admin
>config = {_id:"sd2", members: [{_id: 0, host:"198.16.8.54:27102"},{_id: 1, host:"198.16.8.55:27102"},{_id: 2, host:"198.16.8.53:27102","arbiterOnly":true}]};

指定优先级的方式：
config = {_id:"sd2", members: [{_id: 0, host:"198.16.8.54:27102",priority:2},{_id: 1, host:"198.16.8.55:27102",priority:1},{_id: 2, host:"198.16.8.53:27102","arbiterOnly":true}]};

>rs.initiate(config)

同样可通过命令rs.config()查看副本集状态

配置副本集sd3：
在53/54/55中任意一台机器上运行如下命令：

./bin/mongo --port 27103
>use admin			//切到admin
>config = {_id:"sd3", members: [{_id: 0, host:"198.16.8.55:27103"},{_id: 1, host:"198.16.8.53:27103"},{_id: 2, host:"198.16.8.54:27103","arbiterOnly":true}]};
>rs.initiate(config)
>

配置configserver（config server的数量只能为1或3）
在198.16.8.53/54/55上运行如下命令：

numactl --interleave=all ./bin/mongod -f conf/configsvr.conf

在53/54/55任何一台机器上运行如下命令，进行配置
./bin/mongo --port 30000
>use admin
>config = {_id:"cs",configsvr:true, members:[{_id:0,host:"198.16.8.53:30000"},{_id:1,host:"198.16.8.54:30000"},{_id:2,host:"198.16.8.55:30000"}]}
>rs.initiate(config)

启动mongos
在53上运行如下命令：

numactl --interleave=all ./bin/mongos -f conf/mongos.conf

配置shard
登陆mongos所在服务器，连接到mongos：
./bin/mongo --port 40000
mongos> use admin		//切到admin
//加入shards， maxsize单位为M，指定了shard可使用的最大磁盘空间，122880代表120G（可根据服务器磁盘容量调整）， name用于指定每个shard的名字（需要和配置文件中一致）
mongos> db.runCommand({addshard:"sd1/198.16.8.53:27101,198.16.8.54:27101,198.16.8.55:27101",name:"sd1", maxSize:122880})
{ "shardAdded" : "sd1", "ok" : 1 }
mongos> db.runCommand({addshard:"sd2/198.16.8.54:27102,198.16.8.55:27102,198.16.8.53:27102",name:"sd2", maxSize:122880})
{ "shardAdded" : "sd2", "ok" : 1 }
mongos> db.runCommand({addshard:"sd3/198.16.8.55:27103,198.16.8.53:27103,198.16.8.54:27103",name:"sd3", maxSize:122880})
{ "shardAdded" : "sd3", "ok" : 1 }

mongos> db.runCommand({listshards : 1 })	//检查分片添加情况，如出现以下结果则表示配置成功
{
        "shards" : [
                {
                        "_id" : "sd1",
                        "host" : "sd1/198.16.8.53:27101,198.16.8.54:27101",
                        "maxSize" : NumberLong(122880)
                },
                {
                        "_id" : "sd2",
                        "host" : "sd2/198.16.8.54:27102,198.16.8.55:27102",
                        "maxSize" : NumberLong(122880)
                },
                {
                        "_id" : "sd3",
                        "host" : "sd3/198.16.8.53:27103,198.16.8.55:27103",
                        "maxSize" : NumberLong(122880)
                }
        ],
        "ok" : 1
}

到此集群配置完成，接下来就是新建用户，建库，建表，分库，分表，然后就可以进行查询插入操作了

justdoit180

关注

1
点赞
踩
4

收藏

觉得还不错? 一键收藏
0
评论
mongodb的3台服务器shard集群搭建

1.硬件条件53~55 共3台服务器，操作系统均是centos 7.5，服务器相关配置修改：echo never > /sys/kernel/mm/transparent_hugepage/enabledecho never > /sys/kernel/mm/transparent_hugepage/defragecho 0 > /proc/sys/vm/zone_re...
复制链接

扫一扫