概述
MongoDB分片是使用多个服务器存储数据的方法,以支持巨大的数据存储和对数据进行操作。分片技术可以满足MongoDB数据量大量增长的需求,当一台MongoDB服务器不足以存储海量数据或不足以提供可接受的读写吞吐量时,我们就可以在多台服务器上分割数据,使得数据库系统能存储和处理更多的数据。
MongoDB分片群集的组成
- Shard:分片服务器,用于存储实际的数据块,实际生产环境中一个shard server角色可由几台服务器组成一个Replica Set承担,防止主机单点故障。
- Config Server:配置服务器,存储了整个分片群集的配置信息,其中包括chunk信息。
- Routers:路由器,客户端由此接入,且让整个群集看上去像单一数据库,前端应用可以透明使用。
部署MongoDB分片群集
实验环境:
- 1台路由实例(端口27017)
- 1台配置实例(端口37017)
- 2台shard实例(端口47017,47018)
1.安装MongoDB3.2
[root@localhost ~]# yum install openssl-devel -y
[root@localhost tomcat]# tar zxvf mongodb-linux-x86_64-3.2.1.tgz -C /opt/
2.创建4个实例
[root@localhost opt]# mkdir -p /data/mongodb/mongodb{1,2,3,4}
[root@localhost mongodb]# mkdir logs
[root@localhost mongodb]# touch logs/mongodb{1,2,3,4}.log
[root@localhost logs]# chmod -R 777 *.log
[root@localhost logs]# ulimit -n 25000 //最大进程数//
[root@localhost logs]# ulimit -u 25000 //最大文件数//
3.部署配置服务器
[root@localhost bin]# vim mongodb1.conf
port=37017
dbpath=/data/mongodb/mongodb1
logpath=/data/mongodb/logs/mongodb1.log
logappend=true
fork=true
maxConns=5000
storageEngine=mmapv1
configsvr=true
~
1)某节点内存不足时,从其他节点分配内存
[root@localhost bin]# sysctl -w vm.zone_reclaim_mode=0 //内核参数为0的话,那么系统会倾向于从其他节点分配内存//
[root@localhost bin]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[root@localhost bin]# echo never > /sys/kernel/mm/transparent_hugepage/defrag
[root@localhost bin]# ln -s /usr/local/mongodb/bin/mongo /usr/bin/mongo
[root@localhost bin]# ln -s /usr/local/mongodb/bin/mongod /usr/bin/mongod
2)启动配置服务器
[root@localhost bin]# mongod -f mongodb1.conf
4.配置分片服务器
[root@localhost bin]# vim mongodb2.conf
port=47017
dbpath=/data/mongodb/mongodb2
logpath=/data/mongodb/logs/mongodb2.log
logappend=true
fork=true
maxConns=5000
storageEngine=mmapv1
shardsvr=true
[root@localhost bin]# vim mongodb3.conf
port=47018
dbpath=/data/mongodb/mongodb3
logpath=/data/mongodb/logs/mongodb3.log
logappend=true
fork=true
maxConns=5000
storageEngine=mmapv1
shardsvr=true
1)启动mongodb2、mongodb3
[root@localhost bin]#mongod -f mongodb2.conf
[root@localhost bin]#mongod -f mongodb3.conf
5.启动路由服务器
[root@localhost bin]# ./mongos --port 27017 --fork --logpath=/usr/local/mongodb/bin/route.log --configdb 192.168.126.141:37017 --chunkSize 1
************显示*******************************************
2018-07-18T09:21:53.507+0800 W SHARDING [main] Running a sharded cluster with fewer than 3 config servers should only be done for testing purposes and is not recommended for production.
about to fork child process, waiting until server is ready for connections.
forked process: 3580
child process started successfully, parent exiting
6.启用分片服务器
[root@localhost bin]# mongo
mongos> show dbs
config 0.031GB
mongos> sh.status() //#shards下为空,没有分片服务器//
mongos> sh.addShard("192.168.126.204:47017")
mongos> sh.addShard("192.168.126.204:47018")
mongos> sh.status()
shards:
{ "_id" : "shard0000", "host" : "192.168.126.141:47017" }
{ "_id" : "shard0001", "host" : "192.168.126.141:47018" }
1)创建Kgc数据库,创建users集合并且插入100000条数据
mongos> use kgc
mongos> db.createCollection('users')
mongos> for(var i=1;i<=100000;i++)db.users.insert({"id":1,"name":"jack"+i})
mongos> db.users.find()
{ "_id" : ObjectId("5b4e9b580f25d0730817aea1"), "id" : 1, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b580f25d0730817aea2"), "id" : 2, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b580f25d0730817aea3"), "id" : 3, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b580f25d0730817aea4"), "id" : 4, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b590f25d0730817aea5"), "id" : 5, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b590f25d0730817aea6"), "id" : 6, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b590f25d0730817aea7"), "id" : 7, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b590f25d0730817aea8"), "id" : 8, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b590f25d0730817aea9"), "id" : 9, "name" : "jack1" }
{ "_id" : ObjectId("5b4e9b590f25d0730817aeaa"), "id" : 10, "name" : "jack1" }
..........//省略//
7.实现分片功能
[root@localhost bin]# ./mongoimport -d kgc -c users --file /opt/testdb.txt
mongos> show dbs
config 0.031GB
kgc 0.078GB
mongos> use kgc
switched to db kgc
mongos> show tables
system.indexes
users
mongos> sh.status() //查看数据库分片信息//
databases:
{ "_id" : "kgc", "primary" : "shard0000", "partitioned" : false } //数据库尚未分片//
1)启用数据库分片
mongos> sh.enableSharding("kgc") //启用数据库分片//
mongos> sh.status()
databases:
{ "_id" : "kgc", "primary" : "shard0000", "partitioned" : true }
mongos> db.users.createIndex({"id":1}) //对users表创建索引//
mongos> sh.shardCollection("kgc.users",{"id":1}) //表分片//
mongos> sh.status()
shards:
{ "_id" : "shard0000", "host" : "192.168.126.141:47017" }
{ "_id" : "shard0001", "host" : "192.168.126.141:47018" }
转载于:https://blog.51cto.com/13642258/2149299