单机部署
运行docker命令创建mongo容器即可
docker run -d \
--restart=always \
--name mongo-single \
-p 27097:27017 \
-v /app/mongo/mongo-single/data:/data/db \
-e MONGO_INITDB_ROOT_USERNAME=admin \
-e MONGO_INITDB_ROOT_PASSWORD=Wq@123456 \
mongo:latest --auth
主从集群
1、准备工作
(1)创建docker网络
docker network create cluster
(2)使用openssl创建集群认证的秘钥
# 创建秘钥
openssl rand -base64 756 > keyfile.key
# 给秘钥创建赋予权限
chmod 600 keyfile.key
# 修改秘钥文件组
chown 999:999 keyfile.key
可能会遇到的问题:
(1)启动容器报错 bad file
,可能是由于秘钥文件没有权限或者组权限不足,需要执行赋予权限以及分配组,权限要 600 ,太高也会报错
2、运行docker命令
sudo docker run -d \
--name mongo1 \
--network cluster \
-p 27017:27017 \
-e MONGO_INITDB_ROOT_USERNAME=root \
-e MONGO_INITDB_ROOT_PASSWORD=123456 \
-v /app/mongo/mongo1/data:/data/db \
-v /app/mongo/mongo1/conf:/data/configdb \
-v /app/mongo/mongo1/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet rs0 --bind_ip_all --keyFile /data/configdb/mongoKeyfile.key --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo2 \
--network cluster \
-p 27018:27017 \
-e MONGO_INITDB_ROOT_USERNAME=root \
-e MONGO_INITDB_ROOT_PASSWORD=123456 \
-v /app/mongo/mongo2/data:/data/db \
-v /app/mongo/mongo2/conf:/data/configdb \
-v /app/mongo/mongo2/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet rs0 --bind_ip_all --keyFile /data/configdb/mongoKeyfile.key --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo3 \
--network cluster \
-p 27019:27017 \
-e MONGO_INITDB_ROOT_USERNAME=root \
-e MONGO_INITDB_ROOT_PASSWORD=123456 \
-v /app/mongo/mongo3/data:/data/db \
-v /app/mongo/mongo3/conf:/data/configdb \
-v /app/mongo/mongo3/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet rs0 --bind_ip_all --keyFile /data/configdb/mongoKeyfile.key --wiredTigerCacheSizeGB 1
注意点:
mongoKeyfile.key
秘钥文件需要同一份,可创建完成后复制到每个节点的配置目录下,例如:/app/mongo/mongo3/conf
--keyFile /data/configdb/mongoKeyfile.key
中的/data/configdb/mongoKeyfile.key
目录需要填写的是容器内的目录,宿主机上的目录容器访问不到
3、初始化副本集
(1)进入任一节点容器中
docker exec -it mongo1 bash
(2)查询MongoDB的命令所在目录(容器中的目录)
whereis mongosh
(3)使用命令进入MongoDB
mongosh
(4)使用Mongo命令初始化副本集
//验证身份
use admin;
db.auth("root", "123456");
//初始化副本
rs.initiate({
_id: "rs0",
members: [
{ _id: 0, host: "192.168.2.17:27017" },
{ _id: 1, host: "192.168.2.17:27018" },
{ _id: 2, host: "192.168.2.17:27019" }
]
});
//查看副本状态
rs.status();
//切换到对应的库
use boatol;
//给对应的库添加可读写权限的用户
db.createUser(
{
user: "admin",
pwd: "123456",
roles: [ { role: "readWrite", db: "boatol" } ]
}
);
问题:
1、BadValue: security.keyFile is required when authorization is enabled with replica sets
开启认证后,秘钥文件可能读取不到或者权限不足,检查秘钥文件的所在目录,也可创建一个空的测试容器,挂载当前目录看看是否将秘钥文件挂载到容器中的对应目录下
分片集群
1、概念
将原来容器重新部署为分片集群,分片集群的三要素
-
分片服务器 (Shard Servers)
分片服务器存储实际的数据,每个分片服务器都是一个独立的MongoDB实例。数据被分片后,每个分片服务器只存储部分数据。分片的目的是为了在多个服务器上分布数据和负载,以提高数据库的读写性能和存储容量。
在分片架构中,每个分片可以是一个独立的MongoDB服务器或副本集。通过水平分片(sharding),数据被分割成更小的部分,并分布到多个分片服务器上。
-
配置服务器 (Config Servers)
配置服务器存储集群的元数据,包括分片配置信息和分片的分布情况。元数据包括每个分片存储的数据范围、每个分片的服务器信息等。
在分片集群中,配置服务器是不可或缺的,集群中的每个组件都需要查询配置服务器以了解数据的分布。配置服务器通常是一个由三个服务器组成的副本集,以确保元数据的高可用性和一致性。
-
路由器 (Mongos)
Mongos是MongoDB的路由器组件,负责接收应用程序的查询请求并将它们路由到正确的分片服务器。Mongos查询配置服务器以了解数据的分布情况,并根据元数据将查询请求分发到相应的分片服务器。
Mongos本身不存储数据,它是一个中间层,提供一个统一的接口,让应用程序可以透明地访问分片数据。Mongos将查询结果从各个分片服务器汇总并返回给应用程序。
分片的工作流程
1、客户端应用程序向Mongos路由器发送查询请求。
2、Mongos路由器查询配置服务器以了解数据的分布情况。
3、 Mongos路由器将查询请求分发到相应的分片服务器。
4、分片服务器处理查询并返回结果给Mongos路由器。
5、 Mongos路由器汇总查询结果并返回给客户端应用程序。
2、停止容器
将三台容器全部停止
docker stop mongo1 mongo2 mongo3
# 移除三个容器,但不删除数据
docker rm -f mongo1 mongo2 mongo3
3、重新配置并启动容器
(1)配置服务器 (Config Servers)
先执行命令创建容器
docker run -d \
--name mongo-config1 \
--network cluster \
-p 27039:27019 \
-v /app/mongo-cluster/mongo-config1/data:/data/db \
-v /app/mongo-cluster/mongo-config1/configdb:/data/configdb \
-v /app/mongo-cluster/mongoKeyfile.key:/data/mongoKeyfile.key \
mongo:latest --replSet configReplSet --configsvr --keyFile /data/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo-config2 \
--network cluster \
-p 27049:27019 \
-v /app/mongo-cluster/mongo-config2/data:/data/db \
-v /app/mongo-cluster/mongo-config2/configdb:/data/configdb \
-v /app/mongo-cluster/mongoKeyfile.key:/data/mongoKeyfile.key \
mongo:latest --replSet configReplSet --configsvr --keyFile /data/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo-config3 \
--network cluster \
-p 27059:27019 \
-v /app/mongo-cluster/mongo-config3/data:/data/db \
-v /app/mongo-cluster/mongo-config3/configdb:/data/configdb \
-v /app/mongo-cluster/mongoKeyfile.key:/data/mongoKeyfile.key \
mongo:latest --replSet configReplSet --configsvr --keyFile /data/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
进入其中一个容器内,初始化集群副本集
# 进入容器内部
docker exec -it mongo-config1 bash
# 进入数据库,初始化副本集,默认配置服务器集群的端口为27019
./usr/bin/mongosh --port 27019
初始化配置服务器的副本集
//初始化副本集
rs.initiate({
_id: "configReplSet",
configsvr: true,
members: [
{ _id: 0, host: "mongo-config1:27019" },
{ _id: 1, host: "mongo-config2:27019" },
{ _id: 2, host: "mongo-config3:27019" }
]
})
//查询主节点
rs.status();
//进入主节点添加用户
db.getSiblingDB("admin").createUser({
user: "admin",
pwd: "Wq@123456",
roles: [{ role: "root", db: "admin" }]
})
use admin;
db.auth("admin","Wq@123456");
注意点:
--configsvr
配置的默认端口为27019
(2)配置分片服务器
第一个分片副本
配置为分片服务器,主要负责存储数据
docker run -d \
--name mongo-shard1 \
--network cluster \
-p 27067:27018 \
-v /app/mongo/mongo-shard1/data:/data/db \
-v /app/mongo/mongo-shard1/configdb:/data/configdb \
-v /app/mongo/mongo-shard1/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet shardSet1 --shardsvr --keyFile /data/configdb/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo-shard2 \
--network cluster \
-p 27077:27018 \
-v /app/mongo/mongo-shard2/data:/data/db \
-v /app/mongo/mongo-shard2/configdb:/data/configdb \
-v /app/mongo/mongo-shard2/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet shardSet1 --shardsvr --keyFile /data/configdb/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo-shard3 \
--network cluster \
-p 27087:27018 \
-v /app/mongo/mongo-shard3/data:/data/db \
-v /app/mongo/mongo-shard3/configdb:/data/configdb \
-v /app/mongo/mongo-shard3/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet shardSet1 --shardsvr --keyFile /data/configdb/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
进入其中一个容器内,初始化集群副本集
# 进入容器内部
docker exec -it mongo-shard1 bash
# 进入数据库,初始化副本集,默认分片集群的端口为27018
./usr/bin/mongosh --port 27018
初始化副本集
//初始化分片副本集
rs.initiate({
_id: "shardSet1",
members: [
{ _id: 0, host: "mongo-shard1:27018" },
{ _id: 1, host: "mongo-shard2:27018" },
{ _id: 2, host: "mongo-shard3:27018" }
]
});
//初始化完成后,查询主节点,并切换到主节点添加用户认证
rs.status();
//添加用户
db.getSiblingDB("admin").createUser({
user: "admin",
pwd: "Wq@123456",
roles: [{ role: "root", db: "admin" }]
});
use admin;
db.auth("admin","Wq@123456");
第二个分片副本
docker run -d \
--name mongo-shard4 \
--network cluster \
-p 27066:27018 \
-v /app/mongo/mongo-shard4/data:/data/db \
-v /app/mongo/mongo-shard4/configdb:/data/configdb \
-v /app/mongo/mongo-shard4/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet shardSet2 --shardsvr --keyFile /data/configdb/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo-shard5 \
--network cluster \
-p 27076:27018 \
-v /app/mongo/mongo-shard5/data:/data/db \
-v /app/mongo/mongo-shard5/configdb:/data/configdb \
-v /app/mongo/mongo-shard5/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet shardSet2 --shardsvr --keyFile /data/configdb/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
docker run -d \
--name mongo-shard6 \
--network cluster \
-p 27086:27018 \
-v /app/mongo/mongo-shard6/data:/data/db \
-v /app/mongo/mongo-shard6/configdb:/data/configdb \
-v /app/mongo/mongo-shard6/conf/mongoKeyfile.key:/data/configdb/mongoKeyfile.key \
mongo:latest --replSet shardSet2 --shardsvr --keyFile /data/configdb/mongoKeyfile.key --auth --wiredTigerCacheSizeGB 1
进入容器
# 进入容器内部
docker exec -it mongo-shard4 bash
# 进入数据库,初始化副本集,默认分片集群的端口为27018
./usr/bin/mongosh --port 27018
初始化副本集
//初始化分片副本集
rs.initiate({
_id: "shardSet2",
members: [
{ _id: 0, host: "mongo-shard4:27018" },
{ _id: 1, host: "mongo-shard5:27018" },
{ _id: 2, host: "mongo-shard6:27018" }
]
});
//初始化完成后,查询主节点,并切换到主节点添加用户认证
rs.status();
//添加用户
db.getSiblingDB("admin").createUser({
user: "admin",
pwd: "Wq@123456",
roles: [{ role: "root", db: "admin" }]
});
use admin;
db.auth("admin","Wq@123456");
分片副本可以在后续添加
注意点:
--shardsvr
: 默认端口为27018
4、Mongos路由容器
创建一个新的容器,充当路由器,并连接到配置服务器,其中配置的ip地址为各个配置服务器
docker run -d \
--name mongos \
--network cluster \
--restart=true \
-p 27017:27017 \
-e MONGO_INITDB_ROOT_USERNAME=admin \
-e MONGO_INITDB_ROOT_PASSWORD=gxmnDb358597 \
-v /app/mongo-cluster/mongoKeyfile.key:/data/mongoKeyfile.key \
mongo:latest mongos --bind_ip_all --keyFile /data/mongoKeyfile.key --configdb configReplSet/mongo-config1:27019,mongo-config2:27019,mongo-config3:27019
进入容器内部,配置分片服务器
# 进入容器内部
docker exec -it mongos bash
# 进入数据库,初始化副本集
./usr/bin/mongosh --port 27017
进入数据库后,验证用户
//切换数据库并验证用户
use admin;
db.auth("admin","Wq@123456");
//添加分片集群
sh.addShard("shardSet1/mongo-shard1:27018,mongo-shard2:27018,mongo-shard3:27018")
sh.addShard("shardSet2/mongo-shard4:27018,mongo-shard5:27018,mongo-shard6:27018")
db.getSiblingDB("admin").createUser({
user: "admin",
pwd: "Wq@123456",
roles: [{ role: "root", db: "admin" }]
});
//查看分片状态
sh.status();
5、数据分片
进入路由服务器的数据库
# 进入容器内部
docker exec -it mongos bash
# 进入数据库,初始化副本集
./usr/bin/mongosh --port 27017
指定需要分片的库
use admin;
db.auth("admin","Wq@123456");
use boatol;
//给对应的库加上用户以及读写权限
db.createUser({
user: "admin",
pwd: "Wq@123456",
roles: [
{
role: "readWrite",
db: "boatol"
}
]
});
//查看当前库的用户
db.getUsers();
//创建索引
db.motorHistoricalTrack.createIndex({ motorDeviceSn: "hashed", trackTime:1 });
//指定需要分片的库,库名为 boatol
sh.enableSharding("boatol");
//指定分片的减名,boatol为库名,motorHistoricalTrack为文档名,motorDeviceSn为分片的键名,使用哈希分片
sh.shardCollection("boatol.motorHistoricalTrack", { motorDeviceSn: "hashed", trackTime:1 });
//验证分片状态
db.getSiblingDB("config").collections.find({ _id: "boatol.motorHistoricalTrack" }).pretty();
//查看集合分片的分布情况(数据分布在哪个分片上)
db.motorHistoricalTrack.getShardDistribution();
//查看分片状态
sh.status();
注意点:
- 需要创建索引后才可指定分片键,分片键基于索引
- 初始的分片库可能会将数据存在一个片区或一个块中,后续数据量多就会自动将数据分布到其他片区
结果:
分片信息
Shard shardSet2 at shardSet2/mongo-shard4:27018,mongo-shard5:27018,mongo-shard6:27018
{
data: '235KiB',
docs: 1061,
chunks: 2,
'estimated data per chunk': '117KiB',
'estimated docs per chunk': 530
}
---
Shard shardSet1 at shardSet1/mongo-shard1:27018,mongo-shard2:27018,mongo-shard3:27018
{
data: '145KiB',
docs: 657,
chunks: 2,
'estimated data per chunk': '72KiB',
'estimated docs per chunk': 328
}
---
Totals
{
data: '380KiB',
docs: 1718,
chunks: 4,
'Shard shardSet2': [
'61.75 % data',
'61.75 % docs in cluster',
'227B avg obj size on shard'
],
'Shard shardSet1': [
'38.24 % data',
'38.24 % docs in cluster',
'227B avg obj size on shard'
]
}
6、一键部署
在确保docker-compose安装之后,编写docker-compose.yml
文件,创建好每个分片的目录
version: '2'
services:
mongo-config1:
image: mongo:latest
ports:
- 27039:27019
networks:
- cluster
restart: always
hostname: mongo-config1
container_name: mongo-config1
volumes:
- /app/mongo/mongo-config1/data:/data/db
- /app/mongo/mongo-config1/configdb:/data/configdb
- /app/mongo/mongo-config1/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet configReplSet
--configsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-config2:
image: mongo:latest
ports:
- 27049:27019
networks:
- cluster
restart: always
hostname: mongo-config2
container_name: mongo-config2
volumes:
- /app/mongo/mongo-config2/data:/data/db
- /app/mongo/mongo-config2/configdb:/data/configdb
- /app/mongo/mongo-config2/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet configReplSet
--configsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-config3:
image: mongo:latest
ports:
- 27059:27019
networks:
- cluster
restart: always
hostname: mongo-config3
container_name: mongo-config3
volumes:
- /app/mongo/mongo-config3/data:/data/db
- /app/mongo/mongo-config3/configdb:/data/configdb
- /app/mongo/mongo-config3/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet configReplSet
--configsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-shard1:
image: mongo:latest
ports:
- 27067:27018
networks:
- cluster
restart: always
hostname: mongo-shard1
container_name: mongo-shard1
volumes:
- /app/mongo/mongo-shard1/data:/data/db
- /app/mongo/mongo-shard1/configdb:/data/configdb
- /app/mongo/mongo-shard1/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet shardSet1
--shardsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-shard2:
image: mongo:latest
ports:
- 27077:27018
networks:
- cluster
restart: always
hostname: mongo-shard2
container_name: mongo-shard2
volumes:
- /app/mongo/mongo-shard2/data:/data/db
- /app/mongo/mongo-shard2/configdb:/data/configdb
- /app/mongo/mongo-shard2/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet shardSet1
--shardsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-shard3:
image: mongo:latest
ports:
- 27087:27018
networks:
- cluster
restart: always
hostname: mongo-shard3
container_name: mongo-shard3
volumes:
- /app/mongo/mongo-shard3/data:/data/db
- /app/mongo/mongo-shard3/configdb:/data/configdb
- /app/mongo/mongo-shard3/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet shardSet1
--shardsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-shard4:
image: mongo:latest
ports:
- 27066:27018
networks:
- cluster
restart: always
hostname: mongo-shard4
container_name: mongo-shard4
volumes:
- /app/mongo/mongo-shard4/data:/data/db
- /app/mongo/mongo-shard4/configdb:/data/configdb
- /app/mongo/mongo-shard4/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet shardSet2
--shardsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-shard5:
image: mongo:latest
ports:
- 27076:27018
networks:
- cluster
restart: always
hostname: mongo-shard5
container_name: mongo-shard5
volumes:
- /app/mongo/mongo-shard5/data:/data/db
- /app/mongo/mongo-shard5/configdb:/data/configdb
- /app/mongo/mongo-shard5/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet shardSet2
--shardsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongo-shard6:
image: mongo:latest
ports:
- 27086:27018
networks:
- cluster
restart: always
hostname: mongo-shard6
container_name: mongo-shard6
volumes:
- /app/mongo/mongo-shard6/data:/data/db
- /app/mongo/mongo-shard6/configdb:/data/configdb
- /app/mongo/mongo-shard6/log:/var/log/mongodb
- /app/mongo/keyfile.key:/data/keyfile.key
command:
--replSet shardSet2
--shardsvr
--keyFile /data/keyfile.key
--auth
--wiredTigerCacheSizeGB 1
mongos:
image: mongo:latest
ports:
- 27017:27017
networks:
- cluster
restart: always
hostname: mongos
container_name: mongos
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: Wq@123456
volumes:
- /app/mongo/keyfile.key:/data/keyfile.key
- /app/mongo/mongos/log:/var/log/mongodb
- /app/mongo/mongos/data:/data/db
command:
mongos
--configdb configReplSet/mongo-config1:27019,mongo-config2:27019,mongo-config3:27019
--bind_ip_all
--keyFile /data/keyfile.key
networks:
cluster:
external: true
如有新增分片,在后续加上即可
启动:
docker-compose up -d
停止:
docker-compose stop
停止并删除
docker-compose down
7、备份与恢复
(1)备份
# 压缩备份
mongodump --host mongodb1.example.net --port 3017 --username user --password "pass" --authenticationDatabase admin | gzip > /path/to/backup/directory/backup.gz
# 本地备份
mongodump --host localhost --port 27017 --username admin --password Wq@123456 --authenticationDatabase admin --db boatol --out /data/db
# 导出json
mongoexport --host localhost --port 27017 --username admin --password Wq@123456 --authenticationDatabase admin --db boatol --collection motorHistoricalTrack --out /data/db/motorHistoricalTrack.json --jsonArray
(2)恢复
# 导入bson数据
mongorestore --host localhost --port 27017 --username admin --password Wq@123456 --authenticationDatabase admin --db boatol --batchSize 1000 --numParallelCollections 1 --collection motorHistoricalTrack /data/db/motorHistoricalTrack.bson
# 导入json数据
mongoimport --host localhost --port 27017 --username admin --password Wq@123456 --authenticationDatabase admin --db boatol --collection motorHistoricalTrack --file /data/db/motorHistoricalTrack.json
参数解析:
--host
: 主机地址--port
:主机端口--username
:用户名--password
:密码--authenticationDatabase
:需要认证的数据库--db
:导入的目标库--batchSize
:一次性导入数量--numParallelCollections
:并行导入集合数- 最后是导入文件目录
注意点:
- 导出的bson文件过大,在使用
mongorestore
导入时会消耗大量内存,导致有的片区节点宕机无法进行导入,需要指定--batchSize
参数