Mongodb4.4.1分片集群部署

1.环境规划

IP	实例
172.26.1.7	mongos（30000） config（27017） shard1主节点（40001） shard2仲裁节点（40002） shard3副节点（40003）
172.26.1.8	mongos（30000） config（27017） shard1副节点（40001） shard2主节点（40002） shard3仲裁节点（40003）
172.26.1.9	mongos（30000） config（27017） shard1仲裁节点（40001） shard2副节点（40002） shard3主节点（40003）

3台机，每台机5个实例，分别mongos 1 个，config server 1 个，shard server 3 个

2.创建相应目录（3台机器执行相同操作）

mkdir -p /mongo/{data,logs,apps,run} 
mkdir -p /mongo/data/shard{1,2,3} 
mkdir -p /mongo/data/config

3.配置环境变量（3台机器执行相同操作）

使用root账号修改配置,/mongo/apps/bin 用于存放mongodb的执行程序

echo 'export PATH=$PATH:/mongo/apps/bin' >> /etc/profile
source /etc/profile

4.创建用户及修改权限（3台机器执行相同操作）

groupadd -g 10001 mongodb 
useradd -u 10001 -g mongodb mongodb 
id mongodb 
passwd mongodb 
输入新的密码：mongo2020 

chown -R mongodb:mongodb /mongo
 
chmod -R 775 /mongo

切换用户

su mongodb

5.下载安装文件（3台机器执行相同操作）

切换目录

cd /mongo/apps

下载文件

wget https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-rhel70-4.4.1.tgz

解压文件

tar -xf mongodb-linux-x86_64-rhel70-4.4.1.tgz
mv mongodb-linux-x86_64-rhel70-4.4.1/bin ./
rm -rf mongodb-linux-x86_64-rhel70-4.4.1

6.创建配置文件（3台机器执行相同操作）

vi /mongo/apps/conf/mongo-config.yml

systemLog:
  destination: file
#注意修改路径
  path: "/mongo/logs/mongo-config.log" 
  logAppend: true
storage:
  journal:
    enabled: true
#注意修改路径
  dbPath: "/mongo/data/config"
  engine: wiredTiger
  wiredTiger:
    engineConfig:
         cacheSizeGB: 12
processManagement:
  fork: true
  pidFilePath: "/mongo/run/mongo-config.pid"
net:
  bindIp: 0.0.0.0
#注意修改端口
  port: 27017
setParameter:
  enableLocalhostAuthBypass: true
replication:
#复制集名称
  replSetName: "mgconfig" 
sharding:
#作为配置服务
  clusterRole: configsvr 

vi /mongo/apps/conf/mongo-shard1.yml

systemLog:
  destination: file
  path: "/mongo/logs/mongo-shard1.log"
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: "/mongo/data/shard1"
processManagement:
  fork: true
  pidFilePath: "/mongo/run/mongo-shard1.pid"
net:
  bindIp: 0.0.0.0
  #注意修改端口
  port: 40001
setParameter:
  enableLocalhostAuthBypass: true
replication:
  #复制集名称
  replSetName: "shard1" 
sharding:
  #作为分片服务
  clusterRole: shardsvr

vi /mongo/apps/conf/mongo-shard2.yml

systemLog:
  destination: file
  path: "/mongo/logs/mongo-shard2.log"
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: "/mongo/data/shard2"
processManagement:
  fork: true
  pidFilePath: "/mongo/run/mongo-shard2.pid"
net:
  bindIp: 0.0.0.0
  #注意修改端口
  port: 40002
setParameter:
  enableLocalhostAuthBypass: true
replication:
  #复制集名称
  replSetName: "shard2" 
sharding:
  #作为分片服务
  clusterRole: shardsvr

vi /mongo/apps/conf/mongo-shard3.yml

systemLog:
  destination: file
  path: "/mongo/logs/mongo-shard3.log"
  logAppend: true
storage:
  journal:
    enabled: true
  dbPath: "/mongo/data/shard3"
processManagement:
  fork: true
  pidFilePath: "/mongo/run/mongo-shard3.pid"
net:
  bindIp: 0.0.0.0
  #注意修改端口
  port: 40003
setParameter:
  enableLocalhostAuthBypass: true
replication:
  #复制集名称
  replSetName: "shard3" 
sharding:
  #作为分片服务
  clusterRole: shardsvr

vi /mongo/apps/conf/mongo-route.yml

systemLog:
  destination: file
  #注意修改路径
  path: "/mongo/logs/mongo-route.log" 
  logAppend: true
processManagement:
  fork: true
  pidFilePath: "/mongo/run/mongo-route.pid"
net:
  bindIp: 0.0.0.0
  #注意修改端口
  port: 30000
setParameter:
  enableLocalhostAuthBypass: true
replication:
  localPingThresholdMs: 15
sharding:
  #关联配置服务
  configDB: mgconfig/172.26.1.7:27017,172.26.1.8:27017,172.26.1.9:27017 

注意：以上bindIp: 0.0.0.0 可以换成对应机器的IP

7.部署配置服务器集群

启动config服务（3台机器执行相同操作）

cd /mongodb/apps/conf/
mongod --config mongo-config.yml
about to fork child process, waiting until server is ready for connections.
forked process: 985
child process started successfully, parent exiting

连接一个实例

mongo 172.26.1.7:27017

初始化复制集

这个 mgconfig 名字一定要和config 配置文件中 replSet 的名字一致

config={_id:"mgconfig",members:[ 
{_id:0,host:"172.26.1.7:27017"}, 
{_id:1,host:"172.26.1.8:27017"}, 
{_id:2,host:"172.26.1.9:27017"}, 
]}
  
rs.initiate(config)

检查状态

rs.status()

复制集配完后，可能状态不会马上改变（可能都是secondary），过几秒就会自动更新

8.部署shard1分片集群

启动3台shard1实例

cd /mongodb/apps/conf/
mongod --config mongo-shard1.yml 

连接一个实例

mongo 172.26.1.7:40001

创建复制集

use admin
 
config={_id:"shard1",members:[ 
{_id:0,host:"172.26.1.7:40001",priority:2}, 
{_id:1,host:"172.26.1.8:40001",priority:1}, 
{_id:2,host:"172.26.1.9:40001",arbiterOnly:true}, 
]}

这个 shard1名字一定要和 shard1配置文件中 replSet 的名字一致

初始化复制集

rs.initiate(config)

检查状态

rs.status()

复制集配完后，可能状态不会马上改变（可能都是secondary），过几秒就会自动更新

9.部署shard2分片集群

启动3台shard2实例

cd /mongodb/apps/conf/
mongod --config mongo-shard2.yml 

连接第二个节点创建复制集

为什么是连接第二个，因为规划的shard2 的主节点是8:40002，仲裁节点不能写数据，所以这里不能连7，要连8

mongo 172.26.1.8:40002

创建复制集

use admin 
config={_id:"shard2",members:[ 
{_id:0,host:"172.26.1.7:40002",arbiterOnly:true}, 
{_id:1,host:"172.26.1.8:40002",priority:2}, 
{_id:2,host:"172.26.1.9:40002",priority:1}, 
]}

这个 shard2名字一定要和 shard2配置文件中 replSet 的名字一致

初始化复制集

rs.initiate(config)

检查状态

rs.status()

复制集配完后，可能状态不会马上改变（可能都是secondary），过几秒就会自动更新

10.部署shard3分片集群

启动3台shard3实例

cd /mongodb/apps/conf/
mongod --config mongo-shard3.yml

连接第三个节点创建复制集

mongo 172.26.1.9:40003

创建复制集

use admin 
config={_id:"shard3",members:[ 
{_id:0,host:"172.26.1.7:40003",priority:1}, 
{_id:1,host:"172.26.1.8:40003",arbiterOnly:true}, 
{_id:2,host:"172.26.1.9:40003",priority:2}, 
]}

这个 shard3名字一定要和 shard3配置文件中 replSet 的名字一致

初始化复制集

rs.initiate(config)

检查状态

rs.status()

复制集配完后，可能状态不会马上改变（可能都是secondary），过几秒就会自动更新

11.启用分片功能

登陆路由节点

mongo 172.26.1.7:30000

use admin

sh.addShard("shard1/172.26.1.7:40001,172.26.1.8:40001,172.26.1.9:40001") 
sh.addShard("shard2/172.26.1.7:40002,172.26.1.8:40002,172.26.1.8:40002")
sh.addShard("shard3/172.26.1.7:40003,172.26.1.8:40003,172.26.1.8:40003")

sh.status()

12.服务脚本

编写服务脚本，便于启动和停止集群

vi /mongo/apps/bin/mongo-manager.sh

写入以下内容

#!/bin/bash
# mongodb script takes care of starting ||stopping ||reload mongom
# chkconfig:- 80 15
# description: Mongo database
# precessname: mongom
user=mongodb
#source function library
#source /etc/init.d/functions
#/mongodb/apps/mongodb/bin
# the localcation of configfile
config_configfile="/mongo/apps/conf/mongo-config.yml"
router_configfile="/mongo/apps/conf/mongo-route.yml"
shard1_configfile="/mongo/apps/conf/mongo-shard1.yml"
shard2_configfile="/mongo/apps/conf/mongo-shard2.yml"
shard3_configfile="/mongo/apps/conf/mongo-shard3.yml"

# the options of start a mongodb server
start_config_options=" --config $config_configfile"
stop_config_options=" --shutdown --dbpath /mongo/data/config"
start_router_options=" --config $router_configfile"
start_shard1_options=" --config $shard1_configfile"
stop_shard1_options=" --shutdown --dbpath /mongo/data/shard1"
start_shard2_options=" --config $shard2_configfile"
stop_shard2_options=" --shutdown --dbpath /mongo/data/shard2"         
start_shard3_options=" --config $shard3_configfile"
stop_shard3_options=" --shutdown --dbpath /mongo/data/shard3"
        
# the localcation of mongod
mongod="/mongo/apps/bin/mongod"
      
# the localcation of mongos
mongos="/mongo/apps/bin/mongos"
    
#where to lockfile
config_lockfile="/mongo/data/config/mongod.lock"
shard1_lockfile="/mongo/data/shard1/mongod.lock"
shard2_lockfile="/mongo/data/shard2/mongod.lock"
shard3_lockfile="/mongo/data/shard3/mongod.lock"
#where to pidfile
config_pidfile="/mongo/run/mongo-config.pid"
router_pidfile="/mongo/run/mongo-route.pid"
shard1_pidfile="/mongo/run/mongo-shard1.pid"
shard2_pidfile="/mongo/run/mongo-shard2.pid"
shard3_pidfile="/mongo/run/mongo-shard3.pid"

#function of start config server
function start-config(){
    #print the tips
    echo -n $"Starting mongod of config_server:"
    $mongod $start_config_options
    #get the result
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        touch $config_lockfile
    fi

}

#function of stop config server
function stop-config(){
    #print the tips
    echo  $"stopping mongod of config_server:"
    $mongod $stop_config_options
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        rm -f $config_lockfile
        rm -f $config_pidfile
    fi

}



#function of start router server
function start-router(){
    #print the tips
    echo  $"Starting mongod of router_server:"
    $mongos $start_router_options

}
#function of stop router server
function stop-router(){
    #print the tips
    echo  $"stopping mongod of router_server:"
	kill `cat $router_pidfile`
	RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        rm -f $router_pidfile
    fi

}


#function of start shard1
function start-shard1(){
    #print the tips
    echo  $"Starting mongod of shard1_server:"
    $mongod $start_shard1_options
    #get the result
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        touch $shard1_lockfile
    fi

}

#function of stop shard1
function stop-shard1(){
    #print the tips
    echo  $"stopping mongod of shard1_server:"
    $mongod $stop_shard1_options
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        rm -f $shard1_lockfile
        rm -f $shard1_pidfile
    fi

}
#function of start shard2
function start-shard2(){
    #print the tips
    echo -n $"Starting mongod of shard2_server:"
    $mongod $start_shard2_options
    #get the result
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        touch $shard2_lockfile
    fi

}

#function of stop shard2
function stop-shard2(){
    #print the tips
    echo  $"stopping mongod of shard2_server:"
    $mongod $stop_shard2_options
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        rm -f $shard2_lockfile
        rm -f $shard2_pidfile
    fi

}

#function of start shard3
function start-shard3(){
    #print the tips
    echo -n $"Starting mongod of shard3_server:"
    $mongod $start_shard3_options
    #get the result
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        touch $shard3_lockfile
    fi

}
#function of stop shard3
function stop-shard3(){
    #print the tips
    echo  $"stopping mongod of shard3_server:"
    $mongod $stop_shard3_options
    RETVAL=$?
    if [ $RETVAL -eq 0 ];then
        rm -f $shard3_lockfile
        rm -f $shard3_pidfile
    fi

}

start(){
    start-config
    echo ''
    start-shard1
    echo ''
    start-shard2
    echo ''
    start-shard3
    echo ''
    start-router

}

stop(){
    stop-shard1
    echo 'stop-shard1 ok'
    stop-shard2
    echo 'stop-shard2 ok'
    stop-shard3
    echo 'stop-shard3 ok'
    stop-router
    echo 'stop-router ok'
    stop-config
    echo 'stop-config ok'
}

RETVAL=0
case "$1" in
    start)
    start
      ;;
    stop)
    stop
      ;;
    start-all)
    start
      ;;
    stop-all)
    stop
      ;;
    start-shard3)
    start-shard3
      ;;
    stop-shard3)
    stop-shard3
      ;;
    start-shard2)
    start-shard2
      ;;
    stop-shard2)
    stop-shard2
      ;;
    start-shard1)
    start-shard1
      ;;
    stop-shard1)
    stop-shard1
      ;;
    start-config)
    start-config
      ;;
    stop-config)
    stop-config
      ;;
    start-router)
    start-router
      ;;
    restart-config |reload-config |force-reload-config)
    stop-config
    start-config
      ;;
    restart-router |reload-router |force-reload-router)
    stop-router
    start-router
      ;;
    restart-shard1 |reload-shard1 |force-reload-shard1)
    stop-shard1
    start-shard1
      ;;
    restart-shard2 |reload-shard2 |force-reload-shard2)
    stop-shard2
    start-shard2
      ;;
    restart-shard3 |reload-shard3 |force-reload-shard3)
    stop-shard3
    start-shard3
      ;;
    restart-all |reload-all |force-reload-all)
    stop
    start
      ;;
    conderstart)
      [ -f $lockfile ] && restart || :

      [ -f $lockfile ] && restart || :
      ;;
    status)
      status $mongod
      RETVAL=$?
       ;;
    *)
    echo "Usage: $0 {start-*|stop-*|restart-*|status|reload-*|force-reload-*|condrestart(* in {all,config,router,shard1,shard2,shard3})}"
esac
exit $RETVAL

13.测试服务器分片功能

use config
 
db.settings.save({"_id":"chunksize","value":1})

模拟写入数据

在tydb库的tyuser表中循环写入6万条数据

use tydb
 
show collections
 
for(i=1;i<=60000;i++){db.tyuser.insert({"id":i,"name":"ty"+i})}

启用数据库分片

sh.enableSharding("tydb")

创建的索引

db.tyuser.createIndex({"id":1})

启用表分片

sh.shardCollection(”tydb.tyuser",{"id":1})

查看分片情况

sh.status()

开启平衡器

use admin 
sh.startBalance()

或者

sh.setBalancerState(true)

关闭平衡器

use admin
 
sh.stopBalancer()

或者

sh.setBalancerState(false)

查看是否关闭

返回flase表示平衡器已关闭，还需要查询均衡器正在运行的 情况

sh.getBalancerState()

14.安全认证

默认的mongodb是不设置认证的。只要ip和端口正确就能连接，这样是不安全的。

生成密钥文件

执行如下命令

cd /mongo
openssl rand -base64 753 >keyFile.key
chmod 400 keyFile.key

将keyFIle.key复制到指定位置

scp keyFile.key mongodb@172.26.1.8://mongo/
scp keyFile.key mongodb@172.26.1.8://mongo/

创建一个管理员账户和密码，

use admin
db.createUser({user:"admin",pwd:"123456",roles:["root"]})
db.auth("admin","123456") 

并将集群中的所有mongod和mongos全部关闭

mongo-manager.sh stop

修改config server，shard1，shard2配置文件，增加如下参数：

security:
  authorization: "enabled"
  keyFile: /mongo/keyFile.key

修改mongos配置文件，增加如下参数：

security:
  keyFile: /mongo/keyFile.key

重启config server，shard1，shard2和mongos

mongo-manager.sh stop

测试连接

mongo --host 172.26.1.7:30000 -u admin -p 

为指定数据库创建账户

use admin
db.createUser({user: "root", pwd: "123456", roles: [{ role: "dbAdmin", db: "gateway_new" }, { role: "readWrite", db: "gateway_new" }]})
db.auth("root","123456") 

mongo --host 172.26.1.7:30000/gateway_new -u root -p 

//测试连接指定数据库