java mongo replica_mongodb多实例安装及replica set配置

最新推荐文章于 2023-02-01 22:33:22 发布

weixin_39808877

最新推荐文章于 2023-02-01 22:33:22 发布

阅读量634

点赞数

文章标签： java mongo replica

本文链接：https://blog.csdn.net/weixin_39808877/article/details/114110618

版权

需求：为了容灾，需要部署两台服务器，便于主备切换，当主服务器挂掉之后，主辅服务器角色永远对换，即使之前主服务器恢复也不再重新恢复之前主辅角色，直到下次发生主备切换

分析：mongodb两种数据复制方式：1、主备，当主服务器挂掉之后，辅服务器不能够提供写服务，故mongodb主备方案不合适。2、replica set，但只有两个节点时，主节点挂掉，辅节点也不能提供写服务，与主备效果一样，所以至少需要三个mongodb实例，但当前只有两台服务器，所以有一台服务器要安装有两个mongodb实例，一台机器上不需要两份数据拷贝，故第三个节点使用arbiter，只进行投票，不会被选为主节点，也不会拷贝数据。另外考虑到需求中主备逻辑，结合replica set特点，arbiter只能在服务器中运行，考虑到主备角色调换，故两台服务器每台需要安装两个mongodb实例，其中一个为arbiter

两台服务器：

192.168.103.5 (原始主服务器)

192.168.103.28 (原始辅服务器)

一、两台服务器分别安装mongodb

安装步骤可参考：https://www.cnblogs.com/qq931399960/p/10425803.html

二、两台服务器分别安装arbiter

1、新建mongodb日志和数据库存储路径

mkdir -p /home/mongo-arbiter/dbmkdir -p /home/mongo-arbiter/log

2、拷贝配置文件

cp /etc/mongod.conf /etc/mongod-arbiter.conf

3、修改配置文件

vim /etc/mongod-arbiter.conf

systemLog.path修改为/home/mongo-arbiter/log/mongod.log

Storage.dbPath修改为/home/mongo-arbiter/db

processManagement.pidFilePath修改为/var/run/mongodb/mongod-arbiter.pid

net.port修改为27018

4、修改权限

chown -R mongod:mongod /home/mongo-arbiter

5、设置arbiter开机启动

cp /usr/lib/systemd/system/mongod.service /usr/lib/systemd/system/mongod-arbiter.service

修改开机启动文件

vim /usr/lib/systemd/system/mongod-arbiter.service

Environment改为："OPTIONS=-f /home/mongo-arbiter/mongod.conf"

PIDFile改为：/var/run/mongodb/mongod-arbiter.pid

使配置生效

systemctl enable mongod-arbiter.service

systemctl daemon-reload

初始化用户(四个实例的用户要一致)

cat ./mongdb/initUser.js | mongo --port 27018 --shell

6、登录检测

mongo -uroot -pabc123 --authenticationDatabase "admin" --port 27018

正常登陆，ok

三、配置replica set

1、停止所有mongodb实例

2、生成keyfile文件

openssl rand -base64 756 > /home/replSetKeychmod 400 /home/replSetKeychown -R mongod:mongod /home/replSetKey

2、将上述文件拷贝到另外一台服务器/home下(路径可自定义，只要四个实例使用相同kefile文件即可)

3、添加/修改mongodb实例配置(如下配置每个实例相同，4个mongodb实例都需要修改，如下列出的只有新加和修改的，不变的未列出)

security:

keyFile:/home/replSetKey ## 新加

replication:

replSetName: mongoSet ## 新加

net:

bindIp:0.0.0.0 ## 修改

4、启动主服务器默认27017端口mongodb实例，启动辅服务器中两个mongodb实例

systemctl start mongod

systemctl start mongod-arbiter

5、查看防火墙是否关闭，或者27017和27018端口是否放开

未关闭防火墙或者以上端口不能访问，则初始化replica set失败

关闭防火墙命令

systemctl stop firewalld

systemctl disable firewalld

6、在主服务器(103.5)进入mongodb命令

mongo -u "root" -p "abc123" --authenticationDatabase "admin"

7、初始化两个memeber的replicat set

config = {_id:"mongoSet", members:[{_id:0,host:"192.168.103.5:27017",priority:2},

{_id:1,host:"192.168.103.28:27017",priority:1}]}

rs.initiate(config,true)

此时可通过rs.status()状态

8、添加arbiter仲裁节点

rs.addArb("192.168.103.28:27018")

rs.status()查看各节点状态，如果有节点stateStr显示not reachable/healthy，health显示为0，则重启该节点，再次通过rs.status()查看是否正常

到此，主备服务基本配置结束

java客户端连接地址为：

mongodb://root:abc123@192.168.103.5:27017,192.168.103.28:27017/?authSource=mydb&replicaSet=mongoSet

四、重新配置replica set

当主备服务器切换，java客户端连接由于配置了replicaSet，可以自动去新主节点进行业务处理，不会有问题，当主设备恢复为后，原主服务器mongodb再次成为了主节点，但其他服务已经在新主服务器上

A、mongodb和web服务不在台机器，中间有网络传输，影响性能

B、此时的replica set中，主节点在辅服务器上，主服务器上只有辅节点和arbiter，当主服务器挂掉，只有一个mongodb节点，不能在提供写服务

综上两点，尤其是第二点，需要重新配置replica set，登录到mongodb主节点

mongo -uroot -pabc123 --authenticationDatabase "admin"

或者在主服务器远程登录恢复的mongodb主节点

mongo -u "root" -p "abc123" --authenticationDatabase "admin" --host 192.168.103.5

重新配置

reconfig = {_id:"mongoSet","protocolVersion" : 1, members:[{_id:0,host:"192.168.103.5:27017",priority:1},

{_id:1,host:"192.168.103.28:27017",priority:2}]}

rs.reconfig(reconfig,true)

添加arbiter节点

rs.addArb("192.168.103.5:27018")

使用rs.status()查看状态，如果有节点stateStr显示not reachable/healthy，health显示为0，则重启该节点，再次通过rs.status()查看是否正常

PS：

1、若有节点stateStr出现not reachable/healthy，health显示为0，则该节点可能是arbiter节点，官方不推荐将arbiter节点安装到存在primary和slave节点的服务器上，不过当前只有两台服务器，也只有这样了，可以正常运行。

2、重新配置时，需要保证之前已存在的host的id是相同的

3、重新配置需要添加protocolVersion

五、使用脚本简化操作

1、提前准备好配置文件和arbiter开机启动service文件，需要的字段都添加上,但replica set相关字段为注释状态，因为打开后，所有节点都将无法进行写操作，新建用户后打开，并重启mongodb

2、提前准备好kefile，两台服务器都使用该文件

3、安装mongodb (数据库和日志路径可以自定义)

#!/bin/bash

basedir=$(cd "$(dirname"$0")";pwd)functioninitenv(){

setenforce0

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

}

########inst mongdb4.0#############functioninst(){cp -f ./mongdb/mongod.conf /etc/rpm-ivh ./mongdb/*rpm

cd /home

mkdir -p mongo-db

mkdir -p mongo-log

chown -R mongod:mongod ./mongo-db

chown -R mongod:mongod ./mongo-log

systemctl enable mongod

systemctl start mongod

}

function initdb(){

cd $basedir

cat ./mongdb/initUser.js | mongo --shell

cat ./mongdb/initIndex.js | mongo -u "root" -p "abc123" --authenticationDatabase "admin"

}

function cpReplicaSetKey(){

cd $basedir

cp ./mongdb/replSetKey /home/

chmod 400 /home/replSetKey

chown -R mongod:mongod /home/replSetKey

}

function openReplicaSet(){

sed -i 's/#keyFile/keyFile/g' /etc/mongod.conf

sed -i 's/#replication/replication/g' /etc/mongod.conf

sed -i 's/#replSetName/replSetName/g' /etc/mongod.conf

}

function stopMongo(){

systemctl stop mongod

}

function startMongo(){

systemctl start mongod

}

function main(){

initenv

inst

initdb

stopMongo

cpReplicaSetKey

openReplicaSet

startMongo

}

main | tee -a ./log/inst_mongodb.log

View Code

4、安装mongodb arbiter节点(在rs.addArb()之前其实还不能说是arbiter节点，只是起了个名字叫arbiter节点)

#!/bin/bash

basedir=$(cd "$(dirname"$0")";pwd)functioninitenv(){

setenforce0

sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

}

########inst mongdb4.0#############functioninst(){cp -f ./mongdb/mongod-arbiter.conf /etc/

cp -f ./mongdb/mongod-arbiter.service /usr/lib/systemd/system/

mkdir -p /home/mongodb-arbiter/logmkdir -p /home/mongodb-arbiter/dbchown -R mongod:mongod /home/mongodb-arbiter

systemctl enable mongod-arbiter

systemctl start mongod-arbiter

}functioninitdb(){

cd $basedircat ./mongdb/initUser.js | mongo --port 27018 --shellcat ./mongdb/initIndex.js | mongo --port 27018 -u "root" -p "abc123" --authenticationDatabase "admin"}functionopenReplicaSet(){sed -i 's/#keyFile/keyFile/g' /etc/mongod-arbiter.confsed -i 's/#replication/replication/g' /etc/mongod-arbiter.confsed -i 's/#replSetName/replSetName/g' /etc/mongod-arbiter.conf

}functionstopMongo(){

systemctl stop mongod

}functionstartMongo(){

systemctl start mongod

}functionmain(){

initenv

inst

initdb

stopMongo

openReplicaSet

startMongo

}

main| tee -a ./log/inst_mongodb_arbiter.log

View Code

5、初始replica set

#!/bin/bash

basedir=$(cd "$(dirname"$0")";pwd)

source ./app.properties

nowtime=`date --date='0 days ago' "+%Y%m%d%H%M%S"`

daytime=`date --date='0 days ago' "+%Y%m%d"`

logPath=./log/init_repliaset_$daytime.logfunctionlog(){echo $1 | tee -a $logPath

}functionlogfile(){echo $1 >>$logPath

}functiontestConfig(){

log"check relate config."

if [ "$masterServerIp" == "masterIp" ];thenlog"ERROR: masterServerIp is not correct in the file app.properties"exit1

if [ "$slaveServerIp" == "slaveIp" ];thenlog"ERROR: slaveServerIp is not correct in the file app.properties"exit1

filocal grepIp=`ifconfig | grep$masterServerIp`

## it is must executein the master server which you input in the app.properties filewhen first initiate ths replica setif [ "$grepIp" == "" ];thenlog"the current server is not the master server, please login on to $masterServerIp and execute this script."exit1

fi## test the connectionecho "use admin" >test.jsecho "rs.slaveOk()" >>test.jsecho "show dbs" >>test.jscat test.js | mongo --host $slaveServerIp -uroot -pabc123 --authenticationDatabase "admin" >>$logPathif [ $? != 0 ];thenlog"mongodb instance which port is 27017 in $slaveServerIp server can not connect."exit1;fi

cat test.js | mongo --host $masterServerIp -uroot -pabc123 --authenticationDatabase "admin" >>$logPathif [ $? != 0 ];thenlog"mongodb instance which port is 27017 in $slaveServerIp server can not connect."exit1;fi

rm -rf test.js

log"check relate config end."}functioninitTwoMemberReplicaSet(){

log"init two member replica set."

echo "use admin" >init.jsecho "config = {_id:\"mongoSet\", members:[{_id:0,host:\"$masterServerIp:27017\",priority:2},{_id:1,host:\"$slaveServerIp:27017\",priority:1}]}" >>init.jsecho "rs.initiate(config, {force:true})" >>init.jscat init.js | mongo -uroot -pabc123 --authenticationDatabase "admin" >>$logPath

##sleep 3 second forprimary election.sleep3s

log"init two member replica set end."}functionisElectPrimary(){

log"check is the master node selected."local masterStr=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "PRIMARY" -B 4|grep "name" | grep$masterServerIp`

local count=0

while [[ "$masterStr" == "" ]];do

sleep1s

count=$((count+1))

masterStr=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "PRIMARY" -B 4|grep "name" | grep$masterServerIp`if [ "$masterStr" != "" ];thenlog"the master node is selected."breakfi

if [ "$count" == "30" ];thenlog"ERROR: arbiter configure timeout, please configure manually with 'rs.addArb(\"$slaveServerIp:27018\")' in $masterServerIp:27017"exit1

donelog"check is the master node selected end."}functionaddArbiter(){

log"add arbiter ..."mongo-uroot -pabc123 --authenticationDatabase "admin" --eval "rs.addArb(\"$slaveServerIp:27018\")"log"add arbiter end."}functionchekArbiter(){

log"check arbiter."local arbiterStr=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "$slaveServerIp:27018" -A 3| grep "stateStr"`if [ "$arbiterStr" != "ARBITER" ];thenlog"ERROR: abriter $slaveServerIp:27018: $arbiterStr, may be you need restart mongodb with 'systemctl restart mongod-arbiter' command in $slaveServerIp."

filog"check arbiter end."}

logfile"**************************$nowtime**********************************"testConfig

initTwoMemberReplicaSet

isElectPrimary

addArbiter

chekArbiter

log"init end..."

View Code

6、重新配置replica set

#!/bin/bash

basedir=$(cd "$(dirname"$0")";pwd)

source ./app.properties

nowtime=`date --date='0 days ago' "+%Y%m%d%H%M%S"`

daytime=`date --date='0 days ago' "+%Y%m%d"`

logPath=./log/init_repliaset_$daytime.logfunctionlog(){echo $1 | tee -a $logPath

}functionlogfile(){echo $1 >>$logPath

}functiontestConfig(){

log"check relate config."

if [ "$masterServerIp" == "masterIp" ];thenlog"ERROR: masterServerIp is not correct in the file app.properties"exit1

if [ "$slaveServerIp" == "slaveIp" ];thenlog"ERROR: slaveServerIp is not correct in the file app.properties"exit1

fi## get current replica set master node

local currMasterNode=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "PRIMARY" -B 4|grep "name" | cut -d ":" -f 2 | sed 's/\"//g'`if [ "`ifconfig | grep $currMasterNode`" == "" ];thenlog"the current server has no mongodb master node, please login on to $currMasterNode execute this script."exit1

cat test.js | mongo --host $slaveServerIp --port 27018 -uroot -pabc123 --authenticationDatabase "admin" >>$logPathif [ $? != 0 ];thenlog"mongodb instance which port is 27018 in $slaveServerIp server can not connect."## 有时，当主从服务器都重启之后，有一个仲裁节点由于不在replica set中，但配置文件中却存在replica的配置，造成不能够正常启动，但reconfig replica set后，需要将该节点reconfig到replica set中，故此处不能因为不能连接就执行exit，但输出错误信息给出提示，即使服务未启动，

## 执行rs.addArb()也可以添加仲裁节点，但必须确定添加的仲裁节点会启动，否则即使通过rs.remove删除掉了该仲裁节点，但主节点日志中也会一直打印错误信息，fi

rm -rf test.js

log"check relate config end."}functioninitTwoMemberReplicaSet(){

log"reconfig two member replica set."

echo "use admin" >reconfig.js

## the _id must be the same as before

##find the master server id inprevious replica set

##find the slave server id inprevious replica set

local slaveId=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "$slaveServerIp:27017" -B 1|grep "_id" | cut -d ":" -f 2|sed 's/,//g'`echo "config = {_id:\"mongoSet\","protocolVersion": 1, members:[{_id:$masterId,host:\"$masterServerIp:27017\",priority:2},{_id:$slaveId,host:\"$slaveServerIp:27017\",priority:1}]}" >>reconfig.jsecho "rs.reconfig(config, {force:true})" >>reconfig.jscat reconfig.js | mongo -uroot -pabc123 --authenticationDatabase "admin" >>$logPath

##sleep 3 second forprimary election.sleep3s

log"reconfig two member replica set end."}functionisElectPrimary(){

local count=0

while [[ "$masterStr" == "" ]];do

sleep1s

count=$((count+1))

if [ "$count" == "30" ];thenlog"ERROR: arbiter configure timeout, please configure manually with 'rs.addArb(\"$slaveServerIp:27018\")' in $masterServerIp:27017"exit1

donelog"check is the master node selected end."}functionaddArbiter(){

log"remove arbiter in the master server."## master serverdo not deploy arbiter, when reconfig, after the master node reselected you must specify the --post fordelete or add

local beforeArbiterIp=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "ARBITER" -B 4|grep "name" | cut -d ":" -f 2 | sed 's/\"//g'`if [ "`echo $masterServerIp | grep $beforeArbiterIp`" != "" ];thenlog"remove $masterServerIp:27018 in replica set."mongo-uroot -pabc123 --authenticationDatabase "admin" --host $masterServerIp --eval "rs.remove(\"$masterServerIp:27018\")"

filog"remove arbiter in the master server end."log"add arbiter ..."mongo-uroot -pabc123 --authenticationDatabase "admin" --host $masterServerIp --eval "rs.addArb(\"$slaveServerIp:27018\")"log"add arbiter end."}functionchekArbiter(){

log"check arbiter."local arbiterStr=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "$slaveServerIp:27018" -A 3 | grep "ARBITER"`

## arbiter connect needtime， set to 15seconds

local count=0

while [[ "$arbiterStr" == "" ]];do

sleep1s

count=$((count+1))

arbiterStr=`mongo -uroot -pabc123 --authenticationDatabase "admin" --eval "rs.status()" | grep "stateStr" -B 4|grep "$slaveServerIp:27018" -A 3 | grep "ARBITER"`if [ "$arbiterStr" != "" ];thenlog"the arbiter node is selected."breakfi##wait for 15secondsif [ "$count" == "15" ];thenlog"ERROR: abriter $slaveServerIp:27018: $arbiterStr, may be you need restart mongodb with 'systemctl restart mongod-arbiter' command in $slaveServerIp."breakfi

donelog"check arbiter end."}

logfile"**************************$nowtime**********************************"testConfig

initTwoMemberReplicaSet

isElectPrimary

addArbiter

chekArbiter

log"init end..."

View Code

脚本中通过rs.addArb()添加未启动或者不存在的arbiter节点，可以添加成功，但主节点日志中会一直打印该arbiter节点相关的错误信息，即使通过rs.remove()删除掉该arbiter节点，错误信息也会打印，该问题，重启主节点可以解决

7、配置文件app.properties

masterServerIp=masterIp

slaveServerIp=slaveIp

View Code

执行初始化和重新配置replica set之前需要确认app.properties中配置是正确的

8、无论是初始化replica set或者重新配置replica set，前几次都会出现not reachable/healthy，此时根据shell页面打印的提示，在对应的服务器上执行systemctl restart mongod-arbiter即可

weixin_39808877

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
java mongo replica_mongodb多实例安装及replica set配置

需求：为了容灾，需要部署两台服务器，便于主备切换，当主服务器挂掉之后，主辅服务器角色永远对换，即使之前主服务器恢复也不再重新恢复之前主辅角色，直到下次发生主备切换分析：mongodb两种数据复制方式：1、主备，当主服务器挂掉之后，辅服务器不能够提供写服务，故mongodb主备方案不合适。2、replica set，但只有两个节点时，主节点挂掉，辅节点也不能提供写服务，与主备效果一样，所以至少需要三...
复制链接

扫一扫