csync2和lsyncd实现虚拟机实时双向同步

csync2和lsyncd实现虚拟机实时双向同步

• 最近在搭建一套支付系统，自建机房，我负责搭建系统，在此我会记录搭建的所有过程，以及遇到的问题，所有同行，要是遇到类似问题，评论可以交流交流。这套环境总共使用了11台物理主机，和外网交互的DMZ区，两台主机，负责外网的nginx，APP使用两台物理主机，分成16台虚拟机，APP区的nginx两台主机，负责内网app之间的相互调用以及负载均衡，DB四台物理主机，两台Oracle，两台MySql，一台win10主机，此套环境不允许外网访问数据库，所以安装了台win10便于操作数据库。
• 由于nginx做了负载均衡，所以nginx的配置文件要保持同步，才有本文的主题，实时双向同步，我采用的是csync2和lsyncd实现的。

一、安装cysnc2

• 我们使用yum安装：

sudo yum install http://repo.okay.com.mx/centos/7/x86_64/release/okay-release-1-1.noarch.rpm

sudo yum -y install xinetd

sudo yum install sqlite-devel

sudo yum install csync2

• 启动cysnc2，执行命令

service xinetd start

• 验真是否启动成功，查看端口有没有被监听

netstat -nutpl | grep 30865

二、生成csync2认证key

csync2 -k /etc/csync2.key

这里key存放位置可以自己选择，不过下面lsyncd要配置。这个key是服务器之间认证的一个秘钥。

三、配置csync2

• 如果之前的命令都执行正确，到这一步，etc下面应该已经生成了csync2.cfg。

vi /etc/csync2.cfg

nossl * *;
group mygroup
{
        host nginx1
        host nginx2
        key /etc/csync2.key;
        include /etc/nginx;
        exclude *~ .*;

	action
	 {
        pattern /etc/nginx/nginx.conf;
        pattern /etc/nginx/conf.d/*;
        exec "/usr/bin/systemctl reload nginx";
        logfile "/var/log/csync2_action.log";
        do-local;
	 }

        backup-directory /opt/backup/csync2;
        backup-generations 3;
        auto younger;
}

• host分别是两台nginx所在的机器，这里只能写服务名，不能写ip

vi /etc/hosts

10.13.12.11 nginx1
10.13.12.12 nginx2

• key使用的就是刚生成的csync2.key

• include 是只需要同步的目录，如果是多个目录，include为绝对路径,

include aaaa/aaa
include ccc/bbbb

• exclude是排除的内容， *~ .*，排除~结尾的和.开头的文件或目录

• action里面配置的过滤文件，以及重新载入nginx的配置文件

四、配置各个节点的cysnc2

• 创建nginx1的csync2_nginx1.cfg文件
• 各节点的配置文件格式如csync2_*.cfg,中间的*号为相应节点的名称

touch csync2_nginx1.cfg

nossl * *;
group nginx1
 {
        host nginx1;
        host (nginx2);
        key /etc/csync2.key;
        include /etc/nginx;
        exclude *~ .*;

action
 {
        pattern /etc/nginx/nginx.conf;
        pattern /etc/nginx/conf.d/*;
        exec "/usr/bin/systemctl reload nginx";
        logfile "/var/log/csync2_action.log";
        do-local;
 }

        backup-directory /opt/backup/csync2;
        backup-generations 3;
        auto younger;
 }

• 第二个host加上"()"，即指的是他需要向哪个机器推送数据，同时接受第一个host（未加"（）"）所在机器的数据。
• 创建第二个节点的配置文件
• 创建nginx1的csync2_nginx1.cfg文件

touch csync2_nginx2.cfg

nossl * *;
group nginx2
 {
        host (nginx1);
        host nginx2;
        key /etc/csync2.key;
        include /etc/nginx;
        exclude *~ .*;

action
 {
        pattern /etc/nginx/nginx.conf;
        pattern /etc/nginx/conf.d/*;
        exec "/usr/bin/systemctl reload nginx";
        logfile "/var/log/csync2_action.log";
        do-local;
 }

        backup-directory /opt/backup/csync2;
        backup-generations 3;
        auto younger;
 }

五、初始化同步csync2

• 在两台机器上执行

csync2 -xv

• 注意结尾处，如果正确会提示“Finished with 0 errors.”
• 可以测试新建个文件，执行csync2 -xv，然后到第二台机器上查看有没有同步。
• 到这里同步已经可以实现，只是不能实时而已。

六、安装配置lsyncd

• 还是继续用yum安装lsyncd

yum install lsyncd

如果安装成功，在etc下面会有lsyncd.conf。

vi /etc/lsyncd.conf

----
-- User configuration file for lsyncd.
--
-- Simple example for default rsync, but executing moves through on the target.
--
-- For more examples, see /usr/share/doc/lsyncd*/examples/
--
-- sync{default.rsyncssh, source="/var/www/html", host="localhost", targetdir="/tmp/htmlcopy/"}

settings {
        logident        = "lsyncd",
        logfacility     = "user",
        logfile         = "/var/log/lsyncd/lsyncd.log",
        statusFile      = "/var/log/lsyncd/status.log",
        statusInterval  = 1
}
initSync = {
        delay = 1,
        maxProcesses = 1,
        action = function(inlet)
                local config = inlet.getConfig()
                local elist = inlet.getEvents(function(event)
                        return event.etype ~= "Init"
                end)
                local directory = string.sub(config.source, 1, -2)
                local paths = elist.getPaths(function(etype, path)
                        return "\t" .. config.syncid .. ":" .. directory .. path
                end)
                log("Normal", "Processing syncing list:\n", table.concat(paths, "\n"))
                spawn(elist, "/usr/sbin/csync2", "-C", config.syncid, "-x")
        end,
        collect = function(agent, exitcode)
                local config = agent.config
                if not agent.isList and agent.etype == "Init" then
                        if exitcode == 0 then
                                log("Normal", "Startup of '", config.syncid, "' instance finished.")
                        elseif config.exitcodes and config.exitcodes[exitcode] == "again" then
                                log("Normal", "Retrying startup of '", config.syncid, "' instance.")
                                return "again"
                        else
                                log("Error", "Failure on startup of '", config.syncid, "' instance.")
                                terminate(-1)
                        end
                        return
                end
                local rc = config.exitcodes and config.exitcodes[exitcode]
                if rc == "die" then
                        return rc
                end
                if agent.isList then
                        if rc == "again" then
                                log("Normal", "Retrying events list on exitcode = ", exitcode)
                        else
                                log("Normal", "Finished events list = ", exitcode)
                        end
                else
                        if rc == "again" then
                                log("Normal", "Retrying ", agent.etype, " on ", agent.sourcePath, " = ", exitcode)
                        else
                                log("Normal", "Finished ", agent.etype, " on ", agent.sourcePath, " = ", exitcode)
                        end
                end
                return rc
        end,
        init = function(event)
                local inlet = event.inlet;
                local config = inlet.getConfig();
                log("Normal", "Recursive startup sync: ", config.syncid, ":", config.source)
                spawn(event, "/usr/sbin/csync2", "-C", config.syncid, "-x")
        end,
        prepare = function(config)
                if not config.syncid then
                        error("Missing 'syncid' parameter.", 4)
                end
                local c = "csync2_" .. config.syncid .. ".cfg"
                local f, err = io.open("/etc/" .. c, "r")
                if not f then
                        error("Invalid 'syncid' parameter: " .. err, 4)
                end
                f:close()
        end
}

local sources = {
        -- change the node1 value with respective host
        ["/etc/nginx"] = "nginx1"
}

for key, value in pairs(sources) do
        sync {initSync, source=key, syncid=value}
end

• local f, err = io.open("/etc/" … c, “r”)，这里的“/etc/”就可以看出来，会在/etc下面找对应的csync2_*.cfg，所以 local sources配置项是需要监听哪个目录就写哪个,后边的是指的csync的id,需要和csync2_1.cfg中的nginx1对应，这样就能通过lsyncd监听/etc/nginx目录了。
• 如果是多个监听目录的话，可以在这里配置，当然我们刚刚在csync2_*.cfg也可以配置。

["/etc/nginx/nginx.cnf"] = "nginx01",
["/etc/nginx/conf.d"] = "nginx01"

• 现在启动lsyncd：

service lsyncd start

• 查看启动日志

cat /var/log/lsyncd/lsyncd.log

关注公众号"双城人"，搬砖过程遇到的问题，大家一起探讨，资源共享