Linux-Memcached主主复制+Keepalived高可用

Linux-Memcached主主复制+Keepalived高可用

一.简介：
    Memcached主主复制是指在任意一台Memcached服务器修改数据都会被同步到另外一台。
    Memcached的复制功能支持多个Memcached之间进行互相复制(双向复制，主备都是可读可写的)，可以解决Memcached的容灾问题
    因为Memcached主主复制这种架构，在程序连接时不知道应该连接到哪个服务器，所以需要在前端家VIP地址，实现高可用架构。这里用Keepalived实现，因而Keepalived的作用是用来检测Memcached服务器状态是否正常
    Keepalived不断检测Memcached主服务器的11211端口，如果检测到Memcached服务发生宕机或死机等情况，就会将VIP从主服务器移至从服务器，从而实现Memcached的高可用性
    要使用Memcached复制架构，需要下载支持复制功能的Memcached安装包：
        路径：
            https://sourceforge.net/repcached/memcached-1.2.8-repcached-2.2.tar.gz
    二.实验环境如下：
        192.168.1.101   master      libevent-1.4.14b-stable.tar.gz memcached-1.2.8-repcached-2.2.tar.gz
        192.168.1.102   backup      libevent-1.4.14b-stable.tar.gz  memcached-1.2.8-repcached-2.2.tar.gz
    先决条件：
        1.关闭防火墙和SELINUX
        2.配置本地yum源
     [root@master ~]# cd  /opt

  [root@master opt]# rz

        [root@master opt]# tar xf libevent-1.4.14b-stable.tar.gz
        [root@master opt]# cd libevent-1.4.14b-stable
        [root@master libevent-1.4.14b-stable]# ./configure --prefix=/usr/local/libevent && make && make install
    3.安装带有复制功能的Memcached
    [root@master opt]# tar xf memcached-1.2.8-repcached-2.2.tar.gz
    [root@master opt]# ls
    libevent-1.4.14b-stable         memcached-1.2.8-repcached-2.2         
    libevent-1.4.14b-stable.tar.gz  memcached-1.2.8-repcached-2.2.tar.gz
    [root@master opt]# cd memcached-1.2.8-repcached-2.2
    [root@master memcached-1.2.8-repcached-2.2]# ./configure --prefix=/usr/local/memcached_replication --enable-replication --with-libevent=/usr/local/libevent && make && make install

注意: 如果make的时候报错

memcached.c: 在函数'add_iov'中: memcached.c:696:30: 错误: 'IOV_MAX'未声明(在此函数内第一次使用) memcached.c:696:30: 附注: 每个未声明的标识符在其出现的函数内只报告一次 make[2]: *** [memcached-memcached.o] 错误 1

需要修改memcached.c文件:

/* FreeBSD 4.x doesn't have IOV_MAX exposed. */

#ifndef IOV_MAX

#if defined(__FreeBSD__) || defined(__APPLE__)

# define IOV_MAX 1024

#endif

#endif

复制

改成:

/* FreeBSD 4.x doesn't have IOV_MAX exposed. */

#ifndef IOV_MAX

# define IOV_MAX 1024

#endif

复制

注意: 如果运行/usr/local/bin/memcached -h的时候报错:找不到libevent** 执行: ln -s /usr/local/lib/libevent-1.4.so.2 /usr/lib64/libevent-1.4.so.2 技巧: 执行ldd /usr/local/bin/memcached,看缺少那些库!

 终端调试运行:

执行: /usr/local/bin/memcached -m 100 -u root -p 11411 也可以启动多个守护进程,但是端口不能重复 启动参数说明:

-d 选项是启动一个守护进程,

-m 是分配给Memcache使用的内存数量,单位是MB,默认64MB -M return error on memory exhausted (rather than removing items)

 -u 是运行Memcache的用户,如果当前为root 的话,需要使用此参数指定用户.

-l 是监听的服务器IP地址,默认为所有网卡.

 -p 是设置Memcache的TCP监听的端口,最好是1024以上的端口

 -c 选项是最大运行的并发连接数,默认是1024

-P 是设置保存Memcache的pid文件

 -f chunk size growth factor (default: 1.25) -I Override the size of each slab page. Adjusts max item size(1.4.2版本新增)

    4.启动Memcached服务
        支持复制功能的Memcached安装完成之后，需要将编译安装的libevent-1.4.so.2模块复制到/usr/lib64目录下，否则在启动带有复制功能的Memcached服务时会报错
            [root@backup memcached-1.2.8-repcached-2.2]# ln -s /usr/local/libevent/lib/libevent-1.4.so.2 /usr/lib64

编写服务控制脚本

#vim  /etc/rc.d/init.d/memcached.sh

#!/bin/bash

CMD="/usr/local/memcached_replication/bin/memcached"

start(){

 $CMD -d -m 128 -u root

}

stop(){

 killall memcached;

}

ACTION=$1

 case $ACTION in

 'start')

    start;;

 'stop')

   stop;;

 'restart')

   stop

  sleep 2

  start;;

 *)

 echo 'Usage:{start | stop | restart}'

esac

#chmod   +x   /etc/rc.d/init.d/memcached.sh

                     

 #systemctl      daemon-reload

#systemctl      start   memcached

  5.启动服务时，使用-x指定对端
            [root@backup memcached-1.2.8-repcached-2.2]# /usr/local/memcached_replication/bin/memcached -d -u root -m 128 -x 192.168.1.101
            [root@backup memcached-1.2.8-repcached-2.2]# netstat -anpt | grep memcached
            tcp        0      0 0.0.0.0:11211           0.0.0.0:*               LISTEN      7520/memcached      
            tcp        0      0 0.0.0.0:11212           0.0.0.0:*               LISTEN      7520/memcached      
            tcp6       0      0 :::11211                :::*                    LISTEN      7520/memcached
    6.使用telnet进行简单的验证复制功能
        (1.)在master上插入一条具有特点的键值
        [root@master opt]# telnet 192.168.1.101 11211
            Trying 192.168.1.101...
            Connected to 192.168.1.101.
            Escape character is '^]'.
            set username 0 0 8
            20170226
            STORED
            get username
            VALUE username 0 8
            20170226
            END
            quit
            Connection closed by foreign host.
        (2.)在backup上面查看刚才插入的数据
            [root@backup ~]# telnet 192.168.1.102 11211
            Trying 192.168.1.102...
            Connected to 192.168.1.102.
            Escape character is '^]'.
            get username
            VALUE username 0 8
            20170226
            END
            get username2
            END
            quit
            Connection closed by foreign host
        同理，在backup上插入的数据，在master上也可以查看到，这就是Memcached的主主复制
    7..安装配置Keepalived
        [root@master opt]# yum install keepalived -y
    8.配置主Keepalived

#vim /etc/keepalived/keepalived.conf

        ! Configuration File for keepalived
        global_defs {
           router_id LVS_DEVEL
        }
        vrrp_sync_group cluster {
                group {
                        mem_ba
            }
        }
        vrrp_instance VI_1 {
            state MASTER
            interface ens33
            virtual_router_id 51
            priority 100
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 1111
            }
            virtual_ipaddress {
                192.168.1.250
            }
        }
        virtual_server 192.168.1.250 11211 {
            delay_loop 6
            persistence_timeout 20
            protocol TCP
            sorry_server 192.168.1.102 11211     # 对端
            real_server 192.168.1.101 11211 {       # 本机
                weight 3
            notify_down /root/memcached.sh
                TCP_CHECK {
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                    connect_port 11211
                }
            }
        }
    设置执行脚本如下：
        [root@master keepalived]# echo "/usr/bin/systemctl stop keepalived" > memcached.sh
        [root@master keepalived]# chmod +x memcached.sh
    10.配置备Keepalived

#vim /etc/keepalived/keepalived.conf

        ! Configuration File for keepalived
        global_defs {
           router_id LVS_DEVEL
        }
        vrrp_sync_group cluster {
                group {
                        mem_ba
            }
        }
        vrrp_instance VI_1 {
            state MASTER
            interface ens33
            virtual_router_id 51
            priority 90
            advert_int 1
            authentication {
                auth_type PASS
                auth_pass 1111
            }
            virtual_ipaddress {
                192.168.1.250
            }
        }
        virtual_server 192.168.1.250 11211 {
            delay_loop 6
            persistence_timeout 20
            protocol TCP
            sorry_server 192.168.1.101 11211
            real_server 192.168.1.102 11211 {
                weight 3
            notify_down /root/memcached.sh
                TCP_CHECK {
                    connect_timeout 3
                    nb_get_retry 3
                    delay_before_retry 3
                    connect_port 11211
                }
            }
        }
        同样设置脚本如下：
            [root@backup ~]# echo "/usr/bin/systemctl stop keepalived" > memcached.sh
            [root@backup ~]# chmod +x memcached.sh
    11.测试验证
        分别启动主从的keepalived服务
            [root@master ~]# systemctl start keepalived
            [root@backup ~]# systemctl start keepalived
        (1.)验证主keepalived获取VIP
            使用ip address show命令查看VIP地址：
                [root@master ~]# ip a
                    2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
                        link/ether 00:0c:29:5c:93:64 brd ff:ff:ff:ff:ff:ff
                        inet 192.168.1.101/24 brd 192.168.1.255 scope global ens33
                           valid_lft forever preferred_lft forever
                        inet 192.168.1.250/32 scope global ens33
                           valid_lft forever preferred_lft forever
                        inet6 fe80::f32f:df18:99a7:1152/64 scope link
                           valid_lft forever preferred_lft forever
        (2.)验证高可用性
            关闭master服务器的Memcached服务，在backup服务器上查看地址信息

[root@master ~]#yum install psmisc -y
                [root@master ~]# killall memcached
                [root@backup ~]# ip a
                    2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
                        link/ether 00:0c:29:13:d9:b8 brd ff:ff:ff:ff:ff:ff
                        inet 192.168.1.102/24 brd 192.168.1.255 scope global noprefixroute ens33
                           valid_lft forever preferred_lft forever
                        inet 192.168.1.250/32 scope global ens33
                           valid_lft forever preferred_lft forever
                        inet6 fe80::832b:b762:6f75:d47b/64 scope link noprefixroute
                           valid_lft forever preferred_lft forever