Memcached + Keepalived 实现主主复制

1、环境

服务器名称	IP地址	备注
VIP	10.32.161.130	VIP
memcached-161-131	10.32.161.131	Memcached+Keepalived
memcached-161-132	10.32.161.131	Memcached+Keepalived

2、软件包（两个节点）

memcached-1.2.8-repcached-2.2.1.tar.gz
libevent-2.1.12-stable.tar.gz

 3、Memcached Repcached安装（两个节点）

# Libevent安装
tar xf libevent-2.1.12-stable.tar.gz
cd libevent-2.1.12-stable
./configure --prefix=/usr/local/libevent
make && make install
 
# Memcached Repcached安装
tar xf memcached-1.2.8-repcached-2.2.1.tar.gz
cd memcached-1.2.8-repcached-2.2.1
 
vim memcached.c
原代码如下（55~60行代码）：
/* FreeBSD 4.x doesn't have IOV_MAX exposed. */
#ifndef IOV_MAX
#if defined(__FreeBSD__) || defined(__APPLE__)
# define IOV_MAX 1024
#endif
#endif
修改为：
/* FreeBSD 4.x doesn't have IOV_MAX exposed. */
#ifndef IOV_MAX
//#if defined(__FreeBSD__) || defined(__APPLE__)
# define IOV_MAX 1024
//#endif
#endif
 
./configure --prefix=/usr/local/memcached-repcached --with-libevent=/usr/local/libevent/ --enable-replication
make && make install

4、参数

# ./memcached -h
memcached 1.2.8
repcached 2.2.1
-p <num>      TCP port number to listen on (default: 11211)
-U <num>      UDP port number to listen on (default: 11211, 0 is off)
-s <file>     unix socket path to listen on (disables network support)
-a <mask>     access mask for unix socket, in octal (default 0700)
-l <ip_addr>  interface to listen on, default is INDRR_ANY
-d            run as a daemon
-r            maximize core file limit
-u <username> assume identity of <username> (only when run as root)
-m <num>      max memory to use for items in megabytes, default is 64 MB
-M            return error on memory exhausted (rather than removing items)
-c <num>      max simultaneous connections, default is 1024
-k            lock down all paged memory.  Note that there is a
              limit on how much memory you may lock.  Trying to
              allocate more than that would fail, so be sure you
              set the limit correctly for the user you started
              the daemon with (not for -u <username> user;
              under sh this is done with 'ulimit -S -l NUM_KB').
-v            verbose (print errors/warnings while in event loop)
-vv           very verbose (also print client commands/reponses)
-h            print this help and exit
-i            print memcached and libevent license
-P <file>     save PID in <file>, only used with -d option
-f <factor>   chunk size growth factor, default 1.25
-n <bytes>    minimum space allocated for key+value+flags, default 48
-R            Maximum number of requests per event
              limits the number of requests process for a given con nection
              to prevent starvation.  default 20
-b            Set the backlog queue limit (default 1024)
-x <ip_addr>  hostname or IP address of peer repcached
-X <num:num>  TCP port number for replication. <listen:connect> (default: 11212)

5、服务启动

# 131上启动服务
cd /usr/local/memcached-repcached/
./bin/memcached -d -u root -m 128 -x 10.32.161.132
 
# 132上启动服务
cd /usr/local/memcached-repcached/
./bin/memcached -d -u root -m 128 -x 10.32.161.131

6、Keelalived安装及配置

# 安装软件
dnf install keepalived -y
 
# 修改配置（131节点）
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
        
global_defs {
     router_id lb01
}
 
vrrp_script chk_memcached_port {
    script "/usr/local/memcached/check_memcached.sh"
    interval 2
    fall 2
    rise 1
}
 
vrrp_instance VI_1 {
    state MASTER
    interface ens33  
    virtual_router_id 51
    priority 101
    advert_int 1
    authentication {  
        auth_type PASS
        auth_pass 1234
    }
    virtual_ipaddress {
        10.32.161.130
    }
    track_script {
        chk_memcached_port
    }
}
 
# 修改配置（132节点）
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
        
global_defs {
     router_id lb01
}
 
vrrp_script chk_memcached_port {
    script "/usr/local/memcached/check_memcached.sh"
    interval 2
    fall 2
    rise 1
}
 
vrrp_instance VI_1 {
    state BACKUP
    interface ens33  
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {  
        auth_type PASS
        auth_pass 1234
    }
    virtual_ipaddress {
        10.32.161.130
    }
    track_script {
        chk_memcached_port
    }
}
 
# 启动服务
systemctl enable --now keepalived

　　memcached状态监测脚本

vim check_memcached.sh
#!/bin/bash
#
counter=$(netstat -an | grep "LISTEN" | grep "11211" | wc -l)
if [ "${counter}" -eq 0 ]; then
    systemctl stop keepalived
fi

7、验证

# telnet 10.32.161.130 11211
Trying 10.32.161.130...
Connected to 10.32.161.130.
Escape character is '^]'.
stats
STAT pid 11185
STAT uptime 1839
STAT time 1678693510
STAT version 1.2.8
STAT pointer_size 64
STAT rusage_user 0.046593
STAT rusage_system 0.031851
STAT curr_items 1
STAT total_items 1
STAT bytes 84
STAT curr_connections 7
STAT total_connections 14
STAT connection_structures 8
STAT cmd_flush 0
STAT cmd_get 1
STAT cmd_set 1
STAT get_hits 1
STAT get_misses 0
STAT evictions 0
STAT bytes_read 102
STAT bytes_written 1424
STAT limit_maxbytes 134217728
STAT threads 2
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT replication MASTER
STAT repcached_version 2.2.1
STAT repcached_qi_free 8191
STAT repcached_wdata 0
STAT repcached_wsize 2048
END
set test_key 0 0 10
test_value
STORED
get test_key
VALUE test_key 0 10
test_value
END
quit
Connection closed by foreign host.

软件包下载地址：

https://sourceforge.net/projects/repcached/files/repcached/
https://github.com/libevent/libevent