关于memcached的HA方案探究2

1. 使用repcached
    这个不做多的说明。
    该方案优点：两个Memcached都可以进行读写操作，互相冗余。缺点：只支持单对单（一个master和一个slave），怎么样在多个memcached中使用repcached，我没找到答案。

 

2. 使用MAgent
   项目地址：http://code.google.com/p/memagent/wiki/HowMagentWorks
   安装：

mkdir magent
cd magent
wget http://memagent.googlecode.com/files/magent-0.5.tar.gz
tar -xzvf magent-0.5.tar.gz  
/sbin/ldconfig
sed -i "s#LIBS = -levent#LIBS = -levent -lm#g" Makefile
make

 

分别在11211,11212,11213,11214端口启动4个memcached:

/usr/local/memcached/bin/memcached -d -p 11211 -u memcached -m 2048 -l 127.0.0.1 -p 11211 -c 300 -P/usr/local/memcached/memcached1.pid
/usr/local/memcached/bin/memcached -d -p 11212 -u memcached -m 2048 -l 127.0.0.1 -p 11212 -c 300 -P/usr/local/memcached/memcached2.pid
/usr/local/memcached/bin/memcached -d -p 11213 -u memcached -m 2048 -l 127.0.0.1 -p 11213 -c 300 -P/usr/local/memcached/memcached3.pid
/usr/local/memcached/bin/memcached -d -p 11214 -u memcached -m 2048 -l 127.0.0.1 -p 11214 -c 300 -P/usr/local/memcached/memcached4.pid

 

[root@hd026-test agent]# ps aux | grep mem
533       4097  0.0  0.0 196688  2224 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11211 -u memcached -m 2048 -l 127.0.0.1 -p 11211 -c 300 -P/usr/local/memcached/memcached1.pid
533       4105  0.0  0.0 131304  2224 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11212 -u memcached -m 2048 -l 127.0.0.1 -p 11212 -c 300 -P/usr/local/memcached/memcached2.pid
533       4112  0.0  0.0  65768  2216 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11213 -u memcached -m 2048 -l 127.0.0.1 -p 11213 -c 300 -P/usr/local/memcached/memcached3.pid
533       4123  0.0  0.0 131304  2212 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11214 -u memcached -m 2048 -l 127.0.0.1 -p 11214 -c 300 -P/usr/local/memcached/memcached4.pid
root      4248  0.0  0.0  61136   736 pts/0    S+   15:07   0:00 grep mem

 

分别在12000,12001端口启动两个MAgent:

 

./magent -u root -n 4000 -l 127.0.0.1 -p 12000 -s 127.0.0.1:11211 -s 127.0.0.1:11212 -b 127.0.0.1:11213 -b 127.0.0.1:11214 
./magent -u root -n 4000 -l 127.0.0.1 -p 12001 -s 127.0.0.1:11211 -s 127.0.0.1:11212 -b 127.0.0.1:11213 -b 127.0.0.1:11214 

 

启动参数说明：-p表示运行的端口号，-s表示指定那个memcached作为主，-b表示指定备份memcached；

 

[root@hd026-test agent]# ps aux | grep magent
root      4139  0.0  0.0  17048   520 ?        Ss   14:48   0:00 ./magent -u root -n 4000 -l 127.0.0.1 -p 12000 -s 127.0.0.1:11211 -s 127.0.0.1:11212 -b 127.0.0.1:11213 -b 127.0.0.1:11214
root      4169  0.0  0.0  17048   520 ?        Ss   14:52   0:00 ./magent -u root -n 4000 -l 127.0.0.1 -p 12001 -s 127.0.0.1:11211 -s 127.0.0.1:11212 -b 127.0.0.1:11213 -b 127.0.0.1:11214
root      4288  0.0  0.0  61136   740 pts/0    S+   15:11   0:00 grep magent

 

连接到MAgent-12000，插入key1:

 

[root@hd026-test agent]# telnet localhost 12000
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
set key1 0 0 5
11111
STORED
get key1
VALUE key1 0 5
11111
END
quit
Connection closed by foreign host.

 

连接到MAgent-12001，插入key2;

之后，分别连接Memcached-11211,Memcached-11212,Memcached-11213,Memcached-11214来get key1和key2,
发现key1在11212和11214中，key2在11211和11213中。
结构如下：

 
 

用户在连接到MAgent写数据时，MAgent会根据一定的算法将数据写入一个主memcached和一个备memcached;
这样，在主memcached挂掉时候，MAgent会从备memcached中读取数据，对用户来说是透明的。
实验一下：

kill掉Mem-11211：

 

kill 4097

[root@hd026-test agent]# ps aux | grep mem
533       4105  0.0  0.0 131304  2224 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11212 -u memcached -m 2048 -l 127.0.0.1 -p 11212 -c 300 -P/usr/local/memcached/memcached2.pid
533       4112  0.0  0.0  65768  2216 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11213 -u memcached -m 2048 -l 127.0.0.1 -p 11213 -c 300 -P/usr/local/memcached/memcached3.pid
533       4123  0.0  0.0 131304  2212 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11214 -u memcached -m 2048 -l 127.0.0.1 -p 11214 -c 300 -P/usr/local/memcached/memcached4.pid
root      4265  0.0  0.0  61136   740 pts/0    S+   15:09   0:00 grep mem

 

在连接到MAgent-12000或者MAgent-12001, get key2:

 

[root@hd026-test agent]# telnet localhost 12000
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
get key2
VALUE key2 0 5
22222
END
get key1
VALUE key1 0 5
11111
END
quit
Connection closed by foreign host.
[root@hd026-test agent]# telnet localhost 12001
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
get key2
VALUE key2 0 5
22222
END
get key1
VALUE key1 0 5
11111
END
quit
Connection closed by foreign host.

 

 

数据仍然可以正常获取。
使用该方案的优点是可以有多个memcached主从节点。
缺点是当一个主节点挂掉又恢复时候，从MAgent便会取不到数据。
测试如下：

重启11211端口的memcached

 

[root@hd026-test agent]# ps aux | grep mem
533       4105  0.0  0.0 131304  2224 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11212 -u memcached -m 2048 -l 127.0.0.1 -p 11212 -c 300 -P/usr/local/memcached/memcached2.pid
533       4112  0.0  0.0  65768  2216 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11213 -u memcached -m 2048 -l 127.0.0.1 -p 11213 -c 300 -P/usr/local/memcached/memcached3.pid
533       4123  0.0  0.0 131304  2212 ?        Ssl  14:47   0:00 /usr/local/memcached/bin/memcached -d -p 11214 -u memcached -m 2048 -l 127.0.0.1 -p 11214 -c 300 -P/usr/local/memcached/memcached4.pid
533       4270  0.0  0.0 130280  1144 ?        Ssl  15:09   0:00 /usr/local/memcached/bin/memcached -d -p 11211 -u memcached -m 2048 -l 127.0.0.1 -p 11211 -c 300 -P/usr/local/memcached/memcached1.pid
root      4279  0.0  0.0  61136   740 pts/0    S+   15:10   0:00 grep mem

 

[root@hd026-test agent]# telnet localhost 12000
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
get key2
END
get key1
VALUE key1 0 5
11111
END
quit
Connection closed by foreign host.

 

get key2,发现已经取不到数据，但其实key2在Mem-11213上还是存在的。

 

[root@hd026-test agent]# telnet localhost 11213
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
get key2
VALUE key2 0 5
22222
END
quit
Connection closed by foreign host.

 

 

这其实是个很致命的缺陷，不知道为何没人修复。。或许有其他的方案可以弥补。。