Twemproxy 介绍与使用
Twemproxy是一种代理分片机制,由Twitter开源。Twemproxy作为代理,可接受来自多个程序的访问,按照路由规则,转发给后台的各个Redis服务器,再原路返回。该方案很好的解决了单个Redis实例承载能力的问题。当然,Twemproxy本身也是单点,需要用Keepalived做高可用方案。通过Twemproxy可以使用多台服务器来水平扩张redis服务,可以有效的避免单点故障问题。虽然使用Twemproxy需要更多的硬件资源和在redis性能有一定的损失(twitter测试约20%),但是能够提高整个系统的HA也是相当划算的。不熟悉twemproxy的同学,如果玩过nginx反向代理或者mysql proxy,那么你肯定也懂twemproxy了。其实twemproxy不光实现了redis协议,还实现了memcached协议,什么意思?换句话说,twemproxy不光可以代理redis,还可以代理memcached,官方说明:
twemproxy (pronounced "two-em-proxy"), aka nutcracker is a fast and lightweight proxy for memcachedand redis protocol. It was built primarily to reduce the number of connections to the caching servers on the backend. This, together with protocol pipeling and sharding enables you to horizontally scale your distributed caching architecture.
Twemproxy架构:
但是从上面我们可以看到这样以来Twemproxy就成了单点,所以通常会结合keepalived来实现Twemproxy的高可用。架构图如下:
上面的架构通常只有一台Twemproxy在工作,另外一台处于备机,当一台挂掉以后,vip自动漂移,备机接替工作。关于keepalived相关的资料可以参考我前面相关文章。
1.编译安装:
m4下载地址:http://ftp.gnu.org/gnu/m4/m4-1.4.9.tar.gz
autoconf下载地址:http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
twemproxy下载地址:https://codeload.github.com/twitter/twemproxy/zip/master
twemproxy的安装要求autoconf的版本在2.64以上,否则提示"error: Autoconf version 2.64 or higher is required"。
1)安装m4
-
tar zxvf m4
-1.4
.9.tar.gz
-
cd m4
-1.4
.9
-
./configure &&
make &&
make install
2)安装libtool
-
tar zxvf libtool
-2.2
.4.tar.gz
-
cd libtool
-2.2
.4
-
./configure &&
make &&
make install
3)安装autoconf
安装最新版本
查找旧版本autoconf,并且卸载
rpm -qf /usr/bin/autoconf rpm -e --nodeps autoconf-2.63
安装最新版本
tar zxvf autoconf-2.69.tar.gz cd autoconf-2.69 ./configure --prefix=/usr make && make install
4)安装automake
-
tar zxvf automake
-1.12
.1.tar.gz
-
cd automake
-1.12
.1
-
./configure &&
make &&
make install
5)编译安装twemproxy
unzip twemproxy-master.zip cd twemproxy-master autoreconf -fvi ./configure --prefix=/usr/local/twemproxy make -j 8 make install
拷备nutcracker至/usr/local/sbin
cp /usr/local/twemproxy/sbin/nutcracker /usr/local/sbin/
2.创建相关目录(存放配置文件和pid文件)
cd /usr/local/twemproxy mkdir run conf log pid script
3.添加proxy配置文件
vim /usr/local/twemproxy/conf/nutcracker.yml
内容如下(配置文件参考官方)
alpha: listen: 127.0.0.1:22121 hash: fnv1a_64 distribution: ketama auto_eject_hosts: true redis: true server_retry_timeout: 2000 server_failure_limit: 1 servers: - 127.0.0.1:7000:1 - 127.0.0.1:7001:1 - 127.0.0.1:7002:1 - 127.0.0.1:7003:1 - 127.0.0.1:7004:1 - 127.0.0.1:7005:1
我在本地安装了6个redis实例,不清楚的同学参考redis官网安装或者我前面的文章Redis安装以及主从实现。
4.启动Twemproxy服务
nutcracker -t 测试配置文件
测试配置文件这里有个小坑,本来以为要指定配置文件路径,于是这样检查配置文件:
[root@redis-server ~]# nutcracker -t /usr/local/twemproxy/conf/nutcracker.yml nutcracker: configuration file 'conf/nutcracker.yml' syntax is invalid
后来才反应过来是检查命令所在路径的conf下面的nutcracker.yml文件,于是把conf目录复制到/usr/local/twemproxy/sbin/目录下,再次进行检测:
[root@redis-server sbin]# pwd /usr/local/twemproxy/sbin [root@redis-server sbin]# ll total 808 drwxr-xr-x 2 root root 4096 Apr 10 03:02 conf -rwxr-xr-x 1 root root 819245 Apr 9 23:26 nutcracker [root@redis-server sbin]# ./nutcracker -t nutcracker: configuration file 'conf/nutcracker.yml' syntax is ok [root@redis-server sbin]#
可以看见提示配置文件没有语法错误了。
启动命令:
nutcracker -d -c /usr/local/twemproxy/conf/nutcracker.yml -p /usr/local/twemproxy/run/redisproxy.pid -o /usr/local/twemproxy/run/redisproxy.log
nutcracker用法与命令选项
Usage: nutcracker [-?hVdDt] [-v verbosity level] [-o output file] [-c conf file] [-s stats port] [-a stats addr] [-i stats interval] [-p pid file] [-m mbuf size]
Options: -h, –help : 查看帮助文档,显示命令选项 -V, –version : 查看nutcracker版本 -t, –test-conf : 测试配置脚本的正确性 -d, –daemonize : 以守护进程运行 -D, –describe-stats : 打印状态描述 -v, –verbosity=N : 设置日志级别 (default: 5, min: 0, max: 11) -o, –output=S : 设置日志输出路径,默认为标准错误输出 (default: stderr) -c, –conf-file=S : 指定配置文件路径 (default: conf/nutcracker.yml) -s, –stats-port=N : 设置状态监控端口,默认22222 (default: 22222) -a, –stats-addr=S : 设置状态监控IP,默认0.0.0.0 (default: 0.0.0.0) -i, –stats-interval=N : 设置状态聚合间隔 (default: 30000 msec) -p, –pid-file=S : 指定进程pid文件路径,默认关闭 (default: off) -m, –mbuf-size=N : 设置mbuf块大小,以bytes单位 (default: 16384 bytes)
查看进程,确认启动。
[root@redis-server run]# ps -ef | grep nutcracker | grep -v grep root 809 1 0 03:09 ? 00:00:00 nutcracker -d -c /usr/local/twemproxy/conf/nutcracker.yml -p /usr/local/twemproxy/run/redisproxy.pid -o /usr/local/twemproxy/run/redisproxy.log [root@redis-server run]#
5. 简单测试
[root@redis-server ~]# netstat -nltp | grep nutcracker tcp 0 0 0.0.0.0:22222 0.0.0.0:* LISTEN 809/nutcracker tcp 0 0 127.0.0.1:22121 0.0.0.0:* LISTEN 809/nutcracker [root@redis-server ~]# redis-cli -p 22121 127.0.0.1:22121> set name yaun OK 127.0.0.1:22121> get name "yaun" 127.0.0.1:22121>
总结:
Twemproxy还是非常的靠谱,虽然性能有损失(20%),但是相对来说还是很值得的,而且久经考验,使用非常广泛。后面再进行性能测试。关于更多更加详细的资料请参考官方文档。
安装时遇到的问题
1.checking for GNU M4 that supports accurate traces... configure: error: no acceptable m4 could be found in $PATH.
-
[
root@rac1
autoconf-2.69]
# ./configure --prefix=/usr/
-
checking
for
a
BSD-compatible
install...
/usr/bin/install
-c
-
checking
whether
build
environment
is
sane...
yes
-
checking
for
a
thread-safe
mkdir
-p...
/usr/bin/mkdir
-p
-
checking
for
gawk...
gawk
-
checking
whether
make
sets
$(MAKE)...
yes
-
checking
build
system
type...
x86_64-unknown-linux-gnu
-
checking
host
system
type...
x86_64-unknown-linux-gnu
-
configure:
autobuild
project...
GNU
Autoconf
-
configure:
autobuild
revision...
2.69
-
configure:
autobuild
hostname...
rac1
-
configure:
autobuild
timestamp...
20200728T102412Z
-
checking
whether
/bin/sh
-n
is
known
to
work...
yes
-
checking
for
characters
that
cannot
appear
in
file
names...
none
-
checking
whether
directories
can
have
trailing
spaces...
yes
-
checking
for
expr...
/usr/bin/expr
-
checking for GNU M4 that supports accurate traces... configure: error:
no
acceptable
m4
could
be
found
in
$PATH.
-
GNU
M4
1.4
.6
or
later
is
required;
1.4
.16
or
newer
is
recommended.
-
GNU
M4
1.4
.15
uses
a
buggy
replacement
strstr
on
some
systems.
-
Glibc
2.9
-
2.12
and
GNU
M4
1.4
.11
-
1.4
.15
have
another
strstr
bug.
解决方式为安装m4
2.Can't exec "aclocal": No such file or directory at /usr/share/autoconf/Autom4te/FileUtils.pm line 326.
-
-
[
root@rac1
twemproxy-master]
# autoreconf -fvi
-
autoreconf:
Entering
directory
`.'
-
autoreconf: configure.ac:
not
using
Gettext
-
autoreconf: running:
aclocal
-I
m4
--output=aclocal.m4t
-
Can't
exec
"aclocal":
No
such
file
or
directory
at
/usr/share/autoconf/Autom4te/FileUtils.pm
line
326
.
-
autoreconf: failed to run aclocal:
No
such
file
or
directory
解决方式为安装automake
3.configure.ac:36: error: possibly undefined macro: AC_PROG_LIBTOOL
-
[
root@rac1
twemproxy-master]
# autoreconf -fvi
-
autoreconf:
Entering
directory
`.'
-
autoreconf: configure.ac:
not
using
Gettext
-
autoreconf: running:
aclocal
--force
-I
m4
-
autoreconf: configure.ac:
tracing
-
autoreconf: configure.ac:
adding
subdirectory
contrib/yaml-0.1.4
to
autoreconf
-
autoreconf:
Entering
directory
`contrib/yaml-0.1.4'
-
autoreconf: configure.ac:
not
using
Autoconf
-
autoreconf:
Leaving
directory
`contrib/yaml-0.1.4'
-
autoreconf: configure.ac:
creating
directory
config
-
autoreconf: configure.ac:
not
using
Libtool
-
autoreconf: running:
/usr/bin/autoconf
--force
-
configure.ac:36: error: possibly undefined macro:
AC_PROG_LIBTOOL
-
If
this
token
and
others
are
legitimate,
please
use
m4_pattern_allow.
-
See
the
Autoconf
documentation.
-
autoreconf: /usr/bin/autoconf failed with exit status:
1
解决方式为安装libtool