一.memcache的简单介绍
memcache是一套开放源的分布式高速缓存系统。由服务端和客户端组成,以守护程序(监听)方式运行于一个或多个服务器中,随时会接收客户端的连接和操作。
memcache主要把数据对象缓存到内存中,通过在内存里维护一个统一的巨大的hash表。也就是说将数据调用到内存中,然后从内存中读取,从而大大提高读取速度。
memcache基于一个存储键/值对的hashmap进行存储对象到内存中。‘
memcache是用C写的,但是客户端可以用任何语言来编写,并通过memcached协议与守护进程通信
特点:
可以保存的item数据是没有限制的只要内存足够
单进程在32位系统中最大使用内存为2G,若在64位系统则没有限制,这是由于32位系统限制单进程最多可使用2G内存,要使用更多内存,可以分多个端口开启多个Memcached进程
最大30天的数据过期时间,设置为永久的也会在这个时间过期,常量REALTIME_MAXDELTA
单个item最大数据是1MB,超过1MB数据不予存储,常量POWER_BLOCK 1048576进行控制
二.php增加memcache模块
1.工作原理
在php增加memcache中,memcache相当于缓存。当客户需要资源时,由php进行解析,php会先检查客户的请求是否缓存在memcache中,如果是,则直接从memcache中调取数据给客户端不会进行解析,如果没有则php会将请求反馈给后端服务器及逆向嗯解析,解析好的数据一份给客户端,一份缓存在memcache中,下此如果有相同的翻跟请求,则直接从memcache中调取数据给客户端,不会再进行解析
客户的请求流程为
客户端请求—>nginx—>php—>解析结果—>nginx—>客户端
当第一次访问以后有缓存则流程为
客户---->web(nginx)---->php---->缓存中直接调取---->nginx---->客户
2.memcache模块的安装与编译
[root@server4 ~]# vim ~/.bash_profile ##更改环境变量,php的bin目录路径增加到 ~/.bash_profile中,为了方便调用
PATH=$PATH:$HOME/bin:/usr/local/lnmp/mysql/bin:/usr/local/lnmp/php/bin
export PATH
[root@server4 ~]# source ~/.bash_profile
[root@server4 ~]# cd /lnmp
[root@server4 lnmp]# ls
apache-tomcat-7.0.37.tar.gz nginx-1.10.1.tar.gz
cmake-2.8.12.2-2.el7.x86_64.rpm nginx-1.14.0.tar.gz
cmake-2.8.12.2-4.el6.x86_64.rpm nginx-1.15.9
Discuz_X3.2_SC_UTF8.zip nginx-1.15.9.tar.gz
jar nginx-1.8.0-1.el6.ngx.x86_64.rpm
jdk-7u79-linux-x64.tar.gz nginx-sticky-module-ng.tar.gz
libmcrypt-2.5.8-9.el6.x86_64.rpm openresty-1.13.6.1.tar.gz
libmcrypt-devel-2.5.8-9.el6.x86_64.rpm php-5.6.35
lnmp php-5.6.35.tar.bz2
memcache-2.2.5.tgz re2c-0.13.5-1.el6.x86_64.rpm
mysql-5.7.17 rhel6 lanmp.pdf
mysql-boost-5.7.17.tar.gz
[root@server4 lnmp]# tar zxf memcache-2.2.5.tgz
[root@server4 lnmp]# cd memcache-2.2.5
[root@server4 memcache-2.2.5]# ./configure ##memcache中没有./configure文件,所以此命令无法编译
-bash: ./configure: No such file or directory
[root@server4 memcache-2.2.5]# phpize ##扩展php
configuring for:
php api version: 20131106
zend module api no: 20131226
zend extension api no: 220131226
cannot find autoconf. please check your autoconf installation and the
$php_autoconf environment variable. then, rerun this script. ##报错,缺少autoconf
[root@server4 memcache-2.2.5]# yum install autoconf -y ##安装编译工具插件
[root@server4 memcache-2.2.5]# phpize
Configuring for:
PHP Api Version: 20131106
Zend Module Api No: 20131226
Zend Extension Api No: 220131226
[root@server4 memcache-2.2.5]# ./configure
[root@server4 memcache-2.2.5]# ./configure ##编译
[root@server4 memcache-2.2.5]# make && make install
[root@server4 memcache-2.2.5]# yum install memcached -y
[root@server4 memcache-2.2.5]# systemctl start memcached
[root@server4 memcache-2.2.5]# netstat -antlp ##查看memcache的端口为11211
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:11211 0.0.0.0:* LISTEN 4917/memcached
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 648/sshd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 884/master
tcp 0 0 127.0.0.1:9000 0.0.0.0:* LISTEN 4871/php-fpm: maste
tcp 0 0 172.25.4.114:22 172.25.4.250:36828 ESTABLISHED 2216/sshd: root@pts
tcp 0 0 172.25.4.114:22 172.25.4.250:36726 ESTABLISHED 2148/sshd: root@pts
tcp 0 0 172.25.4.114:22 172.25.4.250:36736 ESTABLISHED 2185/sshd: root@pts
tcp6 0 0 :::11211 :::* LISTEN 4917/memcached
tcp6 0 0 :::22 :::* LISTEN 648/sshd
tcp6 0 0 ::1:25 :::* LISTEN 884/master
命令:phpize
phpize是用来扩展php扩展模块的,通过phpize可以建立php的外挂模块,比如你想在原来编译好的php中加入memcached或者ImageMagick等扩展模块,可以使用phpize。或者说,phpize的作用可以这样理解:侦测环境(phpize工具是在php安装目录下,基于这点phpize对应了当时的php环境,所以是要根据该php的配置情况生成对应的configure文件),建立一个configure文件。必须在一个目录下去运行phpize。那么phpize就知道你的的环境是哪个目录,并且configure文件建立在该目下
[root@server4 memcache-2.2.5]# vim /usr/local/lnmp/php/etc/php.ini ##将memcache模块添加到php中
[root@server4 memcache-2.2.5]# php -m | grep memcache ##模块添加成功
memcache
[root@server4 memcache-2.2.5]# telnet localhost 11211 ##测试缓存是否可用
Trying ::1...
Connected to localhost.
Escape character is '^]'.
set name 0 0 4 ##设置key:name; 编号:0; 缓存时间 :无限 ;valuce值:4
luck ##输入的value值
STORED
get name ##通过key值查找对应的value值
VALUE name 0 4
luck
END
set name 0 6 6 ##设置缓存时间为6s,6s后自动删除
westos
STORED
get name ##6s后查看无value值
END
quit
Connection closed by foreign host.
测试php增加的memcache模块
[root@server4 memcache-2.2.5]# cp memcache.php example.php /usr/local/lnmp/nginx/html/ ##将memcache中用来测试php页面的文件复制到nginx的默认发布目录中
[root@server4 memcache-2.2.5]# cd /usr/local/lnmp/nginx/html
[root@server4 html]# vim memcache.php
22 define('ADMIN_USERNAME','memcache'); // Admin Username
23 define('ADMIN_PASSWORD','westos'); // Admin Password
24 define('DATE_FORMAT','Y/m/d H:i:s');
25 define('GRAPH_SIZE',200);
26 define('MAX_ITEM_DUMP',50);
27
28 $MEMCACHE_SERVERS[] = '172.25.4.114:11211'; // add more as an array
29 $MEMCACHE_SERVERS[] = 'mymemcache-server2:11211'; // add more as an array
[root@server4 html]# vim index.php
[root@server4 html]# cat index.php
<?php
phpinfo()
?>
[root@server4 html]# nginx
[root@server4 html]# nginx -s reload
浏览器测试监控界面是否可用
http://172.25.4.114/memcache.php
物理机测试:对两个界面均进行压力测试
[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/index.php
Server Software: nginx/
Server Hostname: 172.25.4.114
Server Port: 80
Document Path: /index.php
Document Length: 83827 bytes
Concurrency Level: 10
Time taken for tests: 35.863 seconds
Complete requests: 5000
Failed requests: 396
(Connect: 0, Receive: 0, Length: 396, Exceptions: 0)
Write errors: 0
Total transferred: 419919566 bytes
HTML transferred: 419134566 bytes
Requests per second: 139.42 [#/sec] (mean)
Time per request: 71.726 [ms] (mean)
Time per request: 7.173 [ms] (mean, across all concurrent requests)
Transfer rate: 11434.57 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 20.0 0 1001
Processing: 9 71 9.3 69 252
Waiting: 2 64 8.0 63 122
Total: 10 72 22.0 69 1071
Percentage of the requests served within a certain time (ms)
50% 69
66% 73
75% 75
80% 77
90% 81
95% 88
98% 97
99% 103
100% 1071 (longest request)
[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/example.php
Server Software: nginx/
Server Hostname: 172.25.4.114
Server Port: 80
Document Path: /example.php
Document Length: 0 bytes
Concurrency Level: 10
Time taken for tests: 3.519 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Non-2xx responses: 5000
Total transferred: 880000 bytes
HTML transferred: 0 bytes
Requests per second: 1421.01 [#/sec] (mean)
Time per request: 7.037 [ms] (mean)
Time per request: 0.704 [ms] (mean, across all concurrent requests)
Transfer rate: 244.24 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.2 0 4
Processing: 2 7 0.6 7 10
Waiting: 1 7 0.6 7 9
Total: 2 7 0.6 7 10
Percentage of the requests served within a certain time (ms)
50% 7
66% 7
75% 7
80% 7
90% 8
95% 8
98% 8
99% 9
100% 10 (longest request)
由此可知:访问index.php很慢,因为没有缓存,而且failed比较多;访问example很快,命中率也高,因为第一次访问以后有缓存
三.nginx添加memcache模块(缓存前移)
原nginx不支持memcache,需要使用OpenResty
1.工作原理
将memcache模块加到nginx服务器中,当客户有请求时,ngiinx先从memcache中寻找数据,如果有数据则直接反馈客户;如果说没有,则 去后端请求数据,将请求得到的数据一份给客户端,一份给memcache用于缓存,下次有同样请求时,则直接从memcache 中拿取数据给客户端
2.什么是openresty
OpenResty(又称:ngx_openresty) 是一个基于 NGINX 的可伸缩的web平台,由中国人章亦春发起,提供了很多高质量的第三方模块
OpenResty 是一个强大的 Web 应用服务器,Web 开发人员可以使用 Lua 脚本语言调动 Nginx 支持的各种 C 以及Lua 模块,更主要的是在性能方面,OpenResty可以 快速构造出足以胜任 10K 以上并发连接响应的超高性能 Web 应用系统
OpenResty 的目标是让 Web 服务直接跑在 Nginx 服务内部,充分利用 Nginx 的非阻塞 I/O 模型,不仅仅对 HTTP 客户端请求,甚至于对远程后端诸如mysql、PostgreSQL、 ~Redis ~Memcaches 等都进行一致的高性能响应
3.openresty的安装编译
[root@server4 ~]# nginx -s stop ##关闭之前的nginx
[root@server4 ~]# cd /lnmp
[root@server4 lnmp]# tar zxf openresty-1.13.6.1.tar.gz
[root@server4 lnmp]# cd openresty-1.13.6.1
[root@server4 openresty-1.13.6.1]# ./configure --prefix=/usr/local/openresty
[root@server4 openresty-1.13.6.1]# gmake && gmake install
4.nginx配置文件的修改
root@server4 openresty-1.13.6.1]# cd /usr/local/openresty/
[root@server4 openresty]# ls
bin COPYRIGHT luajit lualib nginx pod resty.index site
[root@server4 openresty]# cd nginx/
[root@server4 nginx]# cd conf
root@server4 conf]# vim nginx.conf
2 user nginx nginx;
17 http {
18 include mime.types;
19 default_type application/octet-stream;
20 upstream memcache {
21 server localhost:11211;
22 keepalive 512;
23 }
68 location /memc {
69 internal; ##只接收内部的访问,不接收外部的http访问,不安全
70 memc_connect_timeout 100ms;
71 memc_send_timeout 100ms; ##后端服务器数据传回时间
72 memc_read_timeout 100ms; ##连接成功后,后端服务器响应时间
73 set $memc_key $query_string;
74 set $memc_exptime 300;
75 memc_pass memcache;
76 }
77 location ~ \.php$ {
78 set $key $uri$args;
79 srcache_fetch GET /memc $key; ##(这两个配置的作用是:请求php页面时,先会去memcache中找,如果没有,正常访问;)
80 srcache_store PUT /memc $key; ##(访问结束后将结果存到memcache,下次访问时直接从缓存中)
81 root html;
82 fastcgi_pass 127.0.0.1:9000;
83 fastcgi_index index.php;
84 # fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;
85 include fastcgi.conf;
86 }
[root@server4 conf]# /usr/local/openresty/nginx/sbin/nginx
物理机测压:
[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/example.php
Server Software: openresty/1.13.6.1
Server Hostname: 172.25.4.114
Server Port: 80
Document Path: /example.php
Document Length: 16 bytes
Concurrency Level: 10
Time taken for tests: 3.304 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Non-2xx responses: 5000
Total transferred: 960000 bytes
HTML transferred: 80000 bytes
Requests per second: 1513.17 [#/sec] (mean)
Time per request: 6.609 [ms] (mean)
Time per request: 0.661 [ms] (mean, across all concurrent requests)
Transfer rate: 283.72 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 8
Processing: 2 6 0.9 6 16
Waiting: 1 6 0.9 6 16
Total: 3 7 1.0 7 16
Percentage of the requests served within a certain time (ms)
50% 7
66% 7
75% 7
80% 7
90% 8
95% 8
98% 9
99% 10
100% 16 (longest request)
[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/index.php
Server Software: openresty/1.13.6.1
Server Hostname: 172.25.4.114
Server Port: 80
Document Path: /index.php
Document Length: 16 bytes
Concurrency Level: 10
Time taken for tests: 3.213 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Non-2xx responses: 5000
Total transferred: 960000 bytes
HTML transferred: 80000 bytes
Requests per second: 1556.29 [#/sec] (mean)
Time per request: 6.426 [ms] (mean)
Time per request: 0.643 [ms] (mean, across all concurrent requests)
Transfer rate: 291.80 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 4
Processing: 2 6 0.9 6 10
Waiting: 1 6 0.9 6 10
Total: 2 6 0.9 6 11
Percentage of the requests served within a certain time (ms)
50% 6
66% 7
75% 7
80% 7
90% 7
95% 8
98% 8
99% 9
100% 11 (longest request)
相比于php增加memcache模块,此时利用缓存前移缓存速度更快