Linux--memcache模块

一.memcache的简单介绍
memcache是一套开放源的分布式高速缓存系统。由服务端和客户端组成，以守护程序(监听)方式运行于一个或多个服务器中，随时会接收客户端的连接和操作。
memcache主要把数据对象缓存到内存中，通过在内存里维护一个统一的巨大的hash表。也就是说将数据调用到内存中，然后从内存中读取，从而大大提高读取速度。
memcache基于一个存储键/值对的hashmap进行存储对象到内存中。‘
memcache是用C写的，但是客户端可以用任何语言来编写，并通过memcached协议与守护进程通信
特点：
可以保存的item数据是没有限制的只要内存足够
单进程在32位系统中最大使用内存为2G，若在64位系统则没有限制,这是由于32位系统限制单进程最多可使用2G内存,要使用更多内存，可以分多个端口开启多个Memcached进程
最大30天的数据过期时间,设置为永久的也会在这个时间过期，常量REALTIME_MAXDELTA
单个item最大数据是1MB，超过1MB数据不予存储，常量POWER_BLOCK 1048576进行控制
二.php增加memcache模块
1.工作原理
在php增加memcache中，memcache相当于缓存。当客户需要资源时，由php进行解析，php会先检查客户的请求是否缓存在memcache中，如果是，则直接从memcache中调取数据给客户端不会进行解析，如果没有则php会将请求反馈给后端服务器及逆向嗯解析，解析好的数据一份给客户端，一份缓存在memcache中，下此如果有相同的翻跟请求，则直接从memcache中调取数据给客户端，不会再进行解析
客户的请求流程为
客户端请求—>nginx—>php—>解析结果—>nginx—>客户端
当第一次访问以后有缓存则流程为
客户---->web(nginx)---->php---->缓存中直接调取---->nginx---->客户
2.memcache模块的安装与编译

[root@server4 ~]# vim ~/.bash_profile  ##更改环境变量，php的bin目录路径增加到 ~/.bash_profile中，为了方便调用
PATH=$PATH:$HOME/bin:/usr/local/lnmp/mysql/bin:/usr/local/lnmp/php/bin
export PATH
[root@server4 ~]# source ~/.bash_profile 
[root@server4 ~]# cd /lnmp
[root@server4 lnmp]# ls
apache-tomcat-7.0.37.tar.gz             nginx-1.10.1.tar.gz
cmake-2.8.12.2-2.el7.x86_64.rpm         nginx-1.14.0.tar.gz
cmake-2.8.12.2-4.el6.x86_64.rpm         nginx-1.15.9
Discuz_X3.2_SC_UTF8.zip                 nginx-1.15.9.tar.gz
jar                                     nginx-1.8.0-1.el6.ngx.x86_64.rpm
jdk-7u79-linux-x64.tar.gz               nginx-sticky-module-ng.tar.gz
libmcrypt-2.5.8-9.el6.x86_64.rpm        openresty-1.13.6.1.tar.gz
libmcrypt-devel-2.5.8-9.el6.x86_64.rpm  php-5.6.35
lnmp                                    php-5.6.35.tar.bz2
memcache-2.2.5.tgz                      re2c-0.13.5-1.el6.x86_64.rpm
mysql-5.7.17                            rhel6 lanmp.pdf
mysql-boost-5.7.17.tar.gz
[root@server4 lnmp]# tar zxf memcache-2.2.5.tgz
[root@server4 lnmp]# cd memcache-2.2.5
[root@server4 memcache-2.2.5]# ./configure  ##memcache中没有./configure文件，所以此命令无法编译
-bash: ./configure: No such file or directory
[root@server4 memcache-2.2.5]# phpize  ##扩展php
configuring for:
php api version:         20131106
zend module api no:      20131226
zend extension api no:   220131226
cannot find autoconf. please check your autoconf installation and the
$php_autoconf environment variable. then, rerun this script.  ##报错，缺少autoconf
[root@server4 memcache-2.2.5]# yum install autoconf -y  ##安装编译工具插件
[root@server4 memcache-2.2.5]# phpize
Configuring for:
PHP Api Version:         20131106
Zend Module Api No:      20131226
Zend Extension Api No:   220131226
[root@server4 memcache-2.2.5]# ./configure
[root@server4 memcache-2.2.5]# ./configure  ##编译
[root@server4 memcache-2.2.5]# make && make install
[root@server4 memcache-2.2.5]# yum install memcached -y
[root@server4 memcache-2.2.5]# systemctl start memcached
[root@server4 memcache-2.2.5]# netstat -antlp  ##查看memcache的端口为11211
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:11211           0.0.0.0:*               LISTEN      4917/memcached      
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      648/sshd            
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      884/master          
tcp        0      0 127.0.0.1:9000          0.0.0.0:*               LISTEN      4871/php-fpm: maste 
tcp        0      0 172.25.4.114:22         172.25.4.250:36828      ESTABLISHED 2216/sshd: root@pts 
tcp        0      0 172.25.4.114:22         172.25.4.250:36726      ESTABLISHED 2148/sshd: root@pts 
tcp        0      0 172.25.4.114:22         172.25.4.250:36736      ESTABLISHED 2185/sshd: root@pts 
tcp6       0      0 :::11211                :::*                    LISTEN      4917/memcached      
tcp6       0      0 :::22                   :::*                    LISTEN      648/sshd            
tcp6       0      0 ::1:25                  :::*                    LISTEN      884/master

命令：phpize
phpize是用来扩展php扩展模块的，通过phpize可以建立php的外挂模块，比如你想在原来编译好的php中加入memcached或者ImageMagick等扩展模块，可以使用phpize。或者说，phpize的作用可以这样理解：侦测环境(phpize工具是在php安装目录下,基于这点phpize对应了当时的php环境，所以是要根据该php的配置情况生成对应的configure文件)，建立一个configure文件。必须在一个目录下去运行phpize。那么phpize就知道你的的环境是哪个目录，并且configure文件建立在该目下

[root@server4 memcache-2.2.5]# vim /usr/local/lnmp/php/etc/php.ini  ##将memcache模块添加到php中
[root@server4 memcache-2.2.5]# php -m | grep memcache  ##模块添加成功
memcache  
[root@server4 memcache-2.2.5]# telnet localhost 11211  ##测试缓存是否可用
Trying ::1...
Connected to localhost.
Escape character is '^]'.
set name 0 0 4  ##设置key：name； 编号：0； 缓存时间 ：无限 ；valuce值：4
luck  ##输入的value值
STORED
get name  ##通过key值查找对应的value值
VALUE name 0 4
luck
END
set name 0 6 6  ##设置缓存时间为6s，6s后自动删除
westos
STORED
get name  ##6s后查看无value值
END
quit
Connection closed by foreign host.

测试php增加的memcache模块

[root@server4 memcache-2.2.5]# cp memcache.php example.php /usr/local/lnmp/nginx/html/  ##将memcache中用来测试php页面的文件复制到nginx的默认发布目录中
[root@server4 memcache-2.2.5]# cd /usr/local/lnmp/nginx/html
[root@server4 html]# vim memcache.php 
 22 define('ADMIN_USERNAME','memcache');    // Admin Username
 23 define('ADMIN_PASSWORD','westos');      // Admin Password
 24 define('DATE_FORMAT','Y/m/d H:i:s');
 25 define('GRAPH_SIZE',200);
 26 define('MAX_ITEM_DUMP',50);
 27 
 28 $MEMCACHE_SERVERS[] = '172.25.4.114:11211'; // add more as an array
 29 $MEMCACHE_SERVERS[] = 'mymemcache-server2:11211'; // add more as an array
[root@server4 html]# vim index.php
[root@server4 html]# cat index.php
<?php
phpinfo()
?>
[root@server4 html]# nginx
[root@server4 html]# nginx -s reload

浏览器测试监控界面是否可用
http://172.25.4.114/memcache.php


物理机测试：对两个界面均进行压力测试

[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/index.php
Server Software:        nginx/
Server Hostname:        172.25.4.114
Server Port:            80

Document Path:          /index.php
Document Length:        83827 bytes

Concurrency Level:      10
Time taken for tests:   35.863 seconds
Complete requests:      5000
Failed requests:        396
   (Connect: 0, Receive: 0, Length: 396, Exceptions: 0)
Write errors:           0
Total transferred:      419919566 bytes
HTML transferred:       419134566 bytes
Requests per second:    139.42 [#/sec] (mean)
Time per request:       71.726 [ms] (mean)
Time per request:       7.173 [ms] (mean, across all concurrent requests)
Transfer rate:          11434.57 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1  20.0      0    1001
Processing:     9   71   9.3     69     252
Waiting:        2   64   8.0     63     122
Total:         10   72  22.0     69    1071

Percentage of the requests served within a certain time (ms)
  50%     69
  66%     73
  75%     75
  80%     77
  90%     81
  95%     88
  98%     97
  99%    103
 100%   1071 (longest request)

[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/example.php
Server Software:        nginx/
Server Hostname:        172.25.4.114
Server Port:            80

Document Path:          /example.php
Document Length:        0 bytes

Concurrency Level:      10
Time taken for tests:   3.519 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Non-2xx responses:      5000
Total transferred:      880000 bytes
HTML transferred:       0 bytes
Requests per second:    1421.01 [#/sec] (mean)
Time per request:       7.037 [ms] (mean)
Time per request:       0.704 [ms] (mean, across all concurrent requests)
Transfer rate:          244.24 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        0    0   0.2      0       4
Processing:     2    7   0.6      7      10
Waiting:        1    7   0.6      7       9
Total:          2    7   0.6      7      10

Percentage of the requests served within a certain time (ms)
 50%      7
 66%      7
 75%      7
 80%      7
 90%      8
 95%      8
 98%      8
 99%      9
100%     10 (longest request)

由此可知：访问index.php很慢，因为没有缓存,而且failed比较多；访问example很快，命中率也高，因为第一次访问以后有缓存
三.nginx添加memcache模块（缓存前移）
原nginx不支持memcache，需要使用OpenResty
1.工作原理
将memcache模块加到nginx服务器中，当客户有请求时，ngiinx先从memcache中寻找数据，如果有数据则直接反馈客户；如果说没有，则 去后端请求数据，将请求得到的数据一份给客户端，一份给memcache用于缓存，下次有同样请求时，则直接从memcache 中拿取数据给客户端
2.什么是openresty
OpenResty(又称：ngx_openresty) 是一个基于 NGINX 的可伸缩的web平台，由中国人章亦春发起，提供了很多高质量的第三方模块
OpenResty 是一个强大的 Web 应用服务器，Web 开发人员可以使用 Lua 脚本语言调动 Nginx 支持的各种 C 以及Lua 模块,更主要的是在性能方面，OpenResty可以 快速构造出足以胜任 10K 以上并发连接响应的超高性能 Web 应用系统
OpenResty 的目标是让 Web 服务直接跑在 Nginx 服务内部,充分利用 Nginx 的非阻塞 I/O 模型,不仅仅对 HTTP 客户端请求,甚至于对远程后端诸如mysql、PostgreSQL、 ~Redis ~Memcaches 等都进行一致的高性能响应
3.openresty的安装编译

[root@server4 ~]# nginx -s stop  ##关闭之前的nginx
[root@server4 ~]# cd /lnmp
[root@server4 lnmp]# tar zxf openresty-1.13.6.1.tar.gz
[root@server4 lnmp]# cd openresty-1.13.6.1
[root@server4 openresty-1.13.6.1]# ./configure --prefix=/usr/local/openresty
[root@server4 openresty-1.13.6.1]# gmake && gmake install

4.nginx配置文件的修改

root@server4 openresty-1.13.6.1]# cd /usr/local/openresty/
[root@server4 openresty]# ls
bin  COPYRIGHT  luajit  lualib  nginx  pod  resty.index  site
[root@server4 openresty]# cd nginx/
[root@server4 nginx]# cd conf
root@server4 conf]# vim nginx.conf
2 user  nginx nginx;
17 http {
18     include       mime.types;
19     default_type  application/octet-stream;
20     upstream memcache {
21         server localhost:11211;
22         keepalive 512;
23 }
68         location /memc {
69                 internal;  ##只接收内部的访问，不接收外部的http访问，不安全
70                 memc_connect_timeout 100ms;
71                 memc_send_timeout 100ms;  ##后端服务器数据传回时间
72                 memc_read_timeout 100ms;  ##连接成功后，后端服务器响应时间
73                 set $memc_key $query_string;
74                 set $memc_exptime 300;
75                 memc_pass memcache;
76 }
77         location ~ \.php$ {
78             set $key $uri$args;
79             srcache_fetch GET /memc $key;  ##（这两个配置的作用是：请求php页面时，先会去memcache中找，如果没有，正常访问；）
80             srcache_store PUT /memc $key;  ##（访问结束后将结果存到memcache，下次访问时直接从缓存中）
81             root           html;
82             fastcgi_pass   127.0.0.1:9000;
83             fastcgi_index  index.php;
84 #           fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
85             include        fastcgi.conf;
86         }
[root@server4 conf]# /usr/local/openresty/nginx/sbin/nginx

物理机测压：

[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/example.php
Server Software:        openresty/1.13.6.1
Server Hostname:        172.25.4.114
Server Port:            80

Document Path:          /example.php
Document Length:        16 bytes

Concurrency Level:      10
Time taken for tests:   3.304 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Non-2xx responses:      5000
Total transferred:      960000 bytes
HTML transferred:       80000 bytes
Requests per second:    1513.17 [#/sec] (mean)
Time per request:       6.609 [ms] (mean)
Time per request:       0.661 [ms] (mean, across all concurrent requests)
Transfer rate:          283.72 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        0    0   0.4      0       8
Processing:     2    6   0.9      6      16
Waiting:        1    6   0.9      6      16
Total:          3    7   1.0      7      16

Percentage of the requests served within a certain time (ms)
 50%      7
 66%      7
 75%      7
 80%      7
 90%      8
 95%      8
 98%      9
 99%     10
100%     16 (longest request)

[root@foundation4 kiosk]# ab -c 10 -n 5000 http://172.25.4.114/index.php
Server Software:        openresty/1.13.6.1
Server Hostname:        172.25.4.114
Server Port:            80

Document Path:          /index.php
Document Length:        16 bytes

Concurrency Level:      10
Time taken for tests:   3.213 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Non-2xx responses:      5000
Total transferred:      960000 bytes
HTML transferred:       80000 bytes
Requests per second:    1556.29 [#/sec] (mean)
Time per request:       6.426 [ms] (mean)
Time per request:       0.643 [ms] (mean, across all concurrent requests)
Transfer rate:          291.80 [Kbytes/sec] received

Connection Times (ms)
             min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       4
Processing:     2    6   0.9      6      10
Waiting:        1    6   0.9      6      10
Total:          2    6   0.9      6      11

Percentage of the requests served within a certain time (ms)
 50%      6
 66%      7
 75%      7
 80%      7
 90%      7
 95%      8
 98%      8
 99%      9
100%     11 (longest request)

相比于php增加memcache模块，此时利用缓存前移缓存速度更快