- 目录
- 1.NoSQL介绍
- 2.Memached介绍
- 3.装memcached安装
- 4.查看Memcached运行状态
- 5.memcached命令行
- 6.memcached数据导出和导入
- 7.php连接memcached
- 8.memcached中存储session
1.NoSQL介绍
非关系型数据库就是NoSQL，关系型数据库代表MySQL、SQLserver
对于关系型数据库来说，是需要把数据存储到库、表、行、字段里，查询的时候根据条件一行一行地去匹配，当量非常大的时候就很耗费时间和资源，尤其是数据是需要从磁盘里去检索
NoSQL数据库存储原理非常简单（典型的数据类型为k-v），不存在繁杂的关系链，比如mysql查询的时候，需要找到对应的库、表（通常是多个表）以及字段。
NoSQL数据可以存储在内存里，查询速度非常快
NoSQL在性能表现上虽然能优于关系型数据库，但是它并不能完全替代关系型数据库
NoSQL因为没有复杂的数据结构，扩展非常容易，支持分布式
常见NoSQL数据库
k-v形式的：memcached、redis 适合储存用户信息，比如会话、配置文件、参数、购物车等等。这些信息一般都和ID（键）挂钩，这种情景下键值数据库是个很好的选择。
文档数据库：mongodb 将数据以文档的形式储存。每个文档都是一系列数据项的集合。每个数据项都有一个名称与对应的值，值既可以是简单的数据类型，如字符串、数字和日期等；也可以是复杂的类型，如有序列表和关联对象。数据存储的最小单位是文档，同一个表中存储的文档属性可以是不同的，数据可以使用XML、JSON或者JSONB等多种形式存储。
列存储 Hbase
图 Neo4J、Infinite Graph、OrientDB
2.Memached介绍
Memcached是国外社区网站LiveJournal团队开发，目的是为了通过缓存数据库查询结果，减少数据库访问次数，从而提高动态web站点性能。
官方站点 http://www.memcached.org/
数据结构简单(k-v)，数据存放在内存里（重启服务或者重庆服务器数据就会消失，需要不定期的做数据落地） ；多线程 ；基于c/s架构，协议简单 ；基于libevent的事件处理 ；自主内存存储处理（slab allowcation) ；数据过期方式：Lazy Expiration 和 LRU
Memcached的数据流向
Slab allocation
Slab Allocation的原理
将分配的内存分割成各种尺寸的块（chunk）， 并把尺寸相同的块分成组（chunk的集合），每个chunk集合被称为slab。
Memcached的内存分配以Page为单位，Page默认值为1M，可以在启动时通过-I参数来指定。
Slab是由多个Page组成的，Page按照指定大小切割成多个chunk。
Growth factor
Memcached在启动时通过-f选项可以指定 Growth Factor因子。该值控制chunk大小的差异。默认值为1.25。
通过memcached-tool命令查看指定Memcached实例的不同slab状态，可以看到各Item所占大小（chunk大小）差距为1.25
命令：# memcached-tool 127.0.0.1:11211 display
3.装memcached安装
只需要yum安装 ，启动服务
[root@chenshi ~]# yum install -y memcached libmemcached libevent
已安装:
libevent.x86_64 0:2.0.21-4.el7 libmemcached.x86_64 0:1.0.16-5.el7 memcached.x86_64 0:1.4.15-10.el7_3.1
[root@chenshi ~]# systemctl start memcached.service
[root@chenshi ~]# ps aux|grep memcached
memcach+ 11154 0.0 0.0 344080 1664 ? Ssl 13:27 0:00 /usr/bin/memcached -u memcached -p 11211 -m 64 -c 1024
root 11175 0.0 0.0 112720 972 pts/0 S+ 13:28 0:00 grep --color=auto memcached
自定义端口和用户
[root@chenshi ~]# vi /etc/sysconfig/memcached
PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS=""
[root@chenshi ~]# memcached -h
memcached 1.4.15
-p <num> TCP port number to listen on (default: 11211)
-U <num> UDP port number to listen on (default: 11211, 0 is off)
-s <file> UNIX socket path to listen on (disables network support)
-a <mask> access mask for UNIX socket, in octal (default: 0700)
-l <addr> interface to listen on (default: INADDR_ANY, all addresses)
<addr> may be specified as host:port. If you don't specify
a port number, the value you specified with -p or -U is
used. You may specify multiple addresses separated by comma
or by using -l multiple times
-d run as a daemon
-r maximize core file limit
-u <username> assume identity of <username> (only when run as root)
-m <num> max memory to use for items in megabytes (default: 64 MB)
-M return error on memory exhausted (rather than removing items)
-c <num> max simultaneous connections (default: 1024)
-k lock down all paged memory. Note that there is a
limit on how much memory you may lock. Trying to
allocate more than that would fail, so be sure you
set the limit correctly for the user you started
the daemon with (not for -u <username> user;
under sh this is done with 'ulimit -S -l NUM_KB').
-v verbose (print errors/warnings while in event loop)
-vv very verbose (also print client commands/reponses)
-vvv extremely verbose (also print internal state transitions)
-h print this help and exit
-i print memcached and libevent license
-P <file> save PID in <file>, only used with -d option
-f <factor> chunk size growth factor (default: 1.25)
-n <bytes> minimum space allocated for key+value+flags (default: 48)
-L Try to use large memory pages (if available). Increasing
the memory page size could reduce the number of TLB misses
and improve the performance. In order to get large pages
from the OS, memcached will allocate the total item-cache
in one large chunk.
-D <char> Use <char> as the delimiter between key prefixes and IDs.
This is used for per-prefix stats reporting. The default is
":" (colon). If this option is specified, stats collection
is turned on automatically; if not, then it may be turned on
by sending the "stats detail on" command to the server.
-t <num> number of threads to use (default: 4)
-R Maximum number of requests per event, limits the number of
requests process for a given connection to prevent
starvation (default: 20)
-C Disable use of CAS
-b <num> Set the backlog queue limit (default: 1024)
-B Binding protocol - one of ascii, binary, or auto (default)
-I Override the size of each slab page. Adjusts max item size
(default: 1mb, min: 1k, max: 128m)
-S Turn on Sasl authentication
-o Comma separated list of extended or experimental options
- (EXPERIMENTAL) maxconns_fast: immediately close new
connections if over maxconns limit
- hashpower: An integer multiplier for how large the hash
table should be. Can be grown at runtime if not big enough.
Set this based on "STAT hash_power_level" before a
restart.
其中-m指定memcached分配内存
-c指定最大并发数
-u指定运行memcached服务的用户
4.查看Memcached运行状态
需要关注get_hits 、curr_items（命中的项目）
[root@chenshi ~]# memcached-tool 127.0.0.1:11211 stats
#127.0.0.1:11211 Field Value
accepting_conns 1
auth_cmds 0
auth_errors 0
bytes 0
bytes_read 7
bytes_written 0
cas_badval 0
cas_hits 0
cas_misses 0
cmd_flush 0
cmd_get 0
cmd_set 0
cmd_touch 0
conn_yields 0
connection_structures 11
curr_connections 10
curr_items 0
decr_hits 0
decr_misses 0
delete_hits 0
delete_misses 0
evicted_unfetched 0
evictions 0
expired_unfetched 0
get_hits 0
get_misses 0
hash_bytes 524288
hash_is_expanding 0
hash_power_level 16
incr_hits 0
incr_misses 0
libevent 2.0.21-stable
limit_maxbytes 67108864
listen_disabled_num 0
pid 11154
pointer_size 64
reclaimed 0
reserved_fds 20
rusage_system 0.009883
rusage_user 0.005929
threads 4
time 1534743302
total_connections 11
total_items 0
touch_hits 0
touch_misses 0
uptime 430
version 1.4.15
使用nc查看memcached状态
安装
nmap-ncat.x86_64 2:6.40-13.el[root@chenshi ~]# echo stats |nc 127.0.0.1 11211
STAT pid 11154
STAT uptime 2250
STAT time 1534745122
STAT version 1.4.15
STAT libevent 2.0.21-stable
STAT pointer_size 64
STAT rusage_user 0.021608
STAT rusage_system 0.038414
STAT curr_connections 10
STAT total_connections 12
STAT connection_structures 11
STAT reserved_fds 20
STAT cmd_get 0
STAT cmd_set 0
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 13
STAT bytes_written 1025
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT bytes 0
STAT curr_items 0
STAT total_items 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END
memstat查看服务
[root@chenshi ~]# memstat --servers=127.0.0.1:11211
Server: 127.0.0.1 (11211)
pid: 11154
uptime: 2340
time: 1534745212
version: 1.4.15
libevent: 2.0.21-stable
pointer_size: 64
rusage_user: 0.021700
rusage_system: 0.040989
curr_connections: 10
total_connections: 13
connection_structures: 11
reserved_fds: 20
cmd_get: 0
cmd_set: 0
cmd_flush: 0
cmd_touch: 0
get_hits: 0
get_misses: 0
delete_misses: 0
delete_hits: 0
incr_misses: 0
incr_hits: 0
decr_misses: 0
decr_hits: 0
cas_misses: 0
cas_hits: 0
cas_badval: 0
touch_hits: 0
touch_misses: 0
auth_cmds: 0
auth_errors: 0
bytes_read: 30
bytes_written: 2071
limit_maxbytes: 67108864
accepting_conns: 1
listen_disabled_num: 0
threads: 4
conn_yields: 0
hash_power_level: 16
hash_bytes: 524288
hash_is_expanding: 0
bytes: 0
curr_items: 0
total_items: 0
expired_unfetched: 0
evicted_unfetched: 0
evictions: 0
reclaimed: 0
5.memcached命令行
安装telnet包
[root@chenshi ~]# telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
set key2 0 30 2
12
STORED
quit退出
格式
<command name> <key> <flags> <exptime> <bytes>\r\n <data block>\r\n
注：\r\n在windows下是Enter键
<command name> 可以是set, add, replace
set表示按照相应的<key>存储该数据，没有的时候增加，有的时候覆盖
add表示按照相应的<key>添加该数据,但是如果该<key>已经存在则会操作失败
replace表示按照相应的<key>替换数据,但是如果该<key>不存在则操作失败。
<key> 客户端需要保存数据的key
<flags> 是一个16位的无符号的整数(以十进制的方式表示)。
该标志将和需要存储的数据一起存储,并在客户端get数据时返回。
客户端可以将此标志用做特殊用途，此标志对服务器来说是不透明的。
<exptime> 为过期的时间。
若为0表示存储的数据永远不过期(但可被服务器算法：LRU 等替换)。
如果非0(unix时间或者距离此时的秒数),当过期后,服务器可以保证用户得不到该数据(以服务器时间为标准)。
<bytes> 需要存储的字节数，当用户希望存储空数据时<bytes>可以为0
<data block>需要存储的内容，输入完成后，最后客户端需要加上\r\n（直接点击Enter）作为结束标志。
6.memcached数据导出和导入
导出
[root@chenshi ~]# memcached-tool 127.0.0.1:11211 dump > data.txt
导入，使用nc命令
[root@chenshi ~]# nc 127.0.0.1 11211 < data.txt
7.php连接memcached
安装php的memcached扩展
[root@chenshi src]# wget http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz
http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz: 地址缺少协议类型.
[root@chenshi src]# wget http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz
--2018-08-20 15:17:10-- http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz
正在解析主机 www.apelearn.com (www.apelearn.com)... 47.104.7.242
正在连接 www.apelearn.com (www.apelearn.com)|47.104.7.242|:80... 已连接。
已发出 HTTP 请求，正在等待回应... 200 OK
长度：27366 (27K) [application/octet-stream]
正在保存至: “memcache-2.2.3.tgz”
100%[============================================================>] 27,366 20.4KB/s 用时 1.3s
2018-08-20 15:17:11 (20.4 KB/s) - 已保存 “memcache-2.2.3.tgz” [27366/27366])
[root@chenshi src]# ls
apache-tomcat-8.5.32.tar.gz jdk8-downloads-2133151.html nginx-1.8.0.tar.gz
apr-1.6.3 jdk-8u171-linux-x64.tar.gz php-5.6.32
apr-1.6.3.tar.gz mariadb-10.2.6-linux-glibc_214-x86_64.tar.gz php-5.6.32.tar.bz2
apr-util-1.6.1 memcache-2.2.3.tgz php-7.1.6
apr-util-1.6.1.tar.bz2 mysql-5.6.36-linux-glibc2.5-x86_64.tar.gz php-7.1.6.tar.bz2
httpd-2.4.33 mysql-5.6.36.tar.gz phpredis-develop
httpd-2.4.33.tar.gz nginx-1.8.0 phpredis-develop.zi解压安装
[root@chenshi src]# tar zxvf memcache-2.2.3.tgz
package.xml
memcache-2.2.3/config.m4
memcache-2.2.3/config9.m4
memcache-2.2.3/config.w32
memcache-2.2.3/CREDITS
memcache-2.2.3/example.php
memcache-2.2.3/memcache.c
memcache-2.2.3/memcache_queue.c
memcache-2.2.3/memcache_session.c
memcache-2.2.3/memcache_standard_hash.c
memcache-2.2.3/memcache_consistent_hash.c
memcache-2.2.3/memcache.dsp
memcache-2.2.3/php_memcache.h
memcache-2.2.3/memcache_queue.h
memcache-2.2.3/README
[root@chenshi src]# cd memcache-2.2.3/
[root@chenshi memcache-2.2.3]#
[root@chenshi memcache-2.2.3]# /usr/local/php-fpm/bin/phpize
Configuring for:
PHP Api Version: 20131106
Zend Module Api No: 20131226
Zend Extension Api No: 220131226
您在 /var/spool/mail/root 中有新邮件
[root@chenshi memcache-2.2.3]# ./configure --with-php-config=/usr/local/php-fpm/bin/php-config
make && make install
Installing shared extensions: /usr/local/php-fpm/lib/php/extensions/no-debug-non-zts-20131226/
编辑php配置文件，增加一行，检测是否成功加载
[root@chenshi memcache-2.2.3]# vi /usr/local/php-fpm/etc/php.ini
extension=memcache.so
您在 /var/spool/mail/root 中有新邮件
[root@chenshi memcache-2.2.3]# /usr/local/php-fpm/sbin/php-fpm -m
[PHP Modules]
cgi-fcgi
Core
ctype
curl
date
dom
ereg
exif
fileinfo
filter
ftp
gd
hash
iconv
json
libxml
mbstring
mcrypt
memcache
mysql
mysqli
openssl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
Reflection
session
SimpleXML
soap
SPL
sqlite3
standard
tokenizer
xml
xmlreader
xmlwriter
zlib
下载测试脚本
[root@chenshi ~]# curl www.apelearn.com/study_v2/.memcache.txt > 1.php 2>/dev/null
实验成功
[root@chenshi ~]# /usr/local/php-fpm/bin/php 1.php
Get key1 value: This is first value<br>Get key1 value: This is replace value<br>Get key2 value: Array
(
[0] => aaa
[1] => bbb
[2] => ccc
[3] => ddd
)
<br>Get key1 value: <br>Get key2 value
8.memcached中存储session
在lamp/lnmp环境下实现
下载测试脚本，每次执行测试脚本在tmp下都会生成一个sess开头的随机字符文件
[root@chenshi ~]# wget http://study.lishiming.net/.mem_se.txt
[root@chenshi default]# curl localhost/1.php
1534902467<br><br>1534902467<br><br>0ucudqp5mmlg7k63mb6dd5oou5
[root@chenshi default]# ls /tmp/
chen.sock test.log.log-20180814
mysql.sock test.log.log-20180815
php-fcgi.sock test.log.log-20180816
sess_0ucudqp5mmlg7k63mb6dd5oou5 test.log.log-20180817
systemd-private-944ff3d9ac0e44db867d43c34c49b6a2-chronyd.service-q2Ci44 test.log.log-20180818
test.log.log test.log.log-20180819
test.log.log-20180811 test.log.log-20180820
test.log.log-20180812 test.log.log-20180821
test.log.log-20180813
需要把这种文件放到memcached中
编辑php.ini文件，注释掉
;session.save_handler = files
增加
session.save_handler = memcache
session.save_path = "tcp://192.168.1.197:11211"
重启php再次访问 tmp下没有sess文件
[root@chenshi default]# !curl
curl localhost/1.php
1534902846<br><br>1534902846<br><br>bnk199hp3v0bm7s97og8md8v83
[root@chenshi default]# ls /tmp/
chen.sock test.log.log-20180814
mysql.sock test.log.log-20180815
php-fcgi.sock test.log.log-20180816
systemd-private-944ff3d9ac0e44db867d43c34c49b6a2-chronyd.service-q2Ci44 test.log.log-20180817
test.log.log test.log.log-20180818
test.log.log-20180811 test.log.log-20180819
test.log.log-20180812 test.log.log-20180820
test.log.log-20180813 test.log.log-20180821
再次curl查看memcached的导出文件
[root@chenshi default]# memcached-tool 127.0.0.1:11211 dump > data.txt
[root@chenshi default]# cat data.txt
add v01s4u0c26fuakehmtpd47hf25 0 1534904398 37
TEST|i:1534902959;TEST3|i:1534902959;
add tnit882umslssgh3p7ctpjssm0 0 1534904398 37
TEST|i:1534902958;TEST3|i:1534902958;
add rf55q291m35s2mk4q88c9ed911 0 1534904397 37
TEST|i:1534902957;TEST3|i:1534902957;
add bnk199hp3v0bm7s97og8md8v83 0 1534904286 37
TEST|i:1534902846;TEST3|i:1534902846;
扫一扫
193
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
