redis

最新推荐文章于 2024-04-04 11:33:29 发布

yangkaiorange

最新推荐文章于 2024-04-04 11:33:29 发布

阅读量344

点赞数

本文链接：https://blog.csdn.net/yangkaiorange/article/details/83035830

版权

一 redis的介绍

redis 是一个高性能的 key-value 数据库。 redis 的出现,很大程度补偿了
memcached 这类 keyvalue 存储的不足,在部分场合可以对关系数据库起到很
好的补充作用。它提供了 Python,Ruby,Erlang,PHP 客户端,使用很方便。
Redis 的所有数据都是保存在内存中,然后不定期的通过异步方式保存到磁盘上
(这称为“半持久化模式”);也可以把每一次数据变化都写入到一个 append
only file(aof)里面(这称为“全持久化模式”)。

二实验步骤

1 实验环境

1 ）源码安装（server2）

tar zxf redis-4.0.1.tar.gz

cd redis-4.0.1

make && make install

2 ) 运行脚本

[root@server2 utils]# pwd
/root/redis-4.0.1/utils
[root@server2 utils]# ./install_server.sh

3 )配置文件

vim /etc/redis/6379.conf

bind 0.0.0.0 监听所有端口

/etc/init.d/redis_6379 restart

4 ) 查看可执行文件

redis-server: Redis 服务主程序。
redis-cli: Redis 客户端命令行工具,也可以用 telnet 来操作。redis-benchmark: Redis 性能测试工具,用于测试读写性能。
redis-check-aof:检查 redis aof 文件完整性,aof 文件持久化记录服务器执行的所有写
操作命令,用于还原数据。
redis-check-dump:检查 redis rdb 文件的完整性,rdb 就是快照存储, 即按照一定的
策略周期性的将数据保存到磁盘,是默认的持久化方式。
redis-sentinel:
redis-sentinel 是集群管理工具,主要负责主从切换。

简单string数据类型的展示

root@server2 bin]# redis-cli 
127.0.0.1:6379> set name james
OK
127.0.0.1:6379> get name
"james"
127.0.0.1:6379>

2 建立redis主从

1 ）和上面一样在server3上安装redis作为从redis

2 ）vim /etc/redis/6379.conf

slaveof 172.25.1.2 6379 ##指定master redis和其端口号

3 ）/etc/init.d/redis_6379 restart

4 ）查看server2 上建立的key

[root@server3 redis]# redis-cli
127.0.0.1:6379> get name
"james"
127.0.0.1:6379>

5 )再开一台从redis server4 做上述的配置

[root@server4 redis]# redis-cli
127.0.0.1:6379> get name
"james"
127.0.0.1:6379>

表明主从没有问题

3 了解一下redis不定期刷新内存到硬盘策略我们可以看配置文件

[root@server2 bin]# cd /var/lib/redis/
[root@server2 redis]# ls
6379
[root@server2 redis]# cd 6379/
[root@server2 6379]# ls
dump.rdb       #快照存储，依照配置文件给的策略定期写入到硬盘中

半持久化vim /etc/redis/6379.conf

save 900 1  # after 900 sec (15 min) if at least 1 key changed
save 300 10 #   after 300 sec (5 min) if at least 10 keys changed
save 60 10000 #   after 60 sec if at least 10000 keys changed

全持久话 vim /etc/redis/6379.conf

下列几种模式:
always:每次都调用,比如安全,但速度最慢;
everysec:每秒同步,这也是默认方式;
no:不调用 fsync,由操作系统决定何时同步,比如快的模式;
no-appendfsync-on-rewrite:默认值 no。当 AOF fsync 策略设置为 always
或 everysec,后台保存进程会执行大量的 I/O 操作。某些 linux 配置下 redis 可
能会阻塞过多的 fsync()调用。auto-aof-rewrite-percentage:默认值 100
auto-aof-rewrite-min-size:默认值 64mb
appendfsync everysec
hash-max-zipmap-entries:默认值 512,当某个 map 的元素个数达到最大值,
但是其中最大元素的长度没有达到设定阀值时,其 HASH 的编码采用一种特殊的
方式(更有效利用内存)。本参数与下面的参数组合使用来设置这两项阀值。设置
元素个数;

3 redis-sentinel 集群管理工具

1 ) 文件配置

cd /root/redis-4.0.1

cp sentinel.conf /etc/redis/

vim /etc/redis/sentinel.conf

protected-mode no
sentinel monitor mymaster 172.25.1.2 6379 2   master失效起码要有两个sentinel同意
sentinel down-after-milliseconds mymaster 10000  sentinel 认为已经短线所需要的毫秒数

server1 下

cd /etc/redis/

redis-sentinel /etc/redis/sentinel.conf

server2 :

cd /etc/redis/

redis-sentinel /etc/redis/sentinel.conf

server3:

cd /etc/redis/

redis-sentinel /etc/redis/sentinel.conf

2 )高可用测试

开启sentinel集群之后查看其端口

进入到管理器查看所有的状态

[root@server2 ~]# redis-cli -p 26379
127.0.0.1:26379> INFO
# Server
redis_version:4.0.1
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:7c7df7b8dd295323
redis_mode:sentinel
os:Linux 2.6.32-431.el6.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:sync-builtin
gcc_version:4.4.7
process_id:4512
run_id:31c390f00c576fa2c969f817ac63a54c1feba80d
tcp_port:26379
uptime_in_seconds:564
uptime_in_days:0
hz:17
lru_clock:12678773
executable:/etc/redis/redis-sentinel
config_file:/etc/redis/sentinel.conf

# Clients
connected_clients:3
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0

# CPU
used_cpu_sys:0.68
used_cpu_user:0.49
used_cpu_sys_children:0.00
used_cpu_user_children:0.00

# Stats
total_connections_received:3
total_commands_processed:1545
instantaneous_ops_per_sec:2
total_net_input_bytes:84304
total_net_output_bytes:9292
instantaneous_input_kbps:0.13
instantaneous_output_kbps:0.02
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0

# Sentinel
sentinel_masters:1
sentinel_tilt:0
sentinel_running_scripts:0
sentinel_scripts_queue_length:0
sentinel_simulate_failure_flags:0
master0:name=mymaster,status=ok,address=172.25.1.2:6379,slaves=2,sentinels=3
127.0.0.1:26379>

测试：

把master（server2）done掉

[root@server2 ~]# redis-cli
127.0.0.1:6379> SHUTDOWN
not connected>
[root@server2 ~]# redis-cli
Could not connect to Redis at 127.0.0.1:6379: Connection refused
Could not connect to Redis at 127.0.0.1:6379: Connection refused

[root@server2 ~]# redis-cli -p 26379

可以看出在sentinel集群中master从server2 变成了server3

3 ）集群管理

1 在刚才实验中已经把master从server2变成了server3

这个时候我们就需要把server2 中的配置文件，修改为slave

2 vim /etc/redis/6379.conf

slaveof 172.25.1.3 6379 #指向server3为master

appendonly yes

3 测试：

cd /usr/local/

mkdir clustermkdir cluster

cd cluster/

mkdir 700{1..6}

vim redis.conf

port 7001
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
appendonly yes
daemonize yes  #打入后台

redis-server redis.conf

剩下五个结点做同样的操作

注意：port端口依次增加1

cd /root/redis-4.0.1/src

cp redis-trib.rb /usr/local/bin/

4 软件依赖性安装 (需要在网上下载)

yum install rubygems-1.3.7-5.el6.noarch.rpm -y

yum install ruby-2.2.3-1.el6.x86_64.rpm libyaml-0.1.3-4.el6_6.x86_64.rpm -y

5 gem 安装

gem install --local redis-4.0.1.gem

6 集群组的建立

redis-trib.rb create --replicas 1 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 127.0.0.1:7006

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:7001
127.0.0.1:7002
127.0.0.1:7003
Adding replica 127.0.0.1:7004 to 127.0.0.1:7001
Adding replica 127.0.0.1:7005 to 127.0.0.1:7002
Adding replica 127.0.0.1:7006 to 127.0.0.1:7003
M: c08f9e7e0309e123da5083258ead06ebbf30ee99 127.0.0.1:7001
   slots:0-5460 (5461 slots) master
M: 511255730464cce14552b58cdd00fbbd5d8870cb 127.0.0.1:7002
   slots:5461-10922 (5462 slots) master
M: 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671 127.0.0.1:7003
   slots:10923-16383 (5461 slots) master
S: 6fb9d3dbb9f1f9e635cc55151d37773e9ca75630 127.0.0.1:7004
   replicates c08f9e7e0309e123da5083258ead06ebbf30ee99
S: 535302725de00881b912a5b0c24ba35b85399714 127.0.0.1:7005
   replicates 511255730464cce14552b58cdd00fbbd5d8870cb
S: 85657b47c4dc84e448a3f742dbe860729755ae55 127.0.0.1:7006
   replicates 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: c08f9e7e0309e123da5083258ead06ebbf30ee99 127.0.0.1:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 85657b47c4dc84e448a3f742dbe860729755ae55 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671
S: 535302725de00881b912a5b0c24ba35b85399714 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 511255730464cce14552b58cdd00fbbd5d8870cb
S: 6fb9d3dbb9f1f9e635cc55151d37773e9ca75630 127.0.0.1:7004
   slots: (0 slots) slave
   replicates c08f9e7e0309e123da5083258ead06ebbf30ee99
M: 511255730464cce14552b58cdd00fbbd5d8870cb 127.0.0.1:7002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671 127.0.0.1:7003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

redis-trib.rb check 127.0.0.1:7001

>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: c08f9e7e0309e123da5083258ead06ebbf30ee99 127.0.0.1:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 85657b47c4dc84e448a3f742dbe860729755ae55 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671
S: 535302725de00881b912a5b0c24ba35b85399714 127.0.0.1:7005
   slots: (0 slots) slave
   replicates 511255730464cce14552b58cdd00fbbd5d8870cb
S: 6fb9d3dbb9f1f9e635cc55151d37773e9ca75630 127.0.0.1:7004
   slots: (0 slots) slave
   replicates c08f9e7e0309e123da5083258ead06ebbf30ee99
M: 511255730464cce14552b58cdd00fbbd5d8870cb 127.0.0.1:7002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
M: 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671 127.0.0.1:7003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

[root@server2 ~]# redis-cli -c -p 7001
127.0.0.1:7001> set name skr
-> Redirected to slot [5798] located at 127.0.0.1:7002
OK
127.0.0.1:7002> get name
"skr"
127.0.0.1:7002>

测试：将7002 done掉来测试高可用

[root@server2 ~]# redis-cli -p 7002
127.0.0.1:7002> SHUTDOWN
not connected>

[root@server2 ~]# redis-trib.rb check 127.0.0.1:7001
>>> Performing Cluster Check (using node 127.0.0.1:7001)
M: c08f9e7e0309e123da5083258ead06ebbf30ee99 127.0.0.1:7001
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
S: 85657b47c4dc84e448a3f742dbe860729755ae55 127.0.0.1:7006
   slots: (0 slots) slave
   replicates 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671
M: 535302725de00881b912a5b0c24ba35b85399714 127.0.0.1:7005
   slots:5461-10922 (5462 slots) master
   0 additional replica(s)
S: 6fb9d3dbb9f1f9e635cc55151d37773e9ca75630 127.0.0.1:7004
   slots: (0 slots) slave
   replicates c08f9e7e0309e123da5083258ead06ebbf30ee99
M: 9a25a19b26d5b07e48e7f06ffb4f6c69825f6671 127.0.0.1:7003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

结合上图对比127.0.0.1:7005替换了127.0.0.1:7002成了master

4 redis作为msyql的缓冲器

server3 搭建个简易的lnmp架构

1 ）php-fpm

yum install -y php-5.3.3-38.el6.x86_64.rpm           php-gd-5.3.3-38.el6.x86_64.rpm
php-cli-5.3.3-38.el6.x86_64.rpm       php-mbstring-5.3.3-38.el6.x86_64.rpm
php-common-5.3.3-38.el6.x86_64.rpm    php-mysql-5.3.3-38.el6.x86_64.rpm
php-devel-5.3.3-38.el6.x86_64.rpm     php-pdo-5.3.3-38.el6.x86_64.rpm
php-fpm-5.3.3-38.el6.x86_64.rpm

2 ) 安装php的redis扩展

https://github.com/owlient/phpredis

unzip phpredis-master.zip
cd phpredis-master
phpize
./configure 
make && make install

vim /etc/php.ini

date.timezone = Asia/Shanghai
#设置时区

cd /etc/php.d/

cp mysql.ini redis.ini

vim redis.ini

[root@server3 php.d]# php -m | grep redis
redis

cd /etc/php-fpm.d

编辑www.cnf 将user & group 的改成nginx

service php-fpm start

3 ）简单配置nginx

1 yum install -y nginx-1.8.0-1.el6.ngx.x86_64.rpm（安装软件）

2 vim /etc/nginx/conf.d/default.conf

server {
listen
80;
server_name localhost;
#修改 php-fpm 用户location / {
root
/usr/share/nginx/html;
index index.php index.html index.htm;
}
location ~ \.php$ {
root
/usr/share/nginx/html;
fastcgi_pass
127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME
/usr/share/nginx/html/$fastcgi_script_name;
include
fastcgi_params;
}
}

/etc/init.d/nginx restart

server5 上安装mysql

1）yum install -y mysql-server
2 ）

mysql> grant all on test.* to redis@'%' identified by 'james'
    -> ;
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

server3 创建php页面

cd /usr/share/nginx/html/

vim test.php

<?php
        $redis = new Redis();
        $redis->connect('172.25.1.4',6379) or die ("could net connect redis server");
  #      $query = "select * from test limit 9";
        $query = "select * from test";
        for ($key = 1; $key < 10; $key++)
        {
                if (!$redis->get($key))
                {
                        $connect = mysql_connect('172.25.1.5','redis','james');
                        mysql_select_db(test);
                        $result = mysql_query($query);
                        //如果没有找到$key,就将该查询sql的结果缓存到redis
                        while ($row = mysql_fetch_assoc($result))
                        {
                                $redis->set($row['id'],$row['name']);
                        }
                        $myserver = 'mysql';
                        break;
                }
                else
                {
                        $myserver = "redis";
                        $data[$key] = $redis->get($key);
                }
        }

        echo $myserver;
        echo "<br>";
        for ($key = 1; $key < 10; $key++)
{
                echo "number is <b><font color=#FF0000>$key</font></b>";

                echo "<br>";

                echo "name is <b><font color=#FF0000>$data[$key]</font></b>";

                echo "<br>";
        }
?>

server4 上为redis（master）

测试：

这个时候我们可以往server5上导入test.sql来测试

mysql -h 172.25.1.5 -u redis -p < test.sql

[root@server3 redis]# cat test.sql 
use test;
CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO `test` VALUES (1,'test1'),(2,'test2'),(3,'test3'),(4,'test4'),(5,'test5'),(6,'test6'),(7,'test7'),(8,'test8'),(9,'test9');

#DELIMITER $$
#CREATE TRIGGER datatoredis AFTER UPDATE ON test FOR EACH ROW BEGIN
#    SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`)); 
#  END$$
#DELIMITER ;

这个时候我们在serve5 上更新数据库

去访问测试页，会发现没有什么变化。这也就是redis缓存和mysql数据一致性问题怎么解决？

5 配置gearman实现数据同步

Gearman 是一个支持分布式的任务分发框架:
Gearman Job Server: Gearman 核心程序,需要编译安装并以守护进程形式运行在后台。
Gearman Client:可以理解为任务的请求者。
Gearman Worker:任务的真正执行者,一般需要自己编写具体逻辑并通过守护进程方式
运行,Gearman Worker 接收到 Gearman Client 传递的任务内容后,会按顺序处理。
大致流程:下面要编写的 mysql 触发器,就相当于 Gearman 的客户端。修改表,插入表就相当于直接
下发任务。然后通过 lib_mysqludf_json UDF 库函数将关系数据映射为 JSON 格式,然后
在通过 gearman-mysql-udf 插件将任务加入到 Gearman 的任务队列中,最后通过
redis_worker.php,也就是 Gearman 的 worker 端来完成 redis 数据库的更新。

1 ) 安装gearman软件包(server3)

gearmand libgearman-devel libgearman libevent libevent-devel
libevent-doc libevent-headers tokyocabinet

2 ）启动软件

/etc/init.d/gearmand start

安装 lib_mysqludf_json和gearman-mysql-udf （server5）

lib_mysqludf_json UDF 库函数将关系数据映射为 JSON 格式。通常,数据库中的数据映
射为 JSON 格式,是通过程序来转换的。
https://github.com/mysqludf/lib_mysqludf_json

gearman-mysql-udf这个插件是用来管理调用 Gearman 的分布式的队列。
https://launchpad.net/gearman-mysql-udf

1 ）安装 lib_mysqludf_json

unzip lib_mysqludf_json-master.zip

cd lib_mysqludf_json-master

gcc $(mysql_config --cflags) -shared -fPIC -o lib_mysqludf_json.so lib_mysqludf_json.c

注意如果出现报错安装

yum install mysql-devel -y

2 )查看mysql模块目录

3 ）拷贝 lib_mysqludf_json.so 模块:

cp lib_mysqludf_json.so /usr/lib64/mysql/plugin

4 )注册 UDF 函数

安装 gearman-mysql-udf

1 )tar zxf gearman-mysql-udf-0.6.tar.gz

cd gearman-mysql-udf-0.6/

./configure --with-mysql --libdir=/usr/lib64/mysql/plugin/

注意如果出现以下错误

解决办法

make && make install

2 )注册 UDF 函数

cd /usr/lib64/mysql/plugin/

[root@server5 plugin]# ls
ha_archive.so          ha_example.so.0        ha_innodb_plugin.so.0.0.0
ha_archive.so.0        ha_example.so.0.0.0    libgearman_mysql_udf.la
ha_archive.so.0.0.0    ha_federated.so        libgearman_mysql_udf.so
ha_blackhole.so        ha_federated.so.0      libgearman_mysql_udf.so.0
ha_blackhole.so.0      ha_federated.so.0.0.0  libgearman_mysql_udf.so.0.0.0
ha_blackhole.so.0.0.0  ha_innodb_plugin.so    lib_mysqludf_json.so
ha_example.so          ha_innodb_plugin.so.0

mysql> CREATE FUNCTION gman_do_background RETURNS STRING SONAME 'libgearman_mysql_udf.so';
Query OK, 0 rows affected (0.00 sec)

mysql> CREATE FUNCTION gman_servers_set RETURNS STRING SONAME 'libgearman_mysql_udf.so';
Query OK, 0 rows affected (0.00 sec)

mysql>

查看函数

mysql> select * from mysql.func;
+--------------------+-----+-------------------------+----------+
| name               | ret | dl                      | type     |
+--------------------+-----+-------------------------+----------+
| json_object        |   0 | lib_mysqludf_json.so    | function |
| gman_do_background |   0 | libgearman_mysql_udf.so | function |
| gman_servers_set   |   0 | libgearman_mysql_udf.so | function |
+--------------------+-----+-------------------------+----------+
3 rows in set (0.00 sec)

mysql> SELECT gman_servers_set('172.25.1.3:4730');
+-------------------------------------+
| gman_servers_set('172.25.1.3:4730') |
+-------------------------------------+
| 172.25.1.3:4730                     |
+-------------------------------------+
1 row in set (0.00 sec)

mysql>

3 ）编写mysql的触发器

vim test.sql

use test;
#CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB DEFAULT CHARSET=utf8;
#INSERT INTO `test` VALUES (1,'test1'),(2,'test2'),(3,'test3'),(4,'test4'),(5,'test5'),(6,'test6'),(7,'test7'),(8,'test8'),(9,'test9');

DELIMITER $$
CREATE TRIGGER datatoredis AFTER UPDATE ON test FOR EACH ROW BEGIN
    SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`));
  END$$
DELIMITER ;

mysql < test.sql

4 )编写 gearman 的 worker 端(server3)

1软件的安装

tar zxf gearman-1.1.2.tgz

cd gearman-1.1.2

 phpize

./configure

make && make install

2 增加php支持的gearman插件

cd /etc/php.d/
cp redis.ini gearman.ini
vim gearman.ini

3 编辑vim worker.php

<?php
$worker = new GearmanWorker();
$worker->addServer();
$worker->addFunction('syncToRedis', 'syncToRedis');

$redis = new Redis();
$redis->connect('172.25.1.4', 6379);

while($worker->work());
function syncToRedis($job)
{
        global $redis;
        $workString = $job->workload();
        $work = json_decode($workString);
        if(!isset($work->id)){
                return false;
        }
        $redis->set($work->id, $work->name);
}
?>

4 后台运行worker

nohup php /usr/local/worker.php

5 更新mysql中的数据