（Redis系列三）centos7实现redis集群数据迁移

本次迁移是把一个三主0从的数据迁移至三主三从
伪集群：172-22-14-88，三个节点6379\6380\6381
真集群：
172-22-14-105 两个节点 6379\6380
172-22-14-106 两个节点 6379\6380
172-22-14-107 两个节点 6379\6380

先看下三主0从的结构，都是集群
主机为172-22-14-88
[root@host-172-22-14-88 cluster-conf]# /data/soft/redis-3.2.12/bin/redis-cli -p 6379 cluster nodes
4b53748f587e80f7f9209062eba94a2945b0799d 172.22.14.88:6379 myself,master - 0 0 1 connected 0-5460
157fe9a2f93fd27ebefc661cbac2e3de5713c16a 172.22.14.88:6380 master - 0 1539226455397 2 connected 5461-10922
806e842e6adf53ee82e155d0bd9805bdf158b68c 172.22.14.88:6381 master - 0 1539226454394 3 connected 10923-16383

[root@host-172-22-14-88 cluster-conf]# /data/soft/redis-3.2.12/bin/redis-cli -p 6379 cluster nodes|grep master
4b53748f587e80f7f9209062eba94a2945b0799d 172.22.14.88:6379 myself,master - 0 0 1 connected 0-5460
157fe9a2f93fd27ebefc661cbac2e3de5713c16a 172.22.14.88:6380 master - 0 1539226482573 2 connected 5461-10922
806e842e6adf53ee82e155d0bd9805bdf158b68c 172.22.14.88:6381 master - 0 1539226481570 3 connected 10923-16383

[root@host-172-22-14-88 cluster-conf]# ls
6379 6380 6381

本次迁移尝试直接复制aof和rdb数据文件

再查看真集群的结构
[root@host-172-22-14-105 ~]# cd /data/soft/redis-3.2.12/cluster-conf/
[root@host-172-22-14-105 cluster-conf]# ls
6379 6380
[root@host-172-22-14-105 cluster-conf]# /data/soft/redis-3.2.12/bin/redis-cli -c -h 172.22.14.105 -p 6379
172.22.14.105:6379> keys *
(empty list or set) 发现真集群中没数据
172.22.14.105:6379> exit
[root@host-172-22-14-105 cluster-conf]#
[root@host-172-22-14-105 cluster-conf]# /data/soft/redis-3.2.12/bin/redis-cli -p 6379 cluster nodes

[root@host-172-22-14-105 cluster-conf]# /data/soft/redis-3.2.12/bin/redis-cli -p 6379 cluster nodes|grep master

[root@host-172-22-14-105 cluster-conf]#

以下是我的尝试，第一种方法失败，大家没有兴趣的话请直接跳到第二种方法

**

第一种方法：思路是直接复制aof和rdb文件进行替换

**

先关掉该服务器的主节点
[root@host-172-22-14-105 cluster-conf]# cat 6379/redis_6379.pid | xargs kill
[root@host-172-22-14-105 cluster-conf]# /data/soft/redis-3.2.12/bin/redis-cli -p 6379 cluster nodes
Could not connect to Redis at 127.0.0.1:6379: Connection refused
连不上了,查看主从是否切换
[root@host-172-22-14-105 cluster-conf]# /data/soft/redis-3.2.12/bin/redis-cli -p 6380 cluster nodes

主从已完成切换，那么把伪集群的6379数据复制过来，不过先备份旧数据
[root@host-172-22-14-105 cluster-conf]# cd 6379
[root@host-172-22-14-105 6379]# ls
appendonly.aof dump.rdb nodes-6379.conf redis.conf redis.log
[root@host-172-22-14-105 6379]# cp appendonly.aof appendonly.aof.bak
[root@host-172-22-14-105 6379]# cp dump.rdb dump.rdb.bak
[root@host-172-22-14-105 6379]# ls
appendonly.aof appendonly.aof.bak dump.rdb dump.rdb.bak nodes-6379.conf redis.conf redis.log
[root@host-172-22-14-105 6379]#
[root@host-172-22-14-105 6379]# ll
总用量 92
-rw-r–r-- 1 root root 0 10月 9 18:35 appendonly.aof
-rw-r–r-- 1 root root 0 10月 11 11:16 appendonly.aof.bak
-rw-r–r-- 1 root root 77 10月 11 11:10 dump.rdb
-rw-r–r-- 1 root root 77 10月 11 11:16 dump.rdb.bak
-rw-r–r-- 1 root root 757 10月 9 18:40 nodes-6379.conf
-rw-r–r-- 1 root root 46808 10月 9 18:34 redis.conf
-rw-r–r-- 1 root root 25230 10月 11 11:10 redis.log

登录172-22-14-88进行复制
[root@host-172-22-14-88 6379]# ll
总用量 36160
-rw-r–r-- 1 root root 35824026 10月 11 11:15 appendonly.aof
-rw-r–r-- 1 root root 58273 10月 11 11:12 dump.rdb
-rw-r–r-- 1 root root 349 9月 17 16:55 nodes.conf
-rw-r–r-- 1 root root 7 9月 21 11:46 redis_6379.pid
-rw-r–r-- 1 root root 48099 9月 29 15:38 redis.conf
-rw-r–r-- 1 root root 1074932 10月 11 11:12 redis.log

[root@host-172-22-14-88 6379]# scp appendonly.aof root@172.22.14.105:/data/soft/redis-3.2.12/cluster-conf/6379/
The authenticity of host ‘172.22.14.105 (172.22.14.105)’ can’t be established.
ECDSA key fingerprint is 40:62:8e:d2:4b??a6:dd:43:c9:26:7d:1b:bc:d5:af.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added ‘172.22.14.105’ (ECDSA) to the list of known hosts.
root@172.22.14.105’s password:
appendonly.aof 100% 34MB 34.2MB/s 00:00
[root@host-172-22-14-88 6379]# scp dump.rdb root@172.22.14.105:/data/soft/redis-3.2.12/cluster-conf/6379/
root@172.22.14.105’s password:
dump.rdb 100% 57KB 56.9KB/s 00:00

登录172-22-14-105查看，并开启redis的6379节点
[root@host-172-22-14-105 6379]# ll
总用量 35136
-rw-r–r-- 1 root root 35824026 10月 11 11:19 appendonly.aof
-rw-r–r-- 1 root root 0 10月 11 11:16 appendonly.aof.bak
-rw-r–r-- 1 root root 58275 10月 11 11:19 dump.rdb
-rw-r–r-- 1 root root 77 10月 11 11:16 dump.rdb.bak
-rw-r–r-- 1 root root 757 10月 9 18:40 nodes-6379.conf
-rw-r–r-- 1 root root 46808 10月 9 18:34 redis.conf
-rw-r–r-- 1 root root 25230 10月 11 11:10 redis.log

开启节点
[root@host-172-22-14-105 6379]# /data/soft/redis-3.2.12/bin/redis-server /data/soft/redis-3.2.12/cluster-conf/6379/redis.conf &
[1] 19313
查看集群状态
[root@host-172-22-14-105 6379]# /data/soft/redis-3.2.12/bin/redis-cli -p 6379 cluster nodes

查看105服务器的redis状态，可以发现6379为slave,6380为master
[root@host-172-22-14-105 6379]# /data/soft/redis-3.2.12/bin/redis-cli -p 6379 cluster nodes|grep 105

登录6379节点查看数据,发现还是为空。排查原因发现是被集群自动清空异常数据
[root@host-172-22-14-105 6379]# /data/soft/redis-3.2.12/bin/redis-cli -c -h 172.22.14.105 -p 6379
172.22.14.105:6379> keys *
(empty list or set)

[root@host-172-22-14-105 6379]# ll
总用量 100
-rw-r–r-- 1 root root 0 10月 11 11:21 appendonly.aof
-rw-r–r-- 1 root root 0 10月 11 11:16 appendonly.aof.bak
-rw-r–r-- 1 root root 77 10月 11 11:22 dump.rdb
-rw-r–r-- 1 root root 77 10月 11 11:16 dump.rdb.bak
-rw-r–r-- 1 root root 819 10月 11 11:21 nodes-6379.conf
-rw-r–r-- 1 root root 6 10月 11 11:21 redis_6379.pid
-rw-r–r-- 1 root root 46808 10月 9 18:34 redis.conf
-rw-r–r-- 1 root root 29738 10月 11 11:22 redis.log

**

第二种方法，采用开源工具redis-migrate-tool来进行迁移

**
使用 开源工具 redis-migrate-tool来进行迁移,开源项目地址：https://github.com/vipshop/redis-migrate-tool
先下载redis-migrate-tool
[root@host-172-22-14-105 ~]# cd /data/soft/
[root@host-172-22-14-105 ~]# git clone https://github.com/vipshop/redis-migrate-tool.git
[root@host-172-22-14-105 soft]# cd redis-migrate-tool/
[root@host-172-22-14-105 redis-migrate-tool]# ls
ChangeLog configure.ac dep LICENSE m4 Makefile.am nodes README.md rmt.conf src test
[root@host-172-22-14-105 redis-migrate-tool]# autoreconf -fvi

[root@host-172-22-14-105 redis-migrate-tool]# ./configure

[root@host-172-22-14-105 redis-migrate-tool]# make
[root@host-172-22-14-105 redis-migrate-tool]# src/redis-migrate-tool -h
This is redis-migrate-tool-0.1.0
ok,到这里工具安装好了

接下去拷贝数据过来
[root@host-172-22-14-88 cluster-conf]# scp 6379/dump.rdb root@172.22.14.105:/data/soft/20181011data/dump6379.rdb
[root@host-172-22-14-88 cluster-conf]# scp 6380/dump.rdb root@172.22.14.105:/data/soft/20181011data/dump6380.rdb
[root@host-172-22-14-88 cluster-conf]# scp 6381/dump.rdb root@172.22.14.105:/data/soft/20181011data/dump6381.rdb

登录105服务器
[root@host-172-22-14-105 20181011data]# ls
dump6379.rdb dump6380.rdb dump6381.rdb

[root@host-172-22-14-105 redis-migrate-tool]# vim rmt.conf

改为：
[source]
type: rdb file
servers :

• /data/soft/20181011data/dump6379.rdb
• /data/soft/20181011data/dump6380.rdb
• /data/soft/20181011data/dump6381.rdb

[target]
type: redis cluster
servers:
-127.0.0.1:6379

[common]
listen: 0.0.0.0:8888

执行迁移
[root@host-172-22-14-105 redis-migrate-tool]# src/redis-migrate-tool -c rmt.conf -o log -d

状态查看：通过redis-cli连接redis-migrate-tool监控的端口，运行info命令
[root@host-172-22-14-105 ~]# /data/soft/redis-3.2.12/bin/redis-cli -h 172.22.14.105 -p 8888

然后查看日志查看是否完成导入
[root@host-172-22-14-105 redis-migrate-tool]# cat log
[2018-10-11 14:35:33.987] rmt_core.c:525 Nodes count of source group : 3
[2018-10-11 14:35:33.987] rmt_core.c:526 Total threads count : 4
[2018-10-11 14:35:33.987] rmt_core.c:527 Read threads count assigned: 1
[2018-10-11 14:35:33.987] rmt_core.c:528 Write threads count assigned: 3
[2018-10-11 14:35:33.990] rmt_core.c:2444 Total threads count in fact: 3
[2018-10-11 14:35:33.990] rmt_core.c:2445 Read threads count in fact: 0
[2018-10-11 14:35:33.990] rmt_core.c:2446 Write threads count in fact: 3
[2018-10-11 14:35:33.990] rmt_core.c:2488 write thread(0):
[2018-10-11 14:35:33.990] rmt_core.c:2494 /data/soft/20181011data/dump6381.rdb
[2018-10-11 14:35:33.990] rmt_core.c:2488 write thread(1):
[2018-10-11 14:35:33.990] rmt_core.c:2494 /data/soft/20181011data/dump6380.rdb
[2018-10-11 14:35:33.990] rmt_core.c:2488 write thread(2):
[2018-10-11 14:35:33.990] rmt_core.c:2494 /data/soft/20181011data/dump6379.rdb
[2018-10-11 14:35:33.991] rmt_core.c:2551 migrate job is running…
[2018-10-11 14:35:33.994] rmt_redis.c:6601 Rdb file for node[/data/soft/20181011data/dump6381.rdb] parsed finished, use: 0 s.
[2018-10-11 14:35:33.994] rmt_redis.c:6601 Rdb file for node[/data/soft/20181011data/dump6380.rdb] parsed finished, use: 0 s.
[2018-10-11 14:35:33.998] rmt_redis.c:6601 Rdb file for node[/data/soft/20181011data/dump6379.rdb] parsed finished, use: 0 s.

通过日志可以发现都finished了，ok，再通过keys * 检查一下
[root@host-172-22-14-105 ~]# /data/soft/redis-3.2.12/bin/redis-cli -c -h 172.22.14.105 -p 6379
172.22.14.105:6379> keys *

正式完成迁移