集群环境说明
现有三台服务器, 每台服务器分布两个节点一个(6379),一个(6380)
-> ps:实际生产中主从服务器是分布在两个服务器上的
192.168.2.111 -> 节点一(6379), 节点二(6380)
192.168.2.112 -> 节点三(6379), 节点四(6380)
192.168.2.113 -> 节点五(6379), 节点六(6380)
单台服务器目录结构:
/usr/local
-> redis
->bin
->logs
->redis_6379.pid
->redis.conf
-> redis6380
->bin
->logs
->redis_6380.pid
->redis.conf
redis.conf配置项
port xxxx:对应端口
bind 0.0.0.0 (ps:时间开发中写对应的ip,, bind 192.168.2.112 192.168.2.113)
cluster-enabled yes:加入集群
cluster-config-file nodes-6379.conf(对应端口)
cluster-node-timeout 5000
appendonly yes
appendfilename "appendonly.aof"
dir /usr/local/redis/
编写两台redis启动脚本(实际开发不需要,这里是为了测试方便)
vim start_redis.sh
cd /usr/local/redis
./bin/redis-server ./redis.conf
cd ..
cd /usr/local/redis6380
./bin/redis-server ./redis.conf
chmod +x start_redis.sh
关闭执行 pkill redis (关闭所有redis服务)
启动执行 /usr/local/start_redis.sh
redis5开启集群配置:
1:redis-cli --cluster create 192.168.2.111:6379 192.168.2.112:6379 192.168.2.113:6379 192.168.2.111:6380 192.168.2.112:6380 192.168.2.113:6380 --cluster-replicas 1
说明:cluster-replicas 1意味着我们希望为每个主机创建一个从服务
[root@192 redis]# redis-cli --cluster create 192.168.2.111:6379 192.168.2.112:6379 192.168.2.113:6379 192.168.2.111:6380 192.168.2.112:6380 192.168.2.113:6380 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.2.112:6380 to 192.168.2.111:6379
Adding replica 192.168.2.111:6380 to 192.168.2.112:6379
Adding replica 192.168.2.113:6380 to 192.168.2.113:6379
>>> Trying to optimize slaves allocation for anti-affinity
[OK] Perfect anti-affinity obtained!
M: 63cfbaf1e7d7d9fce095d0f485776ba15404bf8a 192.168.2.111:6379
slots:[0-5460] (5461 slots) master
M: 75b477ef4967c1bf50502cb834df3fd5660cfa22 192.168.2.112:6379
slots:[5461-10922] (5462 slots) master
M: ebc80d8dcabe30645b96fdfe08de84f52a42cbff 192.168.2.113:6379
slots:[10923-16383] (5461 slots) master
S: f3dbdddc6c2dc23e1dde2e6e6662c6c5ebf167e9 192.168.2.111:6380
replicates 75b477ef4967c1bf50502cb834df3fd5660cfa22
S: e6d09dcaa252336190e4e8747a60d067572d5156 192.168.2.112:6380
replicates ebc80d8dcabe30645b96fdfe08de84f52a42cbff
S: 5da5d0d7f7036c971926a821b33764fb2c78ea50 192.168.2.113:6380
replicates 63cfbaf1e7d7d9fce095d0f485776ba15404bf8a
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.168.2.111:6379)
M: 63cfbaf1e7d7d9fce095d0f485776ba15404bf8a 192.168.2.111:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 75b477ef4967c1bf50502cb834df3fd5660cfa22 192.168.2.112:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: ebc80d8dcabe30645b96fdfe08de84f52a42cbff 192.168.2.113:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: e6d09dcaa252336190e4e8747a60d067572d5156 192.168.2.112:6380
slots: (0 slots) slave
replicates ebc80d8dcabe30645b96fdfe08de84f52a42cbff
S: f3dbdddc6c2dc23e1dde2e6e6662c6c5ebf167e9 192.168.2.111:6380
slots: (0 slots) slave
replicates 75b477ef4967c1bf50502cb834df3fd5660cfa22
S: 5da5d0d7f7036c971926a821b33764fb2c78ea50 192.168.2.113:6380
slots: (0 slots) slave
replicates 63cfbaf1e7d7d9fce095d0f485776ba15404bf8a
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
问题
1:Could not connect to Redis at 192.168.2.111:6379: No route to host
参考redis与防火墙设置
2:redis-cli --cluster create一直在等待
有可能是监控端口没有开放 端口+10000 -> 例如16379端口
3: [ERR] Node 192.168.2.111:6379 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
1)、将需要新增的节点下aof、rdb等本地备份文件删除;
2)、同时将新Node的集群配置文件删除,即:删除你redis.conf里面cluster-config-file所在的文件;
3)、使用redis-cli -c -h -p登录每个redis节点,使用以下命令
flushdb
cluster reset
4:节点检查
以检查节点“.168.2.111:6381”的状态为例:
redis-cli --cluster check 192.168.2.111:6381
如发现如下这样的错误:
[WARNING] Node 192.168.2.111:6381 has slots in importing state 5461.
[WARNING] The following slots are open: 5461.
可以使用redis命令取消slots迁移(5461为slot的ID):
先连接对应redis-cli:redis-cli --cluster check 192.168.2.111:6381
执行:cluster setslot 5461 stable
集群基础操作
# 连接集群
redis-cli -h 192.168.2.111 -p 6379 -c (ps: -c 代表用集群的方式连接)
# 新增示例
192.168.2.111:6379> set name iwester
-> Redirected to slot [5798] located at 192.168.2.112:6379
OK
ps: 添加会根据key计算放到哪一个哈希槽中,自动跳转
# 获取示例
192.168.2.111:6379> get name
-> Redirected to slot [5798] located at 192.168.2.112:6379
"iwester"
ps: 从111:6379端口获取放到112:6379中的内容,会自动跳转到对应的服务端口
# 检查集群节点是否正常
redis-cli --cluster check 192.168.2.111:6380
-> 可以看到集群中节点情况, 包含keys的数量,分布,包含每个节点分配的槽数。运行状态
# 查看集群信息
192.168.2.112:6379> cluster info
cluster_state:ok -> 状态ok
cluster_slots_assigned:16384 -> 16384个槽
cluster_slots_ok:16384 -> 正常状态槽的数量
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:2
cluster_stats_messages_ping_sent:1029
cluster_stats_messages_pong_sent:1157
cluster_stats_messages_meet_sent:5
cluster_stats_messages_sent:2191
cluster_stats_messages_ping_received:1157
cluster_stats_messages_pong_received:1034
cluster_stats_messages_received:2191
# 查看集群节点信息
192.168.2.112:6379> cluster nodes
ebc80d8dcabe30645b96fdfe08de84f52a42cbff 192.168.2.113:6379@16379 master - 0 1555494741000 3 connected 10923-16383
解释:
-> ebc80d8dcabe30645b96fdfe08de84f52a42cbff:当前节点id
-> 192.168.2.113:6379: ip端口号
-> 16379 : 监控端口号
-> master - 0: 主节点
-> slave 63cfbaf1e7d7d9fce095d0f485776ba15404bf8a:从节点 + 主节点id
-> connected 10923-16383 : 分布的哈希槽区间, 从节点没有哈希槽
63cfbaf1e7d7d9fce095d0f485776ba15404bf8a 192.168.2.111:6379@16379 master - 0 1555494741371 1 connected 0-5460
5da5d0d7f7036c971926a821b33764fb2c78ea50 192.168.2.113:6380@16380 slave 63cfbaf1e7d7d9fce095d0f485776ba15404bf8a 0 1555494742000 6 connected
e6d09dcaa252336190e4e8747a60d067572d5156 192.168.2.112:6380@16380 slave ebc80d8dcabe30645b96fdfe08de84f52a42cbff 0 1555494741000 5 connected
75b477ef4967c1bf50502cb834df3fd5660cfa22 192.168.2.112:6379@16379 myself,master - 0 1555494742000 2 connected 5461-10922
f3dbdddc6c2dc23e1dde2e6e6662c6c5ebf167e9 192.168.2.111:6380@16380 slave 75b477ef4967c1bf50502cb834df3fd5660cfa22 0 1555494742382 4 connected
# 集群操作帮助
root@192 redis6382]# redis-cli --cluster help