Redis集群搭建全记录

 ruby ftp封装实例详解

最近自己用ruby 封装了一个Net::FTP的工具类.

Redis集群是一个提供在多个Redis节点间共享数据的程序集。

  Redis集群中不支持处理多个keys的命令。

  Redis集群通过分区来提供一定程度的可用性。在某个节点宕机或者不可用的时候可以继续处理命令。

Redis集群数据分片

  在Redis集群中,使用数据分片(sharding)而不是一致性hash(consistency hashing)来实现,一个Redis集群包含16384个哈希槽(hash slot),数据库中的每个键都存在这些哈希槽中的某一个,通过CRC16校验后对16384取模来决定。

  加入说现在集群中有三个节点,那么

  ①节点A包含0到5500号哈希槽。

  ②节点B包含5501到11000号哈希槽。

  ③节点C包含11001到16384号哈希槽。

  然后如果我们要增加一个节点的时候,会从ABC三个节点分别得到一部分槽到D上。如果我们移除一个节点的时候,就会把移除的节点的分隔槽移到剩下的槽上面。

Redis主从复制模型

  为了实现Redis的集群的高可用性。Redis提供了一个主从复制模型。每个节点都会有N-1个复制品。

  如果在创建集群的时候,我们为每个节点添加了一个从节点,这时候,如果其中的某个主节点挂掉了,便会把它的从节点做为新的主节点,继续提供服务。但是如果主节点和从节点都挂掉了,那就不可以继续使用了。

Redis集群搭建

  Redis集群由多个云新在集群模式下的Redis实例组成。实例的集群模式需要通过配置来开启。

  下面是一个包含了最少选项的集群配置文件实例:

1

2

3

4

5

6

7

port 7000

cluster-enabled yes

cluster-config-file nodes.conf

cluster-node-timeout 5000

appendonly yes

pidfile /var/run/redis_xxxx.pid

dir /usr/local/redis-cluster/xxxx

port指定了节点的端口号

cluster-enabled yes选项用于开启实例的集群模式

cluster-config-file nodes.conf设置了保存节点的配置文件路径,这个文件无须认为修改,是在集群启动的时候创建。

cluster-node-timeout 5000:设置了方式失败的等待时间。即5秒还访问不了就认为这个节点不可用。

appendonly yes:用于开启aof持久化

 pidfile /var/run/redis_xxxx.pid设置pid文件的位置,其中xxxx为端口号

dir /usr/local/redis-cluster/xxxx设置工作目录,其中xxxx为端口号。

此时目录中的文件是这样的

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

[root@localhost 10:49 /usr/local/redis-cluster]# ll 7000 7001 7002 7003 7004 7005

7000:

总用量 7080

-rw-r--r--. 1 root root  57787 910 10:44 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 09:50 redis-server

7001:

总用量 7080

-rw-r--r--. 1 root root  57787 910 10:46 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7002:

总用量 7080

-rw-r--r--. 1 root root  57787 910 10:46 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7003:

总用量 7080

-rw-r--r--. 1 root root  57787 910 10:47 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7004:

总用量 7080

-rw-r--r--. 1 root root  57787 910 10:48 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7005:

总用量 7080

-rw-r--r--. 1 root root  57787 910 10:48 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

[root@localhost 10:49 /usr/local/redis-cluster]#

下面启动这六个节点

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

[root@localhost 10:51 /usr/local/redis-cluster]# ./7000/redis-server ./7000/redis.conf

3547:C 10 Sep 10:51:58.519 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3547:C 10 Sep 10:51:58.519 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3547, just started

3547:C 10 Sep 10:51:58.519 # Configuration loaded

[root@localhost 10:51 /usr/local/redis-cluster]# ./7001/redis-server ./7001/redis.conf

3552:C 10 Sep 10:52:05.549 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3552:C 10 Sep 10:52:05.550 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3552, just started

3552:C 10 Sep 10:52:05.550 # Configuration loaded

[root@localhost 10:52 /usr/local/redis-cluster]# ./7002/redis-server ./7002/redis.conf

3557:C 10 Sep 10:52:13.098 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3557:C 10 Sep 10:52:13.098 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3557, just started

3557:C 10 Sep 10:52:13.098 # Configuration loaded

[root@localhost 10:52 /usr/local/redis-cluster]# ./7003/redis-server ./7003/redis.conf

3563:C 10 Sep 10:52:18.986 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3563:C 10 Sep 10:52:18.986 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3563, just started

3563:C 10 Sep 10:52:18.986 # Configuration loaded

[root@localhost 10:52 /usr/local/redis-cluster]# ./7004/redis-server ./7004/redis.conf

3568:C 10 Sep 10:52:23.709 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3568:C 10 Sep 10:52:23.710 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3568, just started

3568:C 10 Sep 10:52:23.710 # Configuration loaded

[root@localhost 10:52 /usr/local/redis-cluster]# ./7005/redis-server ./7005/redis.conf

3573:C 10 Sep 10:52:27.146 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

3573:C 10 Sep 10:52:27.147 # Redis version=4.0.1, bits=32, commit=00000000, modified=0, pid=3573, just started

3573:C 10 Sep 10:52:27.147 # Configuration loaded

[root@localhost 10:52 /usr/local/redis-cluster]#

查看启动后的进程:

1

2

3

4

5

6

7

8

9

[root@localhost 10:52 /usr/local/redis-cluster]# ps aux|grep redis

root   3548 0.2 0.2 41964 2288 ?    Ssl 10:51  0:00 ./7000/redis-server 127.0.0.1:7000 [cluster]

root   3553 0.1 0.2 41964 2288 ?    Ssl 10:52  0:00 ./7001/redis-server 127.0.0.1:7001 [cluster]

root   3558 0.2 0.2 41964 2288 ?    Ssl 10:52  0:00 ./7002/redis-server 127.0.0.1:7002 [cluster]

root   3564 0.1 0.2 41964 2292 ?    Ssl 10:52  0:00 ./7003/redis-server 127.0.0.1:7003 [cluster]

root   3569 0.2 0.2 41964 2292 ?    Ssl 10:52  0:00 ./7004/redis-server 127.0.0.1:7004 [cluster]

root   3574 0.1 0.2 41964 2288 ?    Ssl 10:52  0:00 ./7005/redis-server 127.0.0.1:7005 [cluster]

root   3580 0.0 0.0  6048  784 pts/2  S10:52  0:00 grep redis

[root@localhost 10:52 /usr/local/redis-cluster]#

再次查看文件目录:

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

[root@localhost 11:01 /usr/local/redis-cluster]# ll 7000 7001 7002 7003 7004 7005

7000:

总用量 7084

-rw-r--r--. 1 root root    0 910 10:51 appendonly.aof

-rw-r--r--. 1 root root   114 910 10:51 nodes-7000.conf

-rw-r--r--. 1 root root  57787 910 10:44 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 09:50 redis-server

7001:

总用量 7084

-rw-r--r--. 1 root root    0 910 10:52 appendonly.aof

-rw-r--r--. 1 root root   114 910 10:52 nodes-7001.conf

-rw-r--r--. 1 root root  57787 910 10:46 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7002:

总用量 7084

-rw-r--r--. 1 root root    0 910 10:52 appendonly.aof

-rw-r--r--. 1 root root   114 910 10:52 nodes-7002.conf

-rw-r--r--. 1 root root  57787 910 10:46 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7003:

总用量 7084

-rw-r--r--. 1 root root    0 910 10:52 appendonly.aof

-rw-r--r--. 1 root root   114 910 10:52 nodes-7003.conf

-rw-r--r--. 1 root root  57787 910 10:47 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7004:

总用量 7084

-rw-r--r--. 1 root root    0 910 10:52 appendonly.aof

-rw-r--r--. 1 root root   114 910 10:52 nodes-7004.conf

-rw-r--r--. 1 root root  57787 910 10:48 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

7005:

总用量 7084

-rw-r--r--. 1 root root    0 910 10:52 appendonly.aof

-rw-r--r--. 1 root root   114 910 10:52 nodes-7005.conf

-rw-r--r--. 1 root root  57787 910 10:48 redis.conf

-rwxr-xr-x. 1 root root 7185872 910 10:04 redis-server

[root@localhost 11:01 /usr/local/redis-cluster]#

可以看到生成了aof和nodes文件,保存的时候还会有dump文件生成。

在看看pid

1

2

3

4

5

6

7

8

[root@localhost 11:03 /usr/local/redis-cluster]# ll /var/run/redis*

-rw-r--r--. 1 root root 5 910 10:51 /var/run/redis_7000.pid

-rw-r--r--. 1 root root 5 910 10:52 /var/run/redis_7001.pid

-rw-r--r--. 1 root root 5 910 10:52 /var/run/redis_7002.pid

-rw-r--r--. 1 root root 5 910 10:52 /var/run/redis_7003.pid

-rw-r--r--. 1 root root 5 910 10:52 /var/run/redis_7004.pid

-rw-r--r--. 1 root root 5 910 10:52 /var/run/redis_7005.pid

[root@localhost 11:03 /usr/local/redis-cluster]#

接下来,我们就需要使用redis-trib工具和这六个节点来创建集群了。

redis-trib位于redis源码的src目录下。

我们复制一份到redis-cluster目录下。

1

2

3

4

5

6

7

8

9

10

11

12

13

[root@localhost 11:03 /usr/local/redis-cluster]# ll ../redis/src/redis-trib*

-rwxrwxr-x. 1 root root 60843 724 22:58 ../redis/src/redis-trib.rb

[root@localhost 11:04 /usr/local/redis-cluster]# cp ../redis/src/redis-trib.rb ./

[root@localhost 11:05 /usr/local/redis-cluster]# ll

总用量 84

drwxr-xr-x. 2 root root 4096 910 10:51 7000

drwxr-xr-x. 2 root root 4096 910 10:52 7001

drwxr-xr-x. 2 root root 4096 910 10:52 7002

drwxr-xr-x. 2 root root 4096 910 10:52 7003

drwxr-xr-x. 2 root root 4096 910 10:52 7004

drwxr-xr-x. 2 root root 4096 910 10:52 7005

-rwxr-xr-x. 1 root root 60843 910 11:05 redis-trib.rb

[root@localhost 11:05 /usr/local/redis-cluster]#

开始启动集群:

1

2

3

[root@localhost 11:13 /usr/local/redis-cluster]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

/usr/bin/env: ruby: 没有那个文件或目录

[root@localhost 11:15 /usr/local/redis-cluster]#

下面是本人失败的安装版本,后面也有成功的版本。

因为是ruby的程序,所以我们需要安装ruby,

yum -y install  ruby

继续启动集群:

1

2

3

4

[root@localhost 11:21 /usr/local/redis-cluster]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

./redis-trib.rb:24:in `require': no such file to load -- rubygems (LoadError)

    from ./redis-trib.rb:24

[root@localhost 11:22 /usr/local/redis-cluster]#

还是报错了,说需要rubygems,我们继续安装:

yum -y install rubygems

继续启动集群:

1

2

3

4

5

[root@localhost 11:24 /usr/local/redis-cluster]# ./redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

/usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require': no such file to load -- redis (LoadError)

    from /usr/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'

    from ./redis-trib.rb:25

[root@localhost 11:24 /usr/local/redis-cluster]#

依然还是报错了...,我们需要安装更高版本的ruby。

 所以,我决定,先卸载掉1.8的ruby。

yum -y remove ruby

然后重新安装

1

2

3

4

5

6

wget https://cache.ruby-lang.org/pub/ruby/2.4/ruby-2.4.1.tar.gz

tar xzvf ruby-2.4.1.tar.gz

cd ruby-2.4.1

./configure

make

make install

然后查看ruby的版本:

1

2

3

[root@localhost 12:47 ~]# ruby -v

ruby 2.4.1p111 (2017-03-22 revision 58053) [i686-linux]

[root@localhost 12:47 ~]#

最后,为了运行redis-trib我们需要安装redis gem。

如果出现下面的错误

1

2

3

4

5

[root@localhost 13:01 /usr/local/redis-cluster]# gem install redis

ERROR: Loading command: install (LoadError)

    cannot load such file -- zlib

ERROR: While executing gem ... (NoMethodError)

  undefined method `invoke_with_build_args' for nil:NilClass

我们继续开启集群:依然还是报错,几经折腾,还是没解决掉

最好,找到了下面这种方法。

成功版本

通过rvm这个ruby的管理工具来安装ruby。

首先安装rvm,如果下面的命令提示找不到curl,可以通过yum install -y crul来安装crul。

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

[root@localhost 19:45 ~]# curl -L get.rvm.io | bash -s stable

 % Total  % Received % Xferd Average Speed  Time  Time   Time Current

                 Dload Upload  Total  Spent  Left Speed

100 24090 100 24090  0   0 11517   0 0:00:02 0:00:02 --:--:-- 48276

Downloading https://github.com/rvm/rvm/archive/1.29.3.tar.gz

Downloading https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc

gpg: 于 2017年09月11日 星期一 05时59分21秒 JST 创建的签名,使用 RSA,钥匙号 BF04FF17

gpg: 无法检查签名:No public key

Warning, RVM 1.26.0 introduces signed releases and automated check of signatures when GPG software found. Assuming you trust Michal Papis import the mpapis public key (downloading the signatures).

GPG signature verification failed for '/usr/local/rvm/archives/rvm-1.29.3.tgz' - 'https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc'! Try to install GPG v2 and then fetch the public key:

  gpg2 --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

or if it fails:

  command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -

the key can be compared with:

  https://rvm.io/mpapis.asc

  https://keybase.io/mpapis

NOTE: GPG version 2.1.17 have a bug which cause failures during fetching keys from remote server. Please downgrade or upgrade to newer version (if available) or use the second method described above.

[root@localhost 19:45 ~]#

发现失败了,我们按照提示进行操作。

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

[root@localhost 19:45 ~]# curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -

gpg: 钥匙环‘/root/.gnupg/secring.gpg'已建立

gpg: /root/.gnupg/trustdb.gpg:建立了信任度数据库

gpg: 密钥 D39DC0E3:公钥“Michal Papis (RVM signing) <mpapis@gmail.com>”已导入

gpg: 合计被处理的数量:1

gpg:      已导入:1 (RSA: 1)

gpg: 没有找到任何绝对信任的密钥

[root@localhost 19:45 ~]# curl -L get.rvm.io | bash -s stable

 % Total  % Received % Xferd Average Speed  Time  Time   Time Current

                 Dload Upload  Total  Spent  Left Speed

100 24090 100 24090  0   0  3763   0 0:00:06 0:00:06 --:--:-- 3763

Downloading https://github.com/rvm/rvm/archive/1.29.3.tar.gz

Downloading https://github.com/rvm/rvm/releases/download/1.29.3/1.29.3.tar.gz.asc

gpg: 于 2017年09月11日 星期一 05时59分21秒 JST 创建的签名,使用 RSA,钥匙号 BF04FF17

gpg: 完好的签名,来自于“Michal Papis (RVM signing) <mpapis@gmail.com>”

gpg:        亦即“Michal Papis <michal.papis@toptal.com>”

gpg:        亦即“[jpeg image of size 5015]”

gpg: 警告:这把密钥未经受信任的签名认证!

gpg:    没有证据表明这个签名属于它所声称的持有者。

主钥指纹: 409B 6B17 96C2 7546 2A17 0311 3804 BB82 D39D C0E3

子钥指纹: 62C9 E5F4 DA30 0D94 AC36 166B E206 C29F BF04 FF17

GPG verified '/usr/local/rvm/archives/rvm-1.29.3.tgz'

Creating group 'rvm'

Installing RVM to /usr/local/rvm/

Installation of RVM in /usr/local/rvm/ is almost complete:

 * First you need to add all users that will be using rvm to 'rvm' group,

  and logout - login again, anyone using rvm will be operating with `umask u=rwx,g=rwx,o=rx`.

 * To start using RVM you need to run `source /etc/profile.d/rvm.sh`

  in all your open shell windows, in rare cases you need to reopen all shell windows.

[root@localhost 19:46 ~]

发现rvm已经安装成功了,为了让配置立即生效,我们需要用source命令导入rvm的配置文件。

先查看下配置文件的路径,然后导入。

1

2

3

[root@localhost 19:46 ~]# find / -name rvm.sh

/etc/profile.d/rvm.sh

[root@localhost 19:49 ~]# source /etc/profile.d/rvm.sh

接下来就可以安装高版本的ruby了,建议安装2.2.3版本,不然的话,等下gen install redis的时候,可能会报下面的错误:

1

2

3

4

[root@localhost 19:59 ~]# gem install redis

Fetching: redis-4.0.0.gem (100%)

ERROR: Error installing redis:

    redis requires Ruby version >= 2.2.2.

废话少说,接下来安装2.2.3版本的ruby

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

[root@localhost 20:16 ~]# rvm install 2.3.3

Searching for binary rubies, this might take some time.

No binary rubies available for: centos/6/i386/ruby-2.3.3.

Continuing with compilation. Please read 'rvm help mount' to get more information on binary rubies.

Checking requirements for centos.

Requirements installation successful.

Installing Ruby from source to: /usr/local/rvm/rubies/ruby-2.3.3, this may take a while depending on your cpu(s)...

ruby-2.3.3 - #downloading ruby-2.3.3, this may take a while depending on your connection...

 % Total  % Received % Xferd Average Speed  Time  Time   Time Current

                 Dload Upload  Total  Spent  Left Speed

100 13.7M 100 13.7M  0   0 1285k   0 0:00:10 0:00:10 --:--:-- 1394k

ruby-2.3.3 - #extracting ruby-2.3.3 to /usr/local/rvm/src/ruby-2.3.3....

ruby-2.3.3 - #applying patch /usr/local/rvm/patches/ruby/ruby_2_3_gcc7.patch.

ruby-2.3.3 - #applying patch /usr/local/rvm/patches/ruby/2.3.3/random_c_using_NR_prefix.patch.

ruby-2.3.3 - #configuring..........................................................

ruby-2.3.3 - #post-configuration..

ruby-2.3.3 - #compiling....................................................................................-

ruby-2.3.3 - #installing...........................

ruby-2.3.3 - #making binaries executable..

ruby-2.3.3 - #downloading rubygems-2.6.13

ruby-2.3.3 - #extracting rubygems-2.6.13.....

ruby-2.3.3 - #removing old rubygems.........

ruby-2.3.3 - #installing rubygems-2.6.13..........................

ruby-2.3.3 - #gemset created /usr/local/rvm/gems/ruby-2.3.3@global

ruby-2.3.3 - #importing gemset /usr/local/rvm/gemsets/global.gems..........................................|

ruby-2.3.3 - #generating global wrappers........

ruby-2.3.3 - #gemset created /usr/local/rvm/gems/ruby-2.3.3

ruby-2.3.3 - #importing gemsetfile /usr/local/rvm/gemsets/default.gems evaluated to empty gem list

ruby-2.3.3 - #generating default wrappers........

ruby-2.3.3 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake).

Install of ruby-2.3.3 - #complete

Please be aware that you just installed a ruby that requires 2 patches just to be compiled on an up to date linux system.

This may have known and unaccounted for security vulnerabilities.

Please consider upgrading to ruby-2.4.1 which will have all of the latest security patches.

Ruby was built without documentation, to build it run: rvm docs generate-ri

[root@localhost 20:27 ~]# ruby -v

ruby 2.3.3p222 (2016-11-21 revision 56859) [i686-linux]

[root@localhost 20:28 ~]# gem -v

2.6.13

[root@localhost 20:28 ~]#

可以看到,不进ruby安装好了2.3.3版本,还把rubygems也安装好了。

如果要删除掉某个版本的ruby可以用这个命令:rvm remove 1.9.3

我们继续安装:

1

2

3

4

5

6

7

8

[root@localhost 20:28 ~]# gem install redis

Fetching: redis-4.0.0.gem (100%)

Successfully installed redis-4.0.0

Parsing documentation for redis-4.0.0

Installing ri documentation for redis-4.0.0

Done installing documentation for redis after 3 seconds

1 gem installed

[root@localhost 20:33 ~]#

看到这里,折腾了两个晚上终于弄好了。接下来就是使用我们的redis-trib开启集群了。

为了方便,我把redis-trib.rb这个文件复制了一份到/usr/local/bin这个目录下,因为这个目录在PATH下面,里面的命令可以直接执行。

再次确认下六个redis节点的运行是ok的。

1

2

3

4

5

6

7

8

9

[root@localhost 20:35 /usr/local/redis-cluster]# ps aux|grep redis

root   2486 0.1 0.2 46060 2304 ?    Ssl 18:30  0:12 ./7000/redis-server 127.0.0.1:7000 [cluster]

root   2491 0.1 0.2 46060 2308 ?    Ssl 18:30  0:13 ./7001/redis-server 127.0.0.1:7001 [cluster]

root   2496 0.1 0.2 46060 2308 ?    Ssl 18:30  0:13 ./7002/redis-server 127.0.0.1:7002 [cluster]

root   2501 0.1 0.2 46060 2304 ?    Ssl 18:30  0:13 ./7003/redis-server 127.0.0.1:7003 [cluster]

root   2506 0.1 0.2 46060 2308 ?    Ssl 18:30  0:14 ./7004/redis-server 127.0.0.1:7004 [cluster]

root   2511 0.1 0.2 46060 2304 ?    Ssl 18:30  0:12 ./7005/redis-server 127.0.0.1:7005 [cluster]

root   31426 0.0 0.0  6048  784 pts/2  S+  20:35  0:00 grep redis

[root@localhost 20:35 /usr/local/redis-cluster]#

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

[root@localhost 20:35 /usr/local/redis-cluster]# redis-trib.rb create --replicas 1 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005

>>> Creating cluster

>>> Performing hash slots allocation on 6 nodes...

Using 3 masters:

127.0.0.1:7000

127.0.0.1:7001

127.0.0.1:7002

Adding replica 127.0.0.1:7003 to 127.0.0.1:7000

Adding replica 127.0.0.1:7004 to 127.0.0.1:7001

Adding replica 127.0.0.1:7005 to 127.0.0.1:7002

M: 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000

  slots:0-5460 (5461 slots) master

M: 9bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001

  slots:5461-10922 (5462 slots) master

M: 6de604de12b4b4a3be46766bb95ccf4618dc0d75 127.0.0.1:7002

  slots:10923-16383 (5461 slots) master

S: f1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003

  replicates 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2

S: 3ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004

  replicates 9bf2ee5d32fc350ec411d3eaad18f82492796e99

S: a4a09d80a4bb6c82345c376c59e5ae49e1d49701 127.0.0.1:7005

  replicates 6de604de12b4b4a3be46766bb95ccf4618dc0d75

Can I set the above configuration? (type 'yes' to accept): yes

>>> Nodes configuration updated

>>> Assign a different config epoch to each node

>>> Sending CLUSTER MEET messages to join the cluster

Waiting for the cluster to join...

>>> Performing Cluster Check (using node 127.0.0.1:7000)

M: 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000

  slots:0-5460 (5461 slots) master

  1 additional replica(s)

S: f1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003

  slots: (0 slots) slave

  replicates 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2

M: 6de604de12b4b4a3be46766bb95ccf4618dc0d75 127.0.0.1:7002

  slots:10923-16383 (5461 slots) master

  1 additional replica(s)

M: 9bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001

  slots:5461-10922 (5462 slots) master

  1 additional replica(s)

S: 3ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004

  slots: (0 slots) slave

  replicates 9bf2ee5d32fc350ec411d3eaad18f82492796e99

S: a4a09d80a4bb6c82345c376c59e5ae49e1d49701 127.0.0.1:7005

  slots: (0 slots) slave

  replicates 6de604de12b4b4a3be46766bb95ccf4618dc0d75

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

[root@localhost 20:37 /usr/local/redis-cluster]#

在创建的过程中,redis会给出一份预想的配置给我们看,确定没问题就输入yes并回车。系统就给我们创建了三个主,三个从的集群结构。

可以看到All 16384 slots covered。

redis-cli

redis-cli默认情况下连接的是本地的6379端口的redis服务器。

现在有六个端口,所以需要指定参数。

redis-cli -h xxx -p xxx -a xxx

-h:指定服务器

-p:指定端口号

-a:指定密码

-c:开启集群模式

1

2

3

4

5

6

7

8

9

10

[root@localhost 20:53 /usr/local/redis-cluster]# redis-cli -c -p 7000

127.0.0.1:7000> set name zhangsan

-> Redirected to slot [5798] located at 127.0.0.1:7001

OK

127.0.0.1:7001> exit

[root@localhost 20:53 /usr/local/redis-cluster]# redis-cli -c -p 7002

127.0.0.1:7002> get name

-> Redirected to slot [5798] located at 127.0.0.1:7001

"zhangsan"

127.0.0.1:7001>

可以看到,已经可以共享数据了。

 但是,如果我们把某一个主节点杀掉呢?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

[root@localhost 19:52 /usr/local/redis-cluster]# redis-cli -p 7002 debug segfault

Error: Server closed the connection

[root@localhost 19:53 /usr/local/redis-cluster]# redis-trib.rb check 127.0.0.1:7000

>>> Performing Cluster Check (using node 127.0.0.1:7000)

M: 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000

  slots:0-5460 (5461 slots) master

  1 additional replica(s)

M: 9bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001

  slots:5461-10922 (5462 slots) master

  1 additional replica(s)

S: 3ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004

  slots: (0 slots) slave

  replicates 9bf2ee5d32fc350ec411d3eaad18f82492796e99

M: a4a09d80a4bb6c82345c376c59e5ae49e1d49701 127.0.0.1:7005

  slots:10923-16383 (5461 slots) master

  0 additional replica(s)

S: f1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003

  slots: (0 slots) slave

  replicates 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[OK] All 16384 slots covered.

[root@localhost 19:54 /usr/local/redis-cluster]#

可以看到7005端口的节点已经变成主节点了。那如果继续把7005节点也宕掉呢?

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

[root@localhost 19:58 /usr/local/redis-cluster]# redis-cli -p 7005 debug segfault

Error: Server closed the connection

[root@localhost 19:58 /usr/local/redis-cluster]# redis-trib.rb check 127.0.0.1:7000

[ERR] Sorry, can't connect to node 127.0.0.1:7005

>>> Performing Cluster Check (using node 127.0.0.1:7000)

M: 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2 127.0.0.1:7000

  slots:0-5460 (5461 slots) master

  1 additional replica(s)

M: 9bf2ee5d32fc350ec411d3eaad18f82492796e99 127.0.0.1:7001

  slots:5461-10922 (5462 slots) master

  1 additional replica(s)

S: 3ff95d180ec47fc58c9c5f73e2f54354128bfe7e 127.0.0.1:7004

  slots: (0 slots) slave

  replicates 9bf2ee5d32fc350ec411d3eaad18f82492796e99

S: f1bdbd841e37825169426486d6502e2cd99b76fe 127.0.0.1:7003

  slots: (0 slots) slave

  replicates 6d996f9e34f40b02afe06aa9d3c8f18a41875cb2

[OK] All nodes agree about slots configuration.

>>> Check for open slots...

>>> Check slots coverage...

[ERR] Not all 16384 slots are covered by nodes.

[root@localhost 19:58 /usr/local/redis-cluster]#

可以看到集群已经出现问题了。

追记:

在搭建多机多点集群的时候,遇到以下问题:

1、一直在等待...

>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join.......................

解决办法:确认端口是否开放。如果没有开发,可以修改/etc/sysconfig/iptables文件,开放端口(复制一行前面开放端口的来修改),然后service iptables restart重启防火墙。

极端的可以service iptables stop来关闭防火墙

2、槽被利用

1

2

3

4

5

6

7

8

9

10

11

12

/usr/local/rvm/gems/ruby-2.3.3/gems/redis-4.0.0/lib/redis/client.rb:119:in `call': ERR Slot 5461 is already busy (Redis::CommandError)

 from /usr/local/rvm/gems/ruby-2.3.3/gems/redis-4.0.0/lib/redis.rb:2742:in `block in method_missing'

 from /usr/local/rvm/gems/ruby-2.3.3/gems/redis-4.0.0/lib/redis.rb:45:in `block in synchronize'

 from /usr/local/rvm/rubies/ruby-2.3.3/lib/ruby/2.3.0/monitor.rb:214:in `mon_synchronize'

 from /usr/local/rvm/gems/ruby-2.3.3/gems/redis-4.0.0/lib/redis.rb:45:in `synchronize'

 from /usr/local/rvm/gems/ruby-2.3.3/gems/redis-4.0.0/lib/redis.rb:2741:in `method_missing'

 from /usr/local/bin/redis-trib.rb:212:in `flush_node_config'

 from /usr/local/bin/redis-trib.rb:776:in `block in flush_nodes_config'

 from /usr/local/bin/redis-trib.rb:775:in `each'

 from /usr/local/bin/redis-trib.rb:775:in `flush_nodes_config'

 from /usr/local/bin/redis-trib.rb:1296:in `create_cluster_cmd'

 from /usr/local/bin/redis-trib.rb:1700:in `<main>'

解决办法:删除掉在redis.conf的dir目录下的几个文件redis.conf文件除外。

如果还是有相同的问题,可以考虑对所有节点执行下面两个命令。

redis-cli -p 端口号-h 主机物理地址 FLUSHALL

redis-cli -p 端口号-h 主机物理地址 CLUSTER RESET SOFT

  # 获取指定格式的文件名称列表

  # 例如: source = "test/*.txt"

  # 返回: [source/file_name.txt]

1

2

3

4

5

6

7

8

9

10

11

def fetch_remote_filenames(source)

 return [] if source.blank?

 log_info("source is " + source)

 filenames = @current_ftp.nlst(source)

 filenames

end

   # 获取服务器上确切名称的文件

  # 例如: get("test/test.txt")

  # 文件将被保存到本地 tmp/test/test.txt

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

def get(origin_file)

 local_file = local_file(origin_file)

 local_file.gsub("\\", "\\\\") #此处注意是window下执行, 在linux下需要注意改成/

 log_info("Ftp Get: #{origin_file} -> #{local_file}")

 begin

  @current_ftp.getbinaryfile(origin_file, local_file+".tmp")

 rescue

  delete_local_file(local_file+".tmp")

 end

 rename_local_file(local_file+".tmp", local_file) if File.exist?(local_file+".tmp")

end

   # 上传文件到指定的路径

  # 例如: put("tmp\\test\\test.txt", "/test/")

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

def put(origin_file, remote_path)

  return nil if not File.exist?(origin_file)

  _file_name = File.basename(origin_file)

  _root = @current_ftp.getdir

  @current_ftp.chdir(remote_path)

  log_info("Ftp put: #{origin_file} -> #{remote_path}")

  begin

   @current_ftp.putbinaryfile(origin_file, remote_path + _file_name + ".tmp")

  rescue

   delete(remote_path + _file_name + ".tmp")

  end

  @current_ftp.chdir(_root)

  rename(remote_path + _file_name + ".tmp", remote_path + _file_name)

 end

  # 关闭ftp

1

2

3

4

5

6

7

def close

 @current_ftp.close if @current_ftp

end

  # 服务器copy文件 

1

2

3

4

5

6

7

def copy(origin_file, file_path)

 local_file = local_file(origin_file)

 _file_name = File.basename(origin_file)

 begin

      #1. 到本地    

1

2

3

4

5

log_info("FTP get file to:" + local_file+".tmp")

  @current_ftp.getbinaryfile(origin_file, local_file+".tmp")

  return nil if not File.exist?(local_file+".tmp")

      #2. 到服务器   

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

log_info("FTP put file to :" + file_path + _file_name + ".tmp")

  @current_ftp.putbinaryfile(local_file+".tmp", file_path + _file_name + ".tmp")

  #3. 改名字

  rename(file_path + _file_name + ".tmp", file_path + _file_name)

  #5. 删除本地

  delete_local_file(local_file + ".tmp")

 rescue => e

  log_info(e)

  #4. 删除服务器上临时文件

  delete(file_path + origin_file + ".tmp")

  #5. 删除本地

  delete_local_file(local_file + ".tmp")

 end

end

   # 服务器上移动文件

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

def move(origin_file, file_path)

 _file_name = File.basename(origin_file)

 begin

  copy(origin_file, file_path)

  # 删除服务器上源文件

  delete(origin_file)

 rescue => e

  log_info(e)

  # 删除临时文件,如果存在

  delete(file_path + _file_name + ".tmp")

  # 删除服务器上目标文件, 如果存在

  delete(file_path + _file_name)

 end

end

   # 重命名服务器文件

1

2

3

4

5

6

7

8

9

10

11

12

13

def rename(origin_file, file)

 if not @current_ftp.list(origin_file).blank?

  log_info("FTP rename #{origin_file} to #{file}")

  @current_ftp.rename(origin_file, file)

 end

end

  # 删除服务器上的文件

1

2

3

4

5

6

7

8

9

10

11

12

13

def delete(origin_file)

 if not @current_ftp.list(origin_file).blank?

  log_info("FTP delete #{origin_file}")

  @current_ftp.delete(origin_file)

 end

end

 # ftp 是否关闭

1

2

3

4

5

6

7

8

9

def closed?

 @current_ftp.closed?

end

class << self

    # 文件编码转换

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

def convert(src_file, dest_file, from_encode, to_encode )

  log_info("Convert #{src_file} to #{dest_file}")

  cd = Iconv.new(to_encode, from_encode)

  File.open(dest_file, "w") do |out|

   File.open(src_file) do |in_stream|

    in_stream.each_line do |line|

     begin

      new_line = cd.iconv(line)

      out.write(new_line)

     rescue => e

      log_info "convert line error : #{line}"

      next

     end

    end

   end

  end

  cd.close

  dest_file

 end

end

protected

  #生成ftp

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

def create_ftp

 require "net/ftp"

 ftp = Net::FTP.new

 ftp.connect(ftp_host, ftp_port)

 ftp.login(ftp_user, ftp_pwd)

 ftp.passive = ftp_mode

 ftp

end

  #本地路径

1

2

3

4

5

6

7

8

9

10

11

def local_file(file)

  local = File.join("tmp/", file)

  FileUtils.makedirs(File.dirname(local))

  local

 end

  # 删除本地文件 

1

2

3

4

5

6

7

8

9

10

11

12

13

def delete_local_file(file)

 if File.exist?(file)

  log_info("delete local file : " + file)

  File.delete(file)

 end

end

  # 重命名本地文件

1

2

3

4

5

6

7

8

9

10

11

12

13

def rename_local_file(origin_file, file)

 if File.exist?(origin_file)

  log_info("rename local file : " + origin_file + " to " + file)

  File.rename(origin_file, file)

 end

end

  #初始化参数

1

2

3

4

5

6

7

8

9

10

11

def ftp_host; "x.x.x.x" end

 def ftp_port; "21" end

 def ftp_user; "x" end

 def ftp_pwd ; "x" end

 def ftp_mode; true end

end

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

jh035512

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值