codis集群搭建

2019独角兽企业重金招聘Python工程师标准>>>

github文档https://github.com/CodisLabs/codis/blob/master/doc/tutorial_zh.md

####环境

codis-proxy+codis-config：

192.168.10.93 proxy_1

192.168.10.94 proxy_2

192.168.10.95 proxy_3

zookeeper cluster： 192.168.10.91-93:2181

codis-server cluster 1： 192.168.10.91-92:6379

codis-server cluster 2： 192.168.10.91-92:6380

codis-server cluster 3: 192.168.10.93:6379 192.168.10.93:6380

####zookeeper集群搭建 请见http://my.oschina.net/u/1791060/blog/654107

####安装go # yum install go

# vi /etc/profile

export GOPATH=/usr

# source /etc/profile

####下载编译codis

# yum install git -y

# go get -u -d github.com/CodisLabs/codis

# cd /usr/src/github.com/CodisLabs/codis/

# make

# make gotest

####配置codis # vi /usr/src/github.com/CodisLabs/codis/config.ini

coordinator=zookeeper
zk=192.168.10.91:2181,192.168.10.92:2181,192.168.10.93:2181
product=test
dashboard_addr=192.168.10.95:18087
password=
backend_ping_period=5
session_max_timeout=1800
session_max_bufsize=131072
session_max_pipeline=1024
zk_session_timeout=30000
proxy_id=proxy_3       #每台proxy和config都不一样

####启动codis

1. 在192.168.10.95上启动dashboard

# nohup /usr/src/github.com/CodisLabs/codis/bin/codis-config -c /usr/src/github.com/CodisLabs/codis/config.ini dashboard & #nohup启动dashboard

2016/04/07 21:58:03 dashboard.go:160: [INFO] dashboard listening on addr: :18087
2016/04/07 21:58:03 dashboard.go:143: [INFO] dashboard node created: /zk/codis/db_test/dashboard, {"addr": "192.168.10.95:18087", "pid": 13708}
2016/04/07 21:58:03 dashboard.go:144: [WARN] ********** Attention **********
2016/04/07 21:58:03 dashboard.go:145: [WARN] You should use `kill {pid}` rather than `kill -9 {pid}` to stop me,
2016/04/07 21:58:03 dashboard.go:146: [WARN] or the node resisted on zk will not be cleaned when I'm quiting and you must remove it manually
......

查看http://192.168.10.95:18087/admin/是否可正常打开

2.初始化slot信息 # /usr/src/github.com/CodisLabs/codis/bin/codis-config -c /usr/src/github.com/CodisLabs/codis/config.ini slot init

{
  "msg": "OK",
  "ret": 0
}

3.启动3台codis-server #和redis-server启动一样 # vi /etc/codis.conf #配置文件如下，自己改

daemonize yes
pidfile "/tmp/codis.pid"
port 6379
timeout 300
loglevel debug
logfile "/tmp/codis.log"
databases 16
slave-read-only no
save 900 5
save 300 50
save 60 50000
rdbcompression yes
dbfilename "dump.rdb"
dir "/tmp/"
appendonly no
appendfsync always
maxmemory 1gb

# vi /etc/codis2.conf #配置文件如下，自己改

daemonize yes
pidfile "/tmp/codis2.pid"
port 6380
timeout 300
loglevel debug
logfile "/tmp/codis2.log"
databases 16
slave-read-only no
save 900 5
save 300 50
save 60 50000
rdbcompression yes
dbfilename "dump2.rdb"
dir "/tmp/"
appendonly no
appendfsync always
maxmemory 1gb

# /usr/src/github.com/CodisLabs/codis/bin/codis-server /etc/codis.conf #启动codis-server

# /usr/src/github.com/CodisLabs/codis/bin/codis-server /etc/codis2.conf

####添加server group,id为最小1开始的证书，或者在dashboard上添加 # /usr/src/github.com/CodisLabs/codis/bin/codis-config -c /usr/src/github.com/CodisLabs/codis/config.ini server -h

usage:
        codis-config server list
        codis-config server add <group_id> <redis_addr> <role>
        codis-config server remove <group_id> <redis_addr>
        codis-config server promote <group_id> <redis_addr>
        codis-config server add-group <group_id>
        codis-config server remove-group <group_id>

• 添加group1，192.168.10.91-92:6379 原来想把三台 7379的codis都加进group 1，但是发现始终只有1台是offline的就算了 # /usr/src/github.com/CodisLabs/codis/bin/codis-config -c

/usr/src/github.com/CodisLabs/codis/config.ini server add 1 192.168.10.91:6379 master { "msg": "OK", "ret": 0 }

# /usr/src/github.com/CodisLabs/codis/bin/codis-config -c

/usr/src/github.com/CodisLabs/codis/config.ini server add 1 192.168.10.92:6379 slave { "msg": "OK", "ret": 0 }

-添加group2，192.168.10.91-92:6380 略，用命令行添加吧，第一次用dashboard添加的时候dashboard直接崩溃了 结果如下：

-设置slot id 默认slot为0-1023，共1024个slot,设置group1为0-511,group2 512-1023

# /usr/src/github.com/CodisLabs/codis/bin/codis-config -c /usr/src/github.com/CodisLabs/codis/config.ini slot -h

usage:
        codis-config slot init [-f]
        codis-config slot info <slot_id>
        codis-config slot set <slot_id> <group_id> <status>
        codis-config slot range-set <slot_from> <slot_to> <group_id> <status>
        codis-config slot migrate <slot_from> <slot_to> <group_id> [--delay=<delay_time_in_ms>]
        codis-config slot rebalance [--delay=<delay_time_in_ms>]

# /usr/src/github.com/CodisLabs/codis/bin/codis-config -c /usr/src/github.com/CodisLabs/codis/config.ini slot range-set 0 511 1 online

{
  "msg": "OK",
  "ret": 0
}

# /usr/src/github.com/CodisLabs/codis/bin/codis-config -c /usr/src/github.com/CodisLabs/codis/config.ini slot range-set 512 1023 2 online

{
  "msg": "OK",
  "ret": 0
}

####启动codis-proxy # /usr/src/github.com/CodisLabs/codis/bin/codis-proxy -c /usr/src/github.com/CodisLabs/codis/config.ini -L /tmp/codis-proxy.log --cpu=2 --addr=0.0.0.0:19000 --http-addr=0.0.0.0:11000

  _____  ____    ____/ /  (_)  _____
 / ___/ / __ \  / __  /  / /  / ___/
/ /__  / /_/ / / /_/ /  / /  (__  )
\___/  \____/  \__,_/  /_/  /____/

####使用codis-proxy # /data/app/redis/bin/redis-cli -h 192.168.10.93 -p 19000

192.168.10.93:19000> 
192.168.10.93:19000> set m 1
OK
192.168.10.93:19000> get m
"1"

####遇到的坑

• dashboard死掉用起不来 问了度娘，要在zookeeper里执行rmr /zk/codis 实际上会把所有信息干掉，只需要rmr /zk/codis/db_test/dashboard

• 加错codis group id 不小心把group2的机器加到12里去了，删除竟然报错，我去 还得在zookeeper里删除rmr /zk/codis/db_test/servers/group_3

• 不小心点错，把slot0的数据迁移到group0上，实际上不可能有group0，理所当然报错了，后面的任务就一直pending 解决方法就是去zookeeper里找到对应的task删除,后面的任务就能运行了

[zk: localhost:2181(CONNECTED) 2] get /zk/codis/db_test/migrate_tasks/0000000000
{"slot_id":0,"new_group":0,"delay":0,"create_at":"1460043974","percent":0,"status":"error"}
[zk: localhost:2181(CONNECTED) 3] rm /zk/codis/db_test/migrate_tasks/0000000000 

转载于:https://my.oschina.net/u/1791060/blog/654597