一、背景
Consul 是一个支持多数据中心分布式高可用的服务发现和配置共享的服务软件,由 HashiCorp 公司用 Go 语言开发,基于 Mozilla Public License 2.0 的协议进行开源。Consul 支持健康检查,并允许 HTTP 和 DNS 协议调用 API 存储键值对。
Consul 的一致性协议采用 Raft 算法用来保证服务的高可用。使用 GOSSIP 协议管理成员和广播消息,并且支持 ACL 访问控制。
官方原理图
Consul节点间通过gossip协议进行访问。
单个datacenter内的通信端口为8301(TCP and UDP)。跨datacenter的通信端口为8302(TCP and UDP)
二、部署Consul Server节点
2.1、安装go
# wget -c https://dl.google.com/go/go1.12.5.linux-amd64.tar.gz
# tar zxvf go1.12.5.linux-amd64.tar.gz -C /usr/local/
# echo "export PATH=$PATH:/usr/local/go/bin" >> /etc/profile
# source /etc/profile
# go version
输出:go version go1.12.5 linux/amd64,表示安装成功
2.2、配置Consul Cluster
如下以3台consul作为一组Cluster:
192.168.47.34
192.168.47.35
192.168.47.36
每台机器如下操作:
yum -y install unzip
wget -c https://releases.hashicorp.com/consul/1.5.1/consul_1.5.1_linux_amd64.zip
unzip consul_1.5.1_linux_amd64.zip
mv consul /usr/local/bin/
创建目录:/etc/consul.d 和 /data/consul
vim /etc/consul.d/server.json
{
"data_dir": "/data/consul",
"datacenter": "dc1",
"log_level": "WARN",
"server":true,
"bind_addr":"192.168.47.34",
"client_addr":"192.168.47.34",
"ports":{
"dns":53
},
"ui":true,
"retry_join":["192.168.47.35","192.168.47.36"],
"retry_interval":"3s",
"raft_protocol":3,
"rejoin_after_leave":true
}
注意配置文件中bind_addr/client_addr、retry_join按实际IP修改
启动:consul agent --config-dir=/etc/consul.d/ --bootstrap-expect=1 > /data/consul/consul.log 2>&1 &
第一节点启动时额外添加参数:--bootstrap-expect=1
bootstrap-expect:在一个datacenter中期望的server节点数,直到满足该数值,整个集群才会被引导
consul端口如下
# netstat -nlp |grep consul
tcp 0 0 192.168.47.34:8300 0.0.0.0:* LISTEN 1896/consul
tcp 0 0 192.168.47.34:8301 0.0.0.0:* LISTEN 1896/consul
tcp 0 0 192.168.47.34:8302 0.0.0.0:* LISTEN 1896/consul
tcp 0 0 192.168.47.34:8500 0.0.0.0:* LISTEN 1896/consul
tcp 0 0 192.168.47.34:53 0.0.0.0:* LISTEN 1896/consul
udp 0 0 192.168.47.34:8301 0.0.0.0:* 1896/consul
udp 0 0 192.168.47.34:8302 0.0.0.0:* 1896/consul
udp 0 0 192.168.47.34:53 0.0.0.0:* 1896/consul
开放端口:
dns - The DNS server, -1 to disable. Default 8600.
http - The HTTP API, -1 to disable. Default 8500
https - The HTTPS API, -1 to disable. Default -1 (disabled)
rpc - The CLI RPC endpoint. Default 8400
serf_lan - The Serf LAN port. Default 8301
serf_wan - The Serf WAN port. Default 8302
server - Server RPC address. Default 8300
查看节点运行状态:
# consul members --http-addr=http://192.168.47.34:8500
Node Address Status Type Build Protocol DC Segment
47_34.localdomain 192.168.47.34:8301 alive server 1.5.1 2 dc1 <all>
47_35.localdomain 192.168.47.35:8301 alive server 1.5.1 2 dc1 <all>
47_36.localdomain 192.168.47.36:8301 alive server 1.5.1 2 dc1 <all>
# consul operator raft list-peers --http-addr=http://192.168.47.34:8500
Node ID Address State Voter RaftProtocol
node4734 f67e0dfa-3b91-1019-416d-f5af6105afcb 192.168.47.34:8300 follower true 3
node4735 77535023-504c-9329-14ec-915f04886409 192.168.47.35:8300 leader true 3
node4736 e6dd76ea-59ed-331d-8092-e729af13e0cb 192.168.47.36:8300 follower true 3
同样可以通过 http://192.168.47.34:8500 访问Web页面
三、注册服务
以ProxySQL为例,架构图如下:
ProxySQL1,位于 192.168.47.33
ProxySQL2,位于 192.168.47.43
在这两台服务器上同样需要安装consul,并配置如下(以47.33为例):
client.json:
{
"data_dir": "/data/consul",
"enable_script_checks": true,
"server":false,
"bind_addr":"192.168.47.33",
"retry_join":["192.168.47.34","192.168.47.35","192.168.47.36"],
"retry_interval":"30s",
"rejoin_after_leave":true,
"start_join":["192.168.47.34","192.168.47.35","192.168.47.36"]
}
proxysql.json:
{
"service": {
"checks": [
{
"id": "proxysql1",
"interval": "5s",
"args": [
"mysqladmin",
"ping",
"--host=127.0.0.1",
"--port=6033",
"--user=mix",
"--password=123456"
]
}
],
"address": "192.168.47.33",
"name": "db3306",
"port": 6033,
"tags": [
"ProxySQL"
]
}
}
启动:consul agent --config-dir=/etc/consul.d/ > /data/consul/consul.log 2>&1 &
对外DNS服务:db3306.service.consul
使用 dig 查看服务对应IP(默认 -p 53)
[root@node4734 ~]# dig @192.168.47.34 db3306.service.consul SRV
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.2 <<>> @192.168.47.34 db3306.service.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47416
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 3
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;db3306.service.consul. IN SRV
;; ANSWER SECTION:
db3306.service.consul. 0 IN SRV 1 1 6033 47_33.node.dc1.consul.
;; ADDITIONAL SECTION:
47_33.node.dc1.consul. 0 IN A 192.168.47.33
47_33.node.dc1.consul. 0 IN TXT "consul-network-segment="
第二个ProxySQL加入后,一个DNS服务对应两个IP
[root@node4734 ~]# dig @192.168.47.34 db3306.service.consul
; <<>> DiG 9.9.4-RedHat-9.9.4-74.el7_6.2 <<>> @192.168.47.34 db3306.service.consul
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30586
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 3
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;db3306.service.consul. IN A
;; ANSWER SECTION:
db3306.service.consul. 0 IN A 192.168.47.33
db3306.service.consul. 0 IN A 192.168.47.43
参考文档:
Percona Blog:Consul, ProxySQL and MySQL HA