搭建带ACL控制的Consul集群

1.设备

机器ip(机器名)	http端口	Agent类型	节点名称
192.168.43.120	8500	server	consul-server1
192.168.43.121	8500	server	consul-server2
192.168.43.122	8500	client带UI	consul-client1

2.配置文件

文件放入位置：/usr/bin/start-conf/
server1：把其他节点加入集群

{
    "datacenter":"dc1",
    "primary_datacenter":"dc1",
    "bootstrap_expect":1,
    "start_join":[
        "192.168.43.121"
    ],
    "retry_join":[
        "192.168.43.121"
    ],
    "advertise_addr": "192.168.43.120",
    "bind_addr": "192.168.43.120",
    "server":true,
    "connect":{
        "enabled":true
    },
    "node_name":"consul-server1",
    "data_dir":"/opt/consul/data/",
    "enable_script_checks":false,
    "enable_local_script_checks":true,
    "log_file":"/opt/consul/log/",
    "log_level":"info",
    "log_rotate_bytes":100000000,
    "log_rotate_duration":"24h",
    "encrypt":"krCysDJnrQ8dtA7AbJav8g==",
    "acl":{
        "enabled":true,
        "default_policy":"deny",
        "enable_token_persistence":true,
        "tokens":{
            "master":"cd76a0f7-5535-40cc-8696-073462acc6c7"
           }
    }
}

server2

{
    "datacenter":"dc1",
    "primary_datacenter":"dc1",
    "advertise_addr":  "192.168.43.121",
    "bind_addr": "192.168.43.121",
    "server":true,
    "connect":{
        "enabled":true
    },
    "node_name":"consul-server2",
    "data_dir":"/opt/consul/data/",
    "enable_script_checks":false,
    "enable_local_script_checks":true,
    "log_file":"/opt/consul/log/",
    "log_level":"info",
    "log_rotate_bytes":100000000,
    "log_rotate_duration":"24h",
    "encrypt":"krCysDJnrQ8dtA7AbJav8g==",
    "acl":{
        "enabled":true,
        "default_policy":"deny",
        "enable_token_persistence":true,
        "tokens":{
            "master":"cd76a0f7-5535-40cc-8696-073462acc6c7"
        }
    }
}

client

{
    "datacenter":"dc1",
    "primary_datacenter":"dc1",
    "advertise_addr": "192.168.43.122",
    "start_join":[
      "192.168.43.120",
      "192.168.43.121"
    ],
    "retry_join":[
      "192.168.43.120",
      "192.168.43.121"
    ],
    "bind_addr":"192.168.43.122",
    "node_name":"consul-client1",
    "client_addr":"0.0.0.0",
    "connect":{
        "enabled":true
    },
    "data_dir":"/opt/consul/data/",
    "log_file":"/opt/consul/log/",
    "log_level":"info",
    "log_rotate_bytes":100000000,
    "log_rotate_duration":"24h",
    "encrypt":"krCysDJnrQ8dtA7AbJav8g==",
    "ui":true,
    "enable_script_checks":false,
    "enable_local_script_checks":true,
    "disable_remote_exec":true,
    "ports":{
        "http":8500
    },
    "acl":{
        "enabled":true,
        "default_policy":"deny",
        "enable_token_persistence":true,
        "tokens":{
        "agent":"08936ed9-043f-9a53-1a26-9f9d43f18786"
        }
    }
}

启动：
1.iptables规则：
iptables -I INPUT -p udp --dport 8301 -j ACCEPT
iptables -I OUTPUT -p udp --dport 8301 -j ACCEPT
iptables -I INPUT -p tcp --dport 8301 -j ACCEPT
iptables -I OUTPUT -p tcp --dport 8301 -j ACCEPT

iptables -I INPUT -p udp --dport 8300 -j ACCEPT
iptables -I OUTPUT -p udp --dport 8300 -j ACCEPT
iptables -I INPUT -p tcp --dport 8300 -j ACCEPT
iptables -I OUTPUT -p tcp --dport 8300 -j ACCEPT

iptables -I INPUT -p udp --dport 8500 -j ACCEPT
iptables -I OUTPUT -p udp --dport 8500 -j ACCEPT
iptables -I INPUT -p tcp --dport 8500 -j ACCEPT
iptables -I OUTPUT -p tcp --dport 8500 -j ACCEPT
2.所有机器分别在/usr/bin目录下操作：
以server1为例：./consul agent -config-file start-conf/consul-server1.json

3.生成并配置agent-token，解决server agent ACL block问题

当上面的语句执行完之后，会发现协调更新由于ACL被阻塞。如下图：

经过查看官方文档，发现是由于未生成和配置agent-token导致。

在任意一台server上执行下面的语句来生成agent-token：

curl \
    --request PUT \
    --header "X-Consul-Token: cd76a0f7-5535-40cc-8696-073462acc6c7" \
    --data \
'{
  "Name": "Agent Token",
  "Type": "client",
  "Rules": "node \"\" { policy = \"write\" } service \"\" { policy = \"read\" }"
}' http://127.0.0.1:8500/v1/acl/create

此时会返回生成的agent-token

将生成的agent_token设置到每个server agent的配置文件中。
此时consul-server1.json, consul-server2.json, consul-server3.json中acl部分就变为：

"acl":{
        "enabled":true,
        "default_policy":"deny",
        "enable_token_persistence":true,
        "tokens":{
            "master":"cd76a0f7-5535-40cc-8696-073462acc6c7",
             "agent":"deaa315d-98c5-b9f6-6519-4c8f6574a551"
        }
    }

也就是多了agent这个配置。

接着依次重启各个server agent（把之前的进程先停掉）

4.配置环境变量

上面操作都执行完后，执行./consul members可能会没有成员，此时则需要配置环境变量
1.给三个server的环境变量添加CONSUL_HTTP_TOKEN, vim /etc/profile添加下面一句

export CONSUL_HTTP_TOKEN=cd76a0f7-5535-40cc-8696-073462acc6c7

然后，source /etc/profile一下。
为了简单方便，我这里配了最大的权限即master_token
此时发现./consul members已经有数据了

2.给client agent 设置环境变量
由于client agent 带web-ui，这里你的公司不一定对外开放8500端口，可以把它改成7110，方便在外网查看。
不过此时需要添加一个环境变量CONSUL_HTTP_ADDR，来告诉命令行不是使用默认的127.0.0.1:8500
更改client-agent的环境变量,在最后添加下面两行

#consul http-token
export CONSUL_HTTP_TOKEN=cd76a0f7-5535-40cc-8696-073462acc6c7
#only consul-client1 need, because http port has changed to 7110
export CONSUL_HTTP_ADDR=127.0.0.1:7110

此时发现在client agent上执行./consul members也是ok的。

5.给web-ui 设置master_token

浏览器上输入client的ip：8500，点击ACL， 输入master-token即可