docker启用calico网络

环境

说明：在openstack启动的docker虚拟机上启动calico未成功。本文的docker主机是在VMware Workstation上启动的。

操作系统

cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core) 

内核版本

uname -a
Linux docker01 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

docker版本

docker version
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:03:45 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:02:21 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

calico-node版本

docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             latest              f0b02e9d092d        43 hours ago        1.23MB
calico/node         v1.0.2              ff8c7b8fd9dd        3 years ago         257MB

calicoctl版本

calicoctl version
Version:      v1.0.2
Build date:   2017-01-31T04:05:47+0000
Git commit:   7fe3468

etcd版本

etcdctl -v
etcdctl version: 3.3.11
API version: 2

etcd节点

192.168.56.3

docker节点

192.168.56.3
192.168.56.4

安装及配置etcd

安装etcd

yum -y install etcd

配置etcd

cat /etc/etcd/etcd.conf
#更改两项配置，其他的保持默认
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"

改docker配置

所有节点都配置

cat /usr/lib/systemd/system/docker.service
将#ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock改为
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock -H unix:// --cluster-store=etcd://192.168.56.3:2379

重载docker配置

systemctl daemon-reload
systemctl restart docker

配置calico

下载calicoctl

所有节点都配置

wget -O /usr/local/bin/calicoctl https://github.com/projectcalico/calicoctl/releases/download/v1.0.2/calicoctl
cp calicoctl /usr/sbin/
chmod +x /usr/sbin/calicoctl 

已经保存到百度网盘。

配置主机环境变量

默认会去找127.0.0.1:2379，所以这里一定要配置etcd的环境变量。
所有节点都配置

export ETCD_AUTHORITY=192.168.56.3:2379
export ETCD_ENDPOINTS=http://192.168.56.3:2379

echo "export ETCD_AUTHORITY=192.168.56.3:2379" >> /etc/profile
echo "export ETCD_ENDPOINTS=http://192.168.56.3:2379" >> /etc/profile

source /etc/profile

启动calico-node

calicoctl node run

创建calico网络

任意docker节点操作

docker network create --driver calico --ipam-driver calico-ipam cal_net1

节点1启动容器

docker run --net cal_net1 --name calico1 -itd busybox

节点2启动容器

docker run --net cal_net1 --name calico2 -itd busybox

验证网络

[root@docker02 etcd]# docker exec -it calico2 sh
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
20: cali0@if21: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff
    inet 192.168.180.2/32 brd 192.168.180.2 scope global cali0
       valid_lft forever preferred_lft forever
/ # 

[root@docker01 ~]# docker exec -it calico1 sh
/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
12: cali0@if13: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue 
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff
    inet 192.168.159.193/32 brd 192.168.159.193 scope global cali0
       valid_lft forever preferred_lft forever
/ # ping 192.168.180.2
PING 192.168.180.2 (192.168.180.2): 56 data bytes
64 bytes from 192.168.180.2: seq=0 ttl=62 time=3.134 ms
64 bytes from 192.168.180.2: seq=1 ttl=62 time=0.617 ms
64 bytes from 192.168.180.2: seq=2 ttl=62 time=0.211 ms
64 bytes from 192.168.180.2: seq=3 ttl=62 time=3.503 ms
64 bytes from 192.168.180.2: seq=4 ttl=62 time=0.335 ms
64 bytes from 192.168.180.2: seq=5 ttl=62 time=0.327 ms

calicoctl使用

查看calico的资源

[root@docker02 etcd]# calicoctl get profile -o yaml
- apiVersion: v1
  kind: profile
  metadata:
    name: net1
    tags:
    - net1
  spec:
    egress:
    - action: allow
      destination: {}
      source: {}
    ingress:
    - action: allow
      destination: {}
      source:
        tag: net1

[root@docker02 etcd]# calicoctl get ipPools -o yaml
- apiVersion: v1
  kind: ipPool
  metadata:
    cidr: 192.168.0.0/16
  spec: {}
- apiVersion: v1
  kind: ipPool
  metadata:
    cidr: fd80:24e2:f998:72d6::/64
  spec: {}

[root@docker02 etcd]# calicoctl get node -o yaml
- apiVersion: v1
  kind: node
  metadata:
    name: docker01
  spec:
    bgp:
      ipv4Address: 192.168.56.3
- apiVersion: v1
  kind: node
  metadata:
    name: docker02
  spec:
    bgp:
      ipv4Address: 192.168.56.4

[root@docker02 etcd]# calicoctl get profile -o yaml
- apiVersion: v1
  kind: profile
  metadata:
    name: net1
    tags:
    - net1
  spec:
    egress:
    - action: allow
      destination: {}
      source: {}
    ingress:
    - action: allow
      destination: {}
      source:
        tag: net1