docker
作者：DevOps旭
来自：DevOps探路者
我们是一群软件开发者，也是一群DevOps探路者，致力于学习和打造开源DevOps工具，解决企业内部的软件开发过程中遇到的问题，如果你也和我们一样，欢迎加入我们！
一、初识编排
容器编排是通过一个引擎，使容器按照自己所期望的方式依次启动，并且可以实现包括故障自动恢复，扩容等功能。自从2013年docker引爆了容器的狂潮，容器由单机向集群演变的呼声越来越大，容器编排之争也揭开了序幕。在这一场大战中，主要有三大主流框架，docker swarm、kubernetes、mesos。
1、swarm
docker swarm是由docker公司推出的一款编排工具，由一组节点组成，节点包括有docker engine 和docker daemon，节点之间通过RAFT算法实现数据一致。
2、kubernetes
kubernetes是有google在2014年推出的一款划时代的编排产品，kubernetes是由google内部的borg系统系统开源出的一个产品，在推出之后迅速占据了市场，成为了当前容器编排的首选工具。
3、mesos
mesos是一个开源的集群管理工具，可支持多种工作的负载，同时mesos引入了zookeeper来实现集群内的数据一致性。
二、走进docker swarm
1、docker swarm架构及基本概念
docker swarm是docker原生的集群管理工具，在2014年12月首次发布，并在2016年进行了重新设计，推出了v2版本，极大扩展了集群管理的node数量。docker swarm的架构是主从结构，可以配置多个manager节点及worker节点架构图如下。
Node：是swarm集群最小的资源单位，每个节点都是一个docker宿主机。
manager node：swarm集群的管理节点，负责响应外部对集群的操作，对worker节点分发任务，建议配置为多节点以实现高可用。
worker node：负责执行管理节点任务的节点，manager node 也是worker node。
service：用于管理容器的工具，是实现容器编排的基础。
task：service管理的容器。
2、创建一个swarm 集群 docker swarm init
创建swarm是十分简单的，可以通过docker swarm init 来实现，官方文档对于docker init 也提出了一些扩展
Name, shorthandDefaultDescription--advertise-addr指定服务监听地址--autolock自动锁定管理服务的启停--availabilityactive节点可用性--cert-expiry2160h0m0s根证书过期时长--data-path-addr数据流量的接口地址--data-path-port数据流量的端口--default-addr-pool默认地址池--default-addr-pool-mask-length24默认地址池子网掩码长度--dispatcher-heartbeat5s心跳时长--external-ca指定外部ca证书--force-new-cluster强制创建新集群--listen-addr0.0.0.0:2377监听地址--max-snapshots快照数上限--snapshot-interval10000快照间隔--task-history-limit5历史任务保留个数
可以看到，创建集群时可以通过引入参数进行优化，下面我们创建一个最简单的集群，服务器配置如下
docker1 192.168.1.251 1C4Gdocker2 192.168.1.252 1C4Gdocker3 192.168.1.253 1C4G可以通过以下命令创建集群，并查看初始化集群后的信息
[root@docker1 ~]# docker swarm init --advertise-addr 192.168.1.251Swarm initialized: current node (k0hkuik9mipdtn2rkukra6nkr) is now a manager.To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-4sspbo9zthueo375uv6so2h8xu9r5dp7wy80u7yud2q6gwimqv-3davtwn3e8u63f9ejd7orkkar 192.168.1.251:2377To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.[root@docker1 ~]# docker info... Swarm: active NodeID: k0hkuik9mipdtn2rkukra6nkr Is Manager: true ClusterID: ca4sutdp2k0qsehbmu2ojefcf Managers: 1 Nodes: 1 Default Address Pool: 10.0.0.0/8 SubnetSize: 24 Data Path Port: 4789 Orchestration: Task History Retention Limit: 5 Raft: Snapshot Interval: 10000 Number of Old Snapshots to Retain: 0 Heartbeat Tick: 1 Election Tick: 10 Dispatcher: Heartbeat Period: 5 seconds CA Configuration: Expiry Duration: 3 months Force Rotate: 0 Autolock Managers: false Root Rotation In Progress: false Node Address: 192.168.1.251 Manager Addresses: 192.168.1.251:2377 ...3、加入集群 docker join
我们可以看一下官网关于docker join的描述:以node或manager的身份加入集群。
同时官方也提供了一些参数可以对此行为进行优化
Name, shorthandDefaultDescription--advertise-addr监听端口--availabilityactive节点可用性--data-path-addr数据地址接口--listen-addr0.0.0.0:2377监听地址--token加入集群的token
下面我们通过docker返回的token来加入集群
[root@docker2 ~]# docker swarm join --token SWMTKN-1-4sspbo9zthueo375uv6so2h8xu9r5dp7wy80u7yud2q6gwimqv-3davtwn3e8u63f9ejd7orkkar 192.168.1.251:2377This node joined a swarm as a worker.[root@docker3 ~]# docker swarm join --token SWMTKN-1-4sspbo9zthueo375uv6so2h8xu9r5dp7wy80u7yud2q6gwimqv-3davtwn3e8u63f9ejd7orkkar 192.168.1.251:2377This node joined a swarm as a worker.[root@docker1 ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONk0hkuik9mipdtn2rkukra6nkr * docker1 Ready Active Leader 19.03.12ltb6h1yeptq7qf6pirff8zjk2 docker2 Ready Active 19.03.12r3qo3s05knckmx2keji6x3e6y docker3 Ready Active 19.03.12此时可以在manager节点上看到集群的node信息：一个manager两个node。
而此时，我们可以通过docker node promote 将节点提升为manager，也可以通过docker node demote 将manager节点降为node。
[root@docker1 ~]# docker node promote ltb6h1yeptq7qf6pirff8zjk2Node ltb6h1yeptq7qf6pirff8zjk2 promoted to a manager in the swarm.[root@docker1 ~]# docker node promote r3qo3s05knckmx2keji6x3e6yNode r3qo3s05knckmx2keji6x3e6y promoted to a manager in the swarm.[root@docker1 ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONk0hkuik9mipdtn2rkukra6nkr * docker1 Ready Active Leader 19.03.12ltb6h1yeptq7qf6pirff8zjk2 docker2 Ready Active Reachable 19.03.12r3qo3s05knckmx2keji6x3e6y docker3 Ready Active Reachable 19.03.12 [root@docker1 ~]# docker node demote k0hkuik9mipdtn2rkukra6nkr Manager k0hkuik9mipdtn2rkukra6nkr demoted in the swarm.[root@docker1 ~]# docker node ls Error response from daemon: This node is not a swarm manager. Worker nodes can't be used to view or modify cluster state. Please run this command on a manager node or promote the current node to a manager.[root@docker2 ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONk0hkuik9mipdtn2rkukra6nkr docker1 Ready Active 19.03.12ltb6h1yeptq7qf6pirff8zjk2 * docker2 Ready Active Leader 19.03.12r3qo3s05knckmx2keji6x3e6y docker3 Ready Active Reachable 19.03.12通过以上命令演示了节点的管理过程，需要注意的是，当节点出现异常需要移除时，需要先将节点降为node才可以进行移除。
# 驱逐docker1上的容器[root@docker3 ~]# docker node update docker1 --availability drain[root@docker3 ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONk0hkuik9mipdtn2rkukra6nkr docker1 Ready Active Reachable 19.03.12ltb6h1yeptq7qf6pirff8zjk2 docker2 Ready Active Leader 19.03.12r3qo3s05knckmx2keji6x3e6y * docker3 Ready Active Reachable 19.03.12# 将docker1降为worker节点[root@docker3 ~]# docker node demote k0hkuik9mipdtn2rkukra6nkr Manager k0hkuik9mipdtn2rkukra6nkr demoted in the swarm.[root@docker3 ~]# docker node lsID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONk0hkuik9mipdtn2rkukra6nkr docker1 Down Active 19.03.12ltb6h1yeptq7qf6pirff8zjk2 docker2 Ready Active Leader 19.03.12r3qo3s05knckmx2keji6x3e6y * docker3 Ready Active Reachable 19.03.12# 在docker1上执行，强制退出集群[root@docker1 ~]# docker swarm leave --forceNode left the swarm.[root@docker3 ~]# docker node lsID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONk0hkuik9mipdtn2rkukra6nkr docker1 Down Active 19.03.12ltb6h1yeptq7qf6pirff8zjk2 docker2 Ready Active Leader 19.03.12r3qo3s05knckmx2keji6x3e6y * docker3 Ready Active Reachable 19.03.12# 在docker3上删除节点docker1，至此节点彻底在集群中移除[root@docker3 ~]# docker node rm k0hkuik9mipdtn2rkukra6nkr k0hkuik9mipdtn2rkukra6nkr[root@docker3 ~]# docker node lsID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONltb6h1yeptq7qf6pirff8zjk2 docker2 Ready Active Leader 19.03.12r3qo3s05knckmx2keji6x3e6y * docker3 Ready Active Reachable 19.03.12那么如何将节点再次加入集群中呢
# 在任一manager节点执行[root@docker3 ~]# docker swarm join-token managerTo add a manager to this swarm, run the following command: docker swarm join --token SWMTKN-1-4sspbo9zthueo375uv6so2h8xu9r5dp7wy80u7yud2q6gwimqv-dydshw90qt5jyafa1qe60rkdf 192.168.1.253:2377# 在docker1节点执行[root@docker1 ~]# docker swarm join --token SWMTKN-1-4sspbo9zthueo375uv6so2h8xu9r5dp7wy80u7yud2q6gwimqv-dydshw90qt5jyafa1qe60rkdf 192.168.1.253:2377This node joined a swarm as a manager.[root@docker1 ~]# docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSIONz0t7tggoh6vhhkyipjgltynp3 * docker1 Ready Active Reachable 19.03.12ltb6h1yeptq7qf6pirff8zjk2 docker2 Ready Active Leader 19.03.12r3qo3s05knckmx2keji6x3e6y docker3 Ready Active Reachable 19.03.12至此完成节点再次回归集群
4、创建服务service
到此为止我们已经拥有了一个swarm集群了，而docker 集群的存在便是为了实现服务的编排，那么docker是如何实现的服务编排的呢？
[root@docker1 ~]# docker service create > --name nginx > nginxfosdoid3fznf2xlvcy6bbbjesoverall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged 我们创建了一个nginx的service，可以通过docker service ls 来查看服务的列表
[root@docker1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTSfosdoid3fznf nginx replicated 1/1 nginx:latest 还可以通过docker service ps nginx 来查看节点分布
[root@docker1 ~]# docker service ps nginxID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTSm3ctwxv24q6t nginx.1 nginx:latest docker3 Running Running 3 hours ago 同时swarm 也提供了 docker service inspect 来查看service详情
[root@docker1 ~]# docker service inspect nginx[ { "ID": "fosdoid3fznf2xlvcy6bbbjes", "Version": { "Index": 48 }, "CreatedAt": "2020-09-12T12:41:42.897033699Z", "UpdatedAt": "2020-09-12T12:41:42.897033699Z", "Spec": { "Name": "nginx", "Labels": {}, "TaskTemplate": { "ContainerSpec": { "Image": "nginx:latest@sha256:9a1f8ed9e2273e8b3bbcd2e200024adac624c2e5c9b1d420988809f5c0c41a5e", "Init": false, "StopGracePeriod": 10000000000, "DNSConfig": {}, "Isolation": "default" }, "Resources": { "Limits": {}, "Reservations": {} }, "RestartPolicy": { "Condition": "any", "Delay": 5000000000, "MaxAttempts": 0 }, "Placement": { "Platforms": [ { "Architecture": "amd64", "OS": "linux" }, { "OS": "linux" }, { "OS": "linux" }, { "Architecture": "arm64", "OS": "linux" }, { "Architecture": "386", "OS": "linux" }, { "Architecture": "mips64le", "OS": "linux" }, { "Architecture": "ppc64le", "OS": "linux" }, { "Architecture": "s390x", "OS": "linux" } ] }, "ForceUpdate": 0, "Runtime": "container" }, "Mode": { "Replicated": { "Replicas": 1 } }, "UpdateConfig": { "Parallelism": 1, "FailureAction": "pause", "Monitor": 5000000000, "MaxFailureRatio": 0, "Order": "stop-first" }, "RollbackConfig": { "Parallelism": 1, "FailureAction": "pause", "Monitor": 5000000000, "MaxFailureRatio": 0, "Order": "stop-first" }, "EndpointSpec": { "Mode": "vip" } }, "Endpoint": { "Spec": {} } }]为了实现对service更为细致化的管理，docker 在创建service时也提供大量参数
官方提供了大量参数，根据实际需求使用即可。
5、调节service的副本数
在docker swarm中可以在创建service时，通过设置--replicas 来设置副本数，但是当服务运行起来后应该如何处理呢？docker 提供了scale命令来实现这个功能
[root@docker1 ~]# docker service scale nginx=3nginx scaled to 3overall progress: 3 out of 3 tasks 1/3: running [==================================================>] 2/3: running [==================================================>] 3/3: running [==================================================>] verify: Service converged [root@docker1 ~]# docker service lsID NAME MODE REPLICAS IMAGE PORTS1sh4fh4g2bl9 alpine replicated 1/1 alpine:test 8z67urwzg5ii my_nginx replicated 1/1 nginx:latest fosdoid3fznf nginx replicated 3/3 nginx:latest [root@docker1 ~]# docker service ps nginxID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTSm3ctwxv24q6t nginx.1 nginx:latest docker3 Running Running 3 hours ago xgcybi1gs0hp nginx.2 nginx:latest docker2 Running Running 17 seconds ago qiloqf3699p4 nginx.3 nginx:latest docker1 Running Running about a minute ago [root@docker1 ~]# 可以看到在执行完docker service scale命令后，nginx的副本数变成了3，现在我们尝试删除一个容器
[root@docker1 ~]# docker rm -f c11f0464fab1c11f0464fab1[root@docker1 ~]# docker service ps nginxID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTSm3ctwxv24q6t nginx.1 nginx:latest docker3 Running Running 3 hours ago xgcybi1gs0hp nginx.2 nginx:latest docker2 Running Running 3 minutes ago adcbqyf6veje nginx.3 nginx:latest docker1 Running Running about a minute ago qiloqf3699p4 _ nginx.3 nginx:latest docker1 Shutdown Failed about a minute ago "task: non-zero exit (137)" [root@docker1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTS1sh4fh4g2bl9 alpine replicated 1/1 alpine:test 8z67urwzg5ii my_nginx replicated 1/1 nginx:latest fosdoid3fznf nginx replicated 3/3 nginx:latest 可以发现，nginx的副本数还是3，说明容器进行了重建，证明了swarm集群强大的恢复能力。
6、更新数据 docker service update
docker service 在创建时实可以配置很多参数来对service进行限制，包括内存，cpu等等，下面我们创建一个service
[root@docker1 ~]# docker service create > --limit-memory=256M > --limit-cpu=1 > --name nginx > nginx i01awa3cmk7l2k20pjvd3skv7overall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged [root@docker1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTSi01awa3cmk7l nginx replicated 1/1 nginx:latest [root@docker1 ~]# docker service ps nginxID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTSn4cxba7xjexa nginx.1 nginx:latest docker1 Running Running 25 seconds ago [root@docker1 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES92b6bde16c02 nginx:latest "/docker-entrypoint.…" 32 seconds ago Up 31 seconds 80/tcp nginx.1.n4cxba7xjexak9gd03kcvimvm[root@docker1 ~]# docker stats 92b6bde16c02 CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS92b6bde16c02 nginx.1.n4cxba7xjexak9gd03kcvimvm 0.00% 2.836MiB / 256MiB 1.11% 828B / 0B 0B / 0B 2可以看到，这个镜像的资源被全面的限制住了，下面我们来通过docker service update 来调整service的内存和cpu
[root@docker1 ~]# docker service update nginx --limit-memory=512Mnginxoverall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged [root@docker1 ~]# docker service update nginx --limit-cpu=0.5nginxoverall progress: 1 out of 1 tasks 1/1: running [==================================================>] verify: Service converged [root@docker1 ~]# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMESae2b13492b5b nginx:latest "/docker-entrypoint.…" 9 seconds ago Up 8 seconds 80/tcp nginx.1.8rk7l4kjplgf3zl22qvjrcb2w[root@docker1 ~]# docker stats ae2b13492b5b CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDSae2b13492b5b nginx.1.8rk7l4kjplgf3zl22qvjrcb2w 0.00% 2.312MiB / 512MiB 0.45% 828B / 0B 0B / 0B 2可以看到限制被全面更新。同时，update最为关键的是可以用于滚动更新上面，用于实现服务的灰度发布，下面是通过nginx 模拟的灰度发布。
[root@docker1 ~]# docker service update my_nginx > --update-failure-action=rollback > --update-parallelism 1 > --update-delay 30s > --image nginx:1.12.1my_nginxoverall progress: 6 out of 6 tasks 1/6: running [==================================================>] 2/6: running [==================================================>] 3/6: running [==================================================>] 4/6: running [==================================================>] 5/6: running [==================================================>] 6/6: running [==================================================>] verify: Service converged [root@docker1 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTSr75ld2lnp4b2 my_nginx replicated 6/6 nginx:1.12.1 在此次操作中，通过--update-failure-action来限定，滚动发布失败后的行为为回退到上一个版本，通过--update-parallelism来限定滚动更新的步长为1个副本，通过--update-delay 来限定滚动更新的间隔，需要注意的是，通过这三条限定，限定了服务滚动发布的速度，避免因服务更新而导致服务不可用，但是由于docker中没用类似于k8s的readiness的控制器，所以只可以通过控制间隔来确保服务的不中断。
三、docker stack
docker stack是为了应对大规模部署而推出的，可以提供期望状态、滚动升级、简单易用、扩缩容、健康检查等，并将这些功能封装在一个声明式模型当中。docker stack 始于 Docker Swarm来实现的，所以很多特性是来自于docker swarm。
1、docker stack deploy
用于部署或是升级一个stack。docker为docker stack也提供了丰富的选项
Name, shorthandDefaultDescription--bundle-file分布式应用绑定文件的集群路径。--compose-file , -ccompose.yml文件路径。--namespace当--orchestrator指定为kubetnetes时，使用kubernetes的namespace--prune不使用prune--resolve-imagealways是否需要查询注册表解析镜像和平台支持(“always”|”changed”|”never”)--with-registry-auth将注册表信息发送到swarm代理--kubeconfig--orchestrator指定为kubetnetes时，指定Kubernetes config--orchestrator指定编排方式(swarm|kubernetes|all)
创建一个最简单的stack
[root@docker3 ~]# cat compose.ymlversion: "3.3"services: nginx: image: nginx deploy: mode: replicated replicas: 1 restart_policy: condition: on-failure ports: - target: 80 published: 80 mode: host[root@docker3 ~]# docker stack deploy -c compose.yml service[root@docker3 ~]# docker service ls ID NAME MODE REPLICAS IMAGE PORTSr75ld2lnp4b2 my_nginx replicated 6/6 nginx:1.12.1 c933fl4jmhmd service_nginx replicated 1/1 nginx:latest2、docker stack ps
列出部署的stack。
[root@docker3 ~]# docker stack ls NAME SERVICES ORCHESTRATORservice 1 Swarm[root@docker3 ~]# docker stack ps service ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS4vv32yjkiag2 service_nginx.1 nginx:latest docker1 Running Running 2 minutes ago *:80->80/tcp同时docker stack 也提供了大量的选项来使查询结果更加丰富多变
Name, shorthandDefaultDescription--filter , -f过滤输出--format格式化输出--namespace当--orchestrator指定为kubetnetes时，使用kubernetes的namespace--no-resolve用ID展示容器--no-trunc不截断的完整的输出--quiet , -q只输出ID--kubeconfig--orchestrator指定为kubetnetes时，指定Kubernetes config--orchestrator指定编排方式 (swarm|kubernetes|all)
下面分别展示一下效果
# -f[root@docker1 ~]# docker stack ps -f "id=4" service ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS4vv32yjkiag2 service_nginx.1 nginx:latest docker1 Running Running 14 minutes ago *:80->80/tcp[root@docker1 ~]# docker stack ps -f "node=docker1" service ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS4vv32yjkiag2 service_nginx.1 nginx:latest docker1 Running Running 15 minutes ago *:80->80/tcp[root@docker1 ~]# docker stack ps -f "desired-state=running" service ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS4vv32yjkiag2 service_nginx.1 nginx:latest docker1 Running Running 15 minutes ago *:80->80/tcp# --format[root@docker1 ~]# docker stack ps --format "{{.ID}}:{{.Name}}:{{.Image}}:{{.Node}}:{{.DesiredState}}:{{.CurrentState}}:{{.Error}}:{{.Ports}}" service 4vv32yjkiag2:service_nginx.1:nginx:latest:docker1:Running:Running 18 minutes ago::*:80->80/tcp# --no-resolve[root@docker1 ~]# docker stack ps --no-resolve service ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS4vv32yjkiag2 c933fl4jmhmd4y2oqg20zgw2g.1 nginx:latest r9u09chhiipgu35dtax9g75xf Running Running 19 minutes ago *:80->80/tcp# --no-trunc[root@docker1 ~]# docker stack ps --no-trunc service ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS4vv32yjkiag243t2msgeci1q8 service_nginx.1 nginx:latest@sha256:9a1f8ed9e2273e8b3bbcd2e200024adac624c2e5c9b1d420988809f5c0c41a5e docker1 Running Running 19 minutes ago # -q[root@docker1 ~]# docker stack ps -q service 4vv32yjkiag23、docker stack service
显示docker stack 详细信息
[root@docker1 ~]# docker stack services service ID NAME MODE REPLICAS IMAGE PORTSc933fl4jmhmd service_nginx replicated 1/1 nginx:latest 4、docker stack rm
用于删除stack
[root@docker1 ~]# docker stack rm service Removing service service_nginxRemoving network service_default
3347
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
