文章目录
docker网络原理
docker Compose容器编排
构建自动发现的Docker服务架构
实现容器服务自动加入nginx集群
一:Doker网络通信
1.1Docker单机网络拓扑图
1.2 端口映射
- 端口映射机制将容器内的服务提供给外部网络访问
- 可随机或者指定映射端口
docker run -d -P httpd:centos
docker run -d -p 60000:80 httpd:centos
1.3 容器互联
- 在源容器和接收容器间建立一条网络通信隧道
- 使用docker run --link 选项实现容器间互连通信
实现容器互联命令
docker run -d -P --name web1 httpd:centos
docker run -d -P --name web2 --link web1:web1 httpd:centos
docker exec -it web2 /bin/bash
ping web1
二:Docker Compose容器编排
2.1 Docker Compose 容器编排概述
-
Docker Compose的前身是Fig,它是一个定义及运行多个Docker容器的工具
-
使用Docker Compose 不再需要使用shell脚本来启动容器
-
Docker Compose非常适合组合使用多个容器进行开发的场景
备注:浅析Dockerfile与Compose之间区别
compose不仅可以给构建镜像,还可以自动启动
dockerfile只提供镜像
dockerfile一次只能执行一个镜像
compose可以执行多个,他的文件结尾时yaml
2.2 Docker Compose环境准备
[root@localhost ~]# curl -L https://github.com/docker/compose/releases/download/1.21.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
[root@localhost ~]# chmod +x /usr/local/bin/docker-compose
[root@localhost ~]# docker-compose-v
2.3 查看Docker Compose文件结构
[root@localhost compose_lnmp]# vim docker-compose.yml
YAML是一种标记语言很直观的数据序列化格式
文件格式及编写注意事项
- 不支持制表符tab键缩进,需要使用空格缩进
- 通常开头缩进2个空格
- 字符后缩进1个空格,如:冒号、,逗号、-横杠
- 用#号注释
- 如果包含特殊字符用单引号引起来
- 布尔值必须用引号括起来
2.4 Docker Compose配置常用字段
序号 | 字段 | 描述 |
---|---|---|
1 | build dockerfile context | 指定Dockerfile文件名构建镜像上下文路径 |
2 | image | 指定镜像 |
3 | command | 执行命令,覆盖默认命令 |
4 | container name | 指定容器名称,由于容器名称是唯一的,如果自定自定义名称,则无法scale |
5 | deploy | 指定部署和运行服务相关配置,只能在Swam模式使用 |
6 | environment | 添加环境变量 |
7 | network | 加入网络 |
8 | ports | 暴露容器端口,与-p相同,注意端口不能低于60 |
9 | volumes | 挂载宿主机路径或命令卷 |
10 | restart | 重启策略,默认no,always,no-failure,unless-stopped |
11 | hostname | 容器主机名 |
2.5 compose 常用操作命令
序号 | 字段 | 描述 |
---|---|---|
1 | build | 重新构建服务 |
2 | ps | 列出容器 |
3 | up | 创建和启动容器 |
4 | exec | 在容器里面执行命令 |
5 | scale | 指定一个服务容器启动数量,可以理解为副本数,一次性创建容器的个数 |
6 | top | 显示容器进程 |
7 | logs | 查看容器输出 |
8 | down | 删除容器、网络、数据卷和镜像 |
9 | stop、start、restart | 停止、启动、重启服务 |
2.6 Compose命令说明
- 基本的使用格式
docker-compose [选项] [命令] [ARGS]
- docker-compose选项
–version 打印版本并退出
–verbose 输出更多调试信息
-f ,–file FILE 使用特定的compose模板文件,默认为docker-compose.yml
-p , --project-name NAME 指定项目名称,默认使用目录名称
2.7 演示使用Docker-compose创建nginx
首先要记得先部署好环境
所有主机都安装docker环境(内容为docker基础)
yum install compose -y
具体可见之前博客
2.7.1 下载compose
[root@ct ~]# curl -L https://github.com/docker/compose/releases/download/1.21.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 638 100 638 0 0 657 0 --:--:-- --:--:-- --:--:-- 657
100 10.3M 100 10.3M 0 0 1245k 0 0:00:08 0:00:08 --:--:-- 1893k
[root@ct ~]# chmod +x /usr/local/bin/docker-compose
2.7.1 创建nginx
[root@ct ~]# mkdir /root/compose_nginx
[root@ct ~]# cd /root/compose_nginx/
[root@ct compose_nginx]#
[root@ct compose_nginx]# mkdir nginx
[root@ct compose_nginx]#
[root@ct compose_nginx]# cp /abc/LNMP-C7/LNMP-C7/nginx-1.12.2.tar.gz /root/compose_nginx/nginx/
[root@ct compose_nginx]# vim /root/compose_nginx/nginx/Dockerfile
FROM centos:7
MAINTAINER to finsh nginx
RUN yum -y update
RUN yum -y install gcc gcc-c++ pcre* make cmake zlib-devel openssh* net-tools lsof telnet passwd vim
ADD nginx-1.12.2.tar.gz /usr/local/src
RUN useradd -M -s /sbin/nologin nginx
WORKDIR /usr/local/src/nginx-1.12.2
RUN (./configure --prefix=/usr/local/nginx --user=nginx --group=nginx --with-http_stub_status_module)
RUN make && make install
ENV PATH /usr/local/nginx/sbin/:$PATH
#RUN ln -s /usr/local/nginx/sbin/* /usr/local/sbin/
EXPOSE 80
EXPOSE 443
RUN echo "daemon off;" >> /usr/local/nginx/conf/nginx.conf
#指关闭守护进程启动
CMD ["nginx"]
[root@ct compose_nginx]# mkdir /root/compose_nginx/wwwroot
[root@ct compose_nginx]# echo "this is gsy" > /root/compose_nginx/wwwroot/index.html
[root@ct compose_nginx]# vim /root/compose_nginx/docker-compose.yml
version: '3'
services:
nginx:
hostname: nginx
build:
context: ./nginx
dockerfile: Dockerfile
ports:
- 1216:80
- 1217:443
networks:
- cluster
volumes:
- ./wwwroot:/usr/local/nginx/html
networks:
cluster:
[root@ct compose_nginx]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
[root@ct compose_nginx]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
systemctl new a2c3616f15d1 34 hours ago 717MB
Warning翻译如下
警告:为服务nginx构建的映像,因为它还不存在。要重建此映像,您必须使用“docker-compose build”或“docker-compose up—build”。
创建compose_nginx_nginx_1……完成
2.7.2 验证docker-compose创建的nginx
三:Docker consul容器服务更新与发现
consul可以算是微服务中的内容
3.1 容器服务更新发现拓扑图
consul template 相当于配置文件模板
consul server会根据这个进行更新
regisrator注册机制
当后面增加了一个容器时,容器会注册registrator,
registrator会发现多了一个容器,便会通知consul server要更新
consul server使用consul template自动更新
3.2 Consul概述
Consul时HashiCorp公司推出的开源工具,用于实现分布式系统的服务发现与配置
Consul的特性
- Consul支持健康检查,允许存储键值对
- 一致性协议采用Raft算法用来保证服务的高可用
- 成员管理和罅隙广播采用GOSSIP协议,支持ACL访问控制
方便部署,与Docker等轻量级容器可无缝配合
四:演示构建自动发现的Docker服务架构
4.1 建立Consul服务注意事项
- 每个提供服务的节点上都要部署和运行Consul的agent
- Consul agent有两种运行模式
- Server
- Client
- Server和Clinet只是Consul集群层面的群分,与搭建在Cluster之上的应用服务无关
4.2 主节点建立Consul服务,即consul agent ——server
先介绍下环境
192.168.247.20 Docker-ce、Compose 3、Consul、Consul-template
192.168.247.142 Docker-ce、registrator
consul使用go语言编写的
[root@ct compose_nginx]# mkdir /root/consul
[root@ct compose_nginx]# cd /root/consul/
[root@ct consul]# ls
[root@ct consul]#
[root@ct consul]# cp /abc/consul_0.9.2_linux_amd64.zip /root/consul/
[root@ct consul]# ls
consul_0.9.2_linux_amd64.zip
[root@ct consul]# unzip consul_0.9.2_linux_amd64.zip
Archive: consul_0.9.2_linux_amd64.zip
inflating: consul
[root@ct consul]# mv consul /usr/bin/
[root@ct consul]# consul agent \
> -server \
> -bootstrap \ //前端框架
> -ui \
> -data-dir=/var/lib/consul-data \
> -bind=192.168.247.20 \
> -client=0.0.0.0 \
> -node=consul-server01 &> /var/log/consul.log&
[1] 3339
[root@ct consul]# jobs
[1]+ Running consul agent -server -bootstrap -ui -data-dir=/var/lib/consul-data -bind=192.168.247.20 -client=0.0.0.0 -node=consul-server01 &>/var/log/consul.log &
[root@ct consul]#
4.3 查看集群信息
[root@ct consul]# consul members
Node Address Status Type Build Protocol DC
consul-server01 192.168.247.20:8301 alive server 0.9.2 2 dc1
[root@ct consul]# consul info | grep leader
leader = true
leader_addr = 192.168.247.20:8300
备注:查看consul info
[root@ct consul]# consul info
agent:
check_monitors = 0
check_ttls = 0
checks = 0
services = 0
build:
prerelease =
revision = 75ca2ca
version = 0.9.2
consul:
bootstrap = true
known_datacenters = 1
leader = true
leader_addr = 192.168.247.20:8300
server = true
raft:
applied_index = 10
commit_index = 10
fsm_pending = 0
last_contact = 0
last_log_index = 10
last_log_term = 2
last_snapshot_index = 0
last_snapshot_term = 0
latest_configuration = [{Suffrage:Voter ID:192.168.247.20:8300 Address:192.168.247.20:8300}]
latest_configuration_index = 1
num_peers = 0
protocol_version = 2
protocol_version_max = 3
protocol_version_min = 0
snapshot_version_max = 1
snapshot_version_min = 0
state = Leader
term = 2
runtime:
arch = amd64
cpu_count = 4
goroutines = 61
max_procs = 4
os = linux
version = go1.8.3
serf_lan:
coordinate_resets = 0
encrypted = false
event_queue = 1
event_time = 2
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 1
members = 1
query_queue = 0
query_time = 1
serf_wan:
coordinate_resets = 0
encrypted = false
event_queue = 0
event_time = 1
failed = 0
health_score = 0
intent_queue = 0
left = 0
member_time = 1
members = 1
query_queue = 0
query_time = 1
4.4 通过http api获取集群信息
curl 127.0.0.1:8500/v1/status/peers //查看集群server成员
curl 127.0.0.1:8500/v1/status/leaders //查看集群Raf leader
curl 127.0.0.1:8500/v1/catalog/services //查看注册的所有服务
curl 127.0.0.1:8500/v1/catalog/nginx //查看nginx服务的信息
curl 127.0.0.1:8500/v1/catalog/nodes //集群节点详细信息
[root@ct consul]# curl 127.0.0.1:8500/v1/status/peers //查看集群server成员
["192.168.247.20:8300"][root@ct consul]#
[root@ct consul]# curl 127.0.0.1:8500/v1/status/leaders //查看集群Raf leader
[root@ct consul]# curl 127.0.0.1:8500/v1/catalog/services //查看注册的所有服务
{"consul":[]}[root@ct consul]#
[root@ct consul]# curl 127.0.0.1:8500/v1/catalog/nginx //查看nginx服务的信息
[root@ct consul]# curl 127.0.0.1:8500/v1/catalog/nodes //集群节点详细信息
[{"ID":"3b8f315c-cb49-0a15-9869-a846f5d2d6fd","Node":"consul-server01","Address":"192.168.247.20","Datacenter":"dc1","TaggedAddresses":{"lan":"192.168.247.20","wan":"192.168.247.20"},"Meta":{},"CreateIndex":5,"ModifyIndex":6}][root@ct consul]#
五:实现容器服务自动加入nginx集群
5.1 安装gliderlabs/registrator gliderlabs/registrator
- 检查容器运行状态
- 自动注册和注销docker容器的服务到服务配置中心
- 目前支持consul、Etcd和skyDNS2
5.2 在192.168.247.142节点即操作
[root@localhost ~]# docker run -d \
--name=registrator \
--net=host \
-v /var/run/docker.sock:/tmp/docker.sock \
--restart=always \
gliderlabs/registrator:latest \
-ip=192.168.247.142 \
consul://192.168.247.20:8500
查看状态
[root@localhost ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
gliderlabs/registrator latest 3b59190c6c80 4 years ago 23.8MB
[root@localhost ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
0fdcf13a88bf gliderlabs/registrator:latest "/bin/registrator -i…" 49 seconds ago Restarting (1) 20 seconds ago registrator
[root@localhost ~]#
5.3 测试服务发现功能是否正常
在registrator节点处创建容器
-h 指定容器的主机名
[root@localhost ~]# docker run -itd -p 83:80 --name test-01 -h test01 nginx
[root@localhost ~]# docker run -itd -p 84:80 --name test-02 -h test02 nginx
[root@localhost ~]# docker run -itd -p 85:80 --name test-03 -h test03 httpd
[root@localhost ~]# docker run -itd -p 86:80 --name test-04 -h test04 httpd
查看容器状态
[root@localhost ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
db4b5d76c22e httpd "httpd-foreground" 5 seconds ago Up 4 seconds 0.0.0.0:86->80/tcp test-04
5794680892c4 httpd "httpd-foreground" About a minute ago Up About a minute 0.0.0.0:85->80/tcp test-03
3db53dacc006 nginx "nginx -g 'daemon of…" About a minute ago Up About a minute 0.0.0.0:84->80/tcp test-02
d7ef2151f770 nginx "nginx -g 'daemon of…" 3 minutes ago Up 3 minutes 0.0.0.0:83->80/tcp test-01
0fdcf13a88bf gliderlabs/registrator:latest "/bin/registrator -i…" 8 minutes ago Restarting (1) Less than a second ago registrator
此时到consul节点,查看8500端口已被打开(之前并没有出现)
[root@ct consul]# netstat -natp | grep 8500
tcp6 0 0 :::8500 :::* LISTEN 3339/consul
You have new mail in /var/spool/mail/root
5.4 验证后端http服务是否注册到consul节点
需要关闭防火墙或者开放8500端口,关闭核心防护
#systemctl stop firewalld
[root@ct consul]# firewall-cmd --get-active-zones
public
interfaces: eth0 eth1
[root@ct consul]# firewall-cmd --zone=public --add-port=8500/tcp
#此方法是临时修改,如果永久修改,需要--permanent然后firewall-cmd --reload
success
[root@ct consul]# setenforce ?
setenforce: SELinux is disabled
5.5 测试网页是否开启
5.6 查看对应容器的日志文件
发现访问IP是物理网卡的网关
[root@localhost ~]# docker logs -f test-01
192.168.247.1 - - [22/Apr/2020:06:44:07 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36" "-"
2020/04/22 06:44:07 [error] 6#6: *2 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 192.168.247.1, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "192.168.247.142:83", referrer: "http://192.168.247.142:83/"
192.168.247.1 - - [22/Apr/2020:06:44:07 +0000] "GET /favicon.ico HTTP/1.1" 404 556 "http://192.168.247.142:83/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36" "-"
5.7 再次查看状态
[root@ct ~]# curl 127.0.0.1:8500/v1/catalog/services
{"consul":[],"httpd":[],"nginx":[]}[root@ct ~]#
六 consul-template实现容器自动加入Nginx集群
- 是基于consul的自动替换配置文件的应用
- 可以查询consul中的服务目录、key、key-values等
- 特别适合动态创建配置文件
consul-template是一个守护进程,用来实时查询consul集群信息,并更新文件系统上任意数量的指定模板,生成配置文件;更新完成以后,可以选择运行shell命令执行更新操作,重新加载nginx
可以查询consul中的服务目录、key、key、value等
这种强大的抽象功能和查询语言模板功能让consul-template特别适合动态的创建配置文件
例如:创建apache/nginx proxy balacers、haproxy backends
6.1 在consul节点安装consul-template,准备模板文件
备注:将脚本文件移动到/usr/bin下
[root@ct ~]# cp /abc/consul-template_0.19.3_linux_amd64.zip .
[root@ct ~]# unzip consul-template_0.19.3_linux_amd64.zip
Archive: consul-template_0.19.3_linux_amd64.zip
inflating: consul-template
[root@ct ~]# ls
anaconda-ks.cfg compose_nginx consul consul-template consul-template_0.19.3_linux_amd64.zip
[root@ct ~]# mv consul-template /usr/bin/
- 准备template nginx模板文件
备注:
此模板用于nginx反向代理模板
nginx.ctmpl跟nginx没有直接关系,
consul是docker的一种自动管理机制
nginx.ctmpl中的参数以变量的形式写入
[root@ct nginx-1.12.2]# mkdir /var/log/nginx/
[root@ct ~]# vim /root/consul/nginx.ctmpl
upstream http_backend {
{
{range service "nginx"}}
server {
{.Address}}:{
{.Port}};
{
{end}}
}
server {
listen 88;
server_name ct 192.168.247.20;
access_log /var/log/nginx/gsy.cn-access.log;
index index.html index.php;
location / {
proxy_set_header HOST $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Client-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://http_backend;
}
}
6.2 consul节点编译安装nginx
[root@ct consul]# yum install gcc gcc-c++ make expat* pcre* perl* zlib* -y
[root@ct consul]# cp /abc/LNMP-C7/LNMP-C7/nginx-1.12.2.tar.gz .
[root@ct consul]# tar xf nginx-1.12.2.tar.gz
[root@ct consul]# ls
consul_0.9.2_linux_amd64.zip nginx-1.12.2 nginx-1.12.2.tar.gz nginx.ctmpl
[root@ct nginx-1.12.2]# ./configure --prefix=/usr/local/nginx
[root@ct nginx-1.12.2]# make && make install
6.3 配置nginx然后启动
[root@ct nginx-1.12.2]# mkdir /usr/local/nginx/conf/vhost/
[root@localhost ~]# vim /usr/local/nginx/conf/nginx.conf
include vhost/*.conf; //添加虚拟主机配置文件路径
[root@ct nginx-1.12.2]# /usr/local/nginx/sbin/nginx
[root@ct nginx-1.12.2]# netstat -natp | grep nginx
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 77583/nginx: master
[root@ct consul]# firewall-cmd --zone=public --add-port=80/tcp
success
6.4 准备工作完毕,启动template,指定template模板文件及生成路径
指定模板路径,/root/consul/nginx.ctmpl,生成到/usr/locla/nginx/conf/vhost/gsy.conf,然后重载nginx -s reload
[root@ct conf.d]# consul-template -consul-addr 192.168.247.20:8500 \
-template "/root/consul/nginx.ctmpl:/usr/local/nginx/conf/vhost/gsy.conf:/usr/local/nginx/sbin/nginx -s reload" \
--log-level=info
输入这段指定后便会进入监控状态
6.5 然后打开另一个终端查看生成配置文件
获取到IP,是依靠consul功能
[root@ct nginx]# cd -
/usr/local/nginx/conf/vhost
[root@ct vhost]# ls
gsy.conf
[root@ct vhost]# vim gsy.conf
upstream http_backend {
server 192.168.247.142:83;
server 192.168.247.142:84;}
server {
listen 88;
server_name ct 192.168.247.20;
access_log /var/log/nginx/gsy.cn-access.log;
index index.html index.php;
location / {
proxy_set_header HOST $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Client-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://http_backend;
}
}
6.6 在客户端测试并结合nginx后端容器节点logs验证
- 如果能访问且后端容器节点logs互为轮询说明服务已自动发现及更新配置文件完毕
[root@localhost ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
db4b5d76c22e httpd "httpd-foreground" 23 hours ago Up 23 hours 0.0.0.0:86->80/tcp test-04
5794680892c4 httpd "httpd-foreground" 23 hours ago Up 23 hours 0.0.0.0:85->80/tcp test-03
3db53dacc006 nginx "nginx -g 'daemon of…" 24 hours ago Up 24 hours 0.0.0.0:84->80/tcp test-02
d7ef2151f770 nginx "nginx -g 'daemon of…" 24 hours ago Up 24 hours 0.0.0.0:83->80/tcp test-01
0fdcf13a88bf gliderlabs/registrator:latest "/bin/registrator -i…" 24 hours ago Up 23 hours registrator
[root@localhost ~]# docker logs -f test-01
[root@localhost ~]# docker logs -f test-02 //新开一个终端
在consul节点开放88端口
[root@ct consul]# firewall-cmd --zone=public --add-port=88/tcp
success
访问一次
再访问一次
多次下来可以发现到是轮询访问后方docker容器的,若想验证更加明显,可以在两个容器内添加不同的首页
七:consul-template测试加入新容器是否会被发现
当前容器数量(已删掉之前的httpd容器)
[root@localhost ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3db53dacc006 nginx "nginx -g 'daemon of…" 24 hours ago Up 24 hours 0.0.0.0:84->80/tcp test-02
d7ef2151f770 nginx "nginx -g 'daemon of…" 24 hours ago Up 24 hours 0.0.0.0:83->80/tcp test-01
0fdcf13a88bf gliderlabs/registrator:latest "/bin/registrator -i…" 24 hours ago Up 24 hours registrator
添加容器
[root@localhost ~]# docker run -itd --name test-03 -p 86:80 -h test03 nginx
c1fee05a5f45734b32816f2acd9717b61c833ed7d9a9fe6f638ff08e0b5466f4
[root@localhost ~]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c1fee05a5f45 nginx "nginx -g 'daemon of…" 7 seconds ago Up 6 seconds 0.0.0.0:86->80/tcp test-03
3db53dacc006 nginx "nginx -g 'daemon of…" 24 hours ago Up 24 hours 0.0.0.0:84->80/tcp test-02
d7ef2151f770 nginx "nginx -g 'daemon of…" 24 hours ago Up 24 hours 0.0.0.0:83->80/tcp test-01
0fdcf13a88bf gliderlabs/registrator:latest "/bin/registrator -i…" 24 hours ago Up 24 hours registrator
添加配置文件
server 192.168.247.142:83;
server 192.168.247.142:84;
server 192.168.247.142:86;}
server {
listen 88;
server_name ct 192.168.247.20;
access_log /var/log/nginx/gsy.cn-access.log;
index index.html index.php;
location / {
proxy_set_header HOST $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Client-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://http_backend;
}
}
验证IP是否可用
刷新了三次,test-03日志出现消息,证实可用