Use overlay networks

Overlay networks（覆盖网络）是一种在现有网络基础上构建的虚拟网络，它通过封装和解封装原有网络的数据包，实现了在原有网络上添加安全、加密、虚拟化等新功能。Overlay networks通常用于实现虚拟化、云计算、容器化等技术，以增加网络的灵活性和可扩展性。在Docker中，overlay networks可以被用于实现多主机间的网络连接，以便在多个Docker守护进程主机之间创建一个分布式网络，该网络允许连接到它的容器安全地通信。您可以通过初始化一个swarm或将一个Docker主机加入到一个已有的swarm中来创建overlay networks。
一旦overlay networks被创建，您可以在其上运行Docker容器，这些容器可以通过overlay networks彼此通信，而无需通过宿主机的网络接口。Overlay networks具有一些关键的优点：

1. 高可用性：如果某个Docker守护进程主机发生故障，由于overlay networks的分布式特性，连接在该网络上的容器仍然可以继续通信。
2. 安全性：Overlay networks提供了一种在容器之间实现加密通信的方法，从而提高了数据的安全性。
3. 灵活性：Overlay networks允许您轻松地创建和管理多个网络，每个网络都可以有自己的配置和规则。
4. 可扩展性：随着应用程序的增长，您可以轻松地添加更多的Docker主机到overlay networks中，以支持更多的容器。
   要使用overlay networks，您需要安装并配置Docker Swarm或Kubernetes等集群管理工具，并确保所有主机都在同一网络上，并且能够相互通信。然后，您可以创建一个overlay networks，并将您的容器连接到该网络。
   使用overlay networks时，您需要注意以下几点：
5. 网络性能：由于overlay networks会增加数据包封装和解封装的过程，因此在某些情况下可能会影响网络性能。
6. 网络配置：overlay networks的配置相对复杂，需要仔细规划和管理，以确保网络的正常运行。
7. 网络管理工具：为了更好地管理和监控overlay networks，您需要使用适当的网络管理工具。
8. 网络安全性：虽然overlay networks提供了加密通信等安全特性，但仍然需要采取其他安全措施，以确保网络的安全性。
   总之，使用overlay networks可以提供高可用性、安全性、灵活性和可扩展性的网络环境，以支持Docker容器等应用程序。但是，您需要仔细规划和管理网络，并采取适当的安全措施，以确保网络的正常运行和安全性。
   在选择和使用overlay networks时，以下是一些建议和最佳实践：
9. 评估需求：在开始使用overlay networks之前，评估您的需求和目标。确定您需要哪些功能和特性，以及overlay networks是否适合您的应用程序。
10. 选择合适的工具：选择适合您需求和环境的overlay networks工具。了解各种工具的优缺点，并选择最适合您的方案。
11. 仔细规划网络：在创建overlay networks之前，仔细规划网络的设计和架构。考虑网络的大小、拓扑结构、路由规则等，以确保网络的正常运行和性能。
12. 配置和管理网络：根据您的规划和需求，配置和管理overlay networks。确保所有主机都在同一网络上，能够相互通信，并正确配置路由、防火墙等规则。
13. 监控和日志记录：为了确保overlay networks的正常运行和性能，实施适当的监控和日志记录机制。收集和分析网络性能数据、错误日志等，以便及时发现和解决问题。
14. 安全措施：采取适当的安全措施来保护overlay networks。使用加密通信、访问控制列表（ACL）、防火墙等安全机制，以防止未经授权的访问和攻击。
15. 备份和恢复：定期备份overlay networks的配置和数据，以便在发生故障或数据丢失时进行恢复。
16. 持续学习和更新：随着技术的不断发展和更新，持续学习和关注overlay networks的最新动态和最佳实践。及时更新您的网络管理工具和配置，以保持网络的稳定性和安全性。
    总之，使用overlay networks需要仔细规划和配置，并采取适当的安全措施。通过遵循这些建议和最佳实践，您可以更好地利用overlay networks的优势，为Docker容器等应用程序提供稳定、安全和高性能的网络环境。
    除了上述提到的建议和最佳实践，还有一些额外的注意事项：
17. 资源管理：Overlay networks可能会增加网络的复杂性，并占用更多的计算和存储资源。因此，需要仔细考虑如何有效地管理和利用这些资源。
18. 可扩展性和灵活性：Overlay networks应具有良好的可扩展性和灵活性，以便适应应用程序和业务需求的变化。
19. 兼容性和互操作性：选择兼容性和互操作性良好的overlay networks工具，以便与其他技术和服务集成。
20. 性能优化：通过优化overlay networks的配置和参数，可以提高网络性能和效率。
21. 故障转移和灾难恢复：设计和实施有效的故障转移和灾难恢复策略，以最大程度地减少网络故障对应用程序的影响。
22. 合规性和审计：确保overlay networks符合相关的法规和合规要求，并进行适当的审计和记录。
23. 培训和支持：提供对overlay networks的培训和支持，以确保用户和管理员能够正确地使用和管理网络。
24. 持续集成和持续部署（CI/CD）：集成overlay networks到CI/CD流程中，以便自动化部署和管理应用程序。
    通过遵循这些注意事项，您可以更好地管理和优化overlay networks，并确保它们能够满足您的需求和目标。
    Estimated reading time: 11 minutes

The overlay network driver creates a distributed network among multiple Docker daemon hosts. This network sits on top of (overlays) the host-specific networks, allowing containers connected to it (including swarm service containers) to communicate securely when encryption is enabled. Docker transparently handles routing of each packet to and from the correct Docker daemon host and the correct destination container.

When you initialize a swarm or join a Docker host to an existing swarm, two new networks are created on that Docker host:

an overlay network called ingress, which handles control and data traffic related to swarm services. When you create a swarm service and do not connect it to a user-defined overlay network, it connects to the ingress network by default.
a bridge network called docker_gwbridge, which connects the individual Docker daemon to the other daemons participating in the swarm.

You can create user-defined overlay networks using docker network create, in the same way that you can create user-defined bridge networks. Services or containers can be connected to more than one network at a time. Services or containers can only communicate across networks they are each connected to.

Although you can connect both swarm services and standalone containers to an overlay network, the default behaviors and configuration concerns are different. For that reason, the rest of this topic is divided into operations that apply to all overlay networks, those that apply to swarm service networks, and those that apply to overlay networks used by standalone containers.
Operations for all overlay networks
Create an overlay network

Prerequisites:

    Firewall rules for Docker daemons using overlay networks

    You need the following ports open to traffic to and from each Docker host participating on an overlay network:
        TCP port 2377 for cluster management communications
        TCP and UDP port 7946 for communication among nodes
        UDP port 4789 for overlay network traffic

    Before you can create an overlay network, you need to either initialize your Docker daemon as a swarm manager using docker swarm init or join it to an existing swarm using docker swarm join. Either of these creates the default ingress overlay network which is used by swarm services by default. You need to do this even if you never plan to use swarm services. Afterward, you can create additional user-defined overlay networks.

To create an overlay network for use with swarm services, use a command like the following:

$ docker network create -d overlay my-overlay

To create an overlay network which can be used by swarm services or standalone containers to communicate with other standalone containers running on other Docker daemons, add the --attachable flag:

$ docker network create -d overlay --attachable my-attachable-overlay

You can specify the IP address range, subnet, gateway, and other options. See docker network create --help for details.
Encrypt traffic on an overlay network

All swarm service management traffic is encrypted by default, using the AES algorithm in GCM mode. Manager nodes in the swarm rotate the key used to encrypt gossip data every 12 hours.

To encrypt application data as well, add --opt encrypted when creating the overlay network. This enables IPSEC encryption at the level of the vxlan. This encryption imposes a non-negligible performance penalty, so you should test this option before using it in production.

When you enable overlay encryption, Docker creates IPSEC tunnels between all the nodes where tasks are scheduled for services attached to the overlay network. These tunnels also use the AES algorithm in GCM mode and manager nodes automatically rotate the keys every 12 hours.

Do not attach Windows nodes to encrypted overlay networks.

Overlay network encryption is not supported on Windows. If a Windows node attempts to connect to an encrypted overlay network, no error is detected but the node cannot communicate.

Swarm mode overlay networks and standalone containers

You can use the overlay network feature with both --opt encrypted --attachable and attach unmanaged containers to that network:

$ docker network create --opt encrypted --driver overlay --attachable my-attachable-multi-host-network

Customize the default ingress network

Most users never need to configure the ingress network, but Docker 17.05 and higher allow you to do so. This can be useful if the automatically-chosen subnet conflicts with one that already exists on your network, or you need to customize other low-level network settings such as the MTU.

Customizing the ingress network involves removing and recreating it. This is usually done before you create any services in the swarm. If you have existing services which publish ports, those services need to be removed before you can remove the ingress network.

During the time that no ingress network exists, existing services which do not publish ports continue to function but are not load-balanced. This affects services which publish ports, such as a WordPress service which publishes port 80.

Inspect the ingress network using docker network inspect ingress, and remove any services whose containers are connected to it. These are services that publish ports, such as a WordPress service which publishes port 80. If all such services are not stopped, the next step fails.

Remove the existing ingress network:

$ docker network rm ingress

WARNING! Before removing the routing-mesh network, make sure all the nodes
in your swarm run the same docker engine version. Otherwise, removal may not
be effective and functionality of newly created ingress networks will be
impaired.
Are you sure you want to continue? [y/N]

Create a new overlay network using the --ingress flag, along with the custom options you want to set. This example sets the MTU to 1200, sets the subnet to 10.11.0.0/16, and sets the gateway to 10.11.0.2.

$ docker network create \
  --driver overlay \
  --ingress \
  --subnet=10.11.0.0/16 \
  --gateway=10.11.0.2 \
  --opt com.docker.network.driver.mtu=1200 \
  my-ingress

    Note: You can name your ingress network something other than ingress, but you can only have one. An attempt to create a second one fails.

Restart the services that you stopped in the first step.

Customize the docker_gwbridge interface

The docker_gwbridge is a virtual bridge that connects the overlay networks (including the ingress network) to an individual Docker daemon’s physical network. Docker creates it automatically when you initialize a swarm or join a Docker host to a swarm, but it is not a Docker device. It exists in the kernel of the Docker host. If you need to customize its settings, you must do so before joining the Docker host to the swarm, or after temporarily removing the host from the swarm.

Stop Docker.

Delete the existing docker_gwbridge interface.

$ sudo ip link set docker_gwbridge down

$ sudo ip link del dev docker_gwbridge

Start Docker. Do not join or initialize the swarm.

Create or re-create the docker_gwbridge bridge manually with your custom settings, using the docker network create command. This example uses the subnet 10.11.0.0/16. For a full list of customizable options, see Bridge driver options.

$ docker network create \
--subnet 10.11.0.0/16 \
--opt com.docker.network.bridge.name=docker_gwbridge \
--opt com.docker.network.bridge.enable_icc=false \
--opt com.docker.network.bridge.enable_ip_masquerade=true \
docker_gwbridge

Initialize or join the swarm. Since the bridge already exists, Docker does not create it with automatic settings.

Operations for swarm services
Publish ports on an overlay network

Swarm services connected to the same overlay network effectively expose all ports to each other. For a port to be accessible outside of the service, that port must be published using the -p or --publish flag on docker service create or docker service update. Both the legacy colon-separated syntax and the newer comma-separated value syntax are supported. The longer syntax is preferred because it is somewhat self-documenting.
Flag value Description
-p 8080:80 or
-p published=8080,target=80 Map TCP port 80 on the service to port 8080 on the routing mesh.
-p 8080:80/udp or
-p published=8080,target=80,protocol=udp Map UDP port 80 on the service to port 8080 on the routing mesh.
-p 8080:80/tcp -p 8080:80/udp or
-p published=8080,target=80,protocol=tcp -p published=8080,target=80,protocol=udp Map TCP port 80 on the service to TCP port 8080 on the routing mesh, and map UDP port 80 on the service to UDP port 8080 on the routing mesh.
Bypass the routing mesh for a swarm service

By default, swarm services which publish ports do so using the routing mesh. When you connect to a published port on any swarm node (whether it is running a given service or not), you are redirected to a worker which is running that service, transparently. Effectively, Docker acts as a load balancer for your swarm services. Services using the routing mesh are running in virtual IP (VIP) mode. Even a service running on each node (by means of the --mode global flag) uses the routing mesh. When using the routing mesh, there is no guarantee about which Docker node services client requests.

To bypass the routing mesh, you can start a service using DNS Round Robin (DNSRR) mode, by setting the --endpoint-mode flag to dnsrr. You must run your own load balancer in front of the service. A DNS query for the service name on the Docker host returns a list of IP addresses for the nodes running the service. Configure your load balancer to consume this list and balance the traffic across the nodes.
Separate control and data traffic

By default, control traffic relating to swarm management and traffic to and from your applications runs over the same network, though the swarm control traffic is encrypted. You can configure Docker to use separate network interfaces for handling the two different types of traffic. When you initialize or join the swarm, specify --advertise-addr and --datapath-addr separately. You must do this for each node joining the swarm.
Operations for standalone containers on overlay networks
Attach a standalone container to an overlay network

The ingress network is created without the --attachable flag, which means that only swarm services can use it, and not standalone containers. You can connect standalone containers to user-defined overlay networks which are created with the --attachable flag. This gives standalone containers running on different Docker daemons the ability to communicate without the need to set up routing on the individual Docker daemon hosts.
Publish ports
Flag value Description
-p 8080:80 Map TCP port 80 in the container to port 8080 on the overlay network.
-p 8080:80/udp Map UDP port 80 in the container to port 8080 on the overlay network.
-p 8080:80/sctp Map SCTP port 80 in the container to port 8080 on the overlay network.
-p 8080:80/tcp -p 8080:80/udp Map TCP port 80 in the container to TCP port 8080 on the overlay network, and map UDP port 80 in the container to UDP port 8080 on the overlay network.
Container discovery

For most situations, you should connect to the service name, which is load-balanced and handled by all containers (“tasks”) backing the service. To get a list of all tasks backing the service, do a DNS lookup for tasks..
Next steps

Go through the overlay networking tutorial
Learn about networking from the container’s point of view
Learn about standalone bridge networks
Learn about Macvlan networks

network, overlay, user-defined, swarm, service