Docker Swarm集群使用NFS共享存储

最新推荐文章于 2024-07-11 10:57:44 发布

写码的猿是攻城狮

最新推荐文章于 2024-07-11 10:57:44 发布

阅读量1.4k

点赞数

分类专栏： Docker

本文链接：https://blog.csdn.net/lms99251/article/details/108702547

版权

Docker 专栏收录该内容

16 篇文章 0 订阅

订阅专栏

安装NFS

系统环境为centos，一共有7台机子组成了Docker Swarm集群，每台机子都要安装NFS

IP	Docker角色	NFS角色
111.111.3.41	Manager	Server
111.111.3.42	Manager	Client
111.111.3.43	Manager	Client
111.111.3.44	Worker	Client
111.111.3.45	Manager	Client
111.111.3.46	Manager	Client
111.111.3.47	Worker	Client

安装依赖

yum -y install nfs-utils

配置文件

使用111.111.3.41为NFS的服务端

先在/root目录下创建share文件夹，并赋予chmod 755权限

配置共享目录

nano /etc/exports
/root/share 111.111.3.0/24(rw,async,insecure,anonuid=1000,anongid=1000,no_root_squash)

生效配置

exportfs -rv

启动NFS服务

systemctl enable rpcbind
systemctl start rpcbind
systemctl enable nfs-server
systemctl start nfs-server

查看NFS服务状态

[root@KD111111003041 ~]#  rpcinfo -p
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    3   tcp    111  portmapper
    100000    2   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100000    3   udp    111  portmapper
    100000    2   udp    111  portmapper
    100024    1   udp  45889  status
    100024    1   tcp  42518  status
    100005    1   udp  20048  mountd
    100005    1   tcp  20048  mountd
    100005    2   udp  20048  mountd
    100005    2   tcp  20048  mountd
    100005    3   udp  20048  mountd
    100005    3   tcp  20048  mountd
    100003    3   tcp   2049  nfs
    100003    4   tcp   2049  nfs
    100227    3   tcp   2049  nfs_acl
    100003    3   udp   2049  nfs
    100003    4   udp   2049  nfs
    100227    3   udp   2049  nfs_acl
    100021    1   udp  37813  nlockmgr
    100021    3   udp  37813  nlockmgr
    100021    4   udp  37813  nlockmgr
    100021    1   tcp  40301  nlockmgr
    100021    3   tcp  40301  nlockmgr
    100021    4   tcp  40301  nlockmgr
[root@KD111111003041 ~]# cat /var/lib/nfs/etab
/root/share	111.111.3.0/24(rw,async,wdelay,hide,nocrossmnt,insecure,no_root_squash,no_all_squash,no_subtree_check,secure_locks,acl,no_pnfs,anonuid=1000,anongid=1000,sec=sys,rw,insecure,no_root_squash,no_all_squash)

Docker Swarm中使用NFS

方式1：Docker Server 发布

创建Docker volume

每个docker节点都要手动创建相同名称的Docker volume

docker volume create --driver local \
    --opt type=nfs \
    --opt o=addr=111.111.3.41,rw \
    --opt device=:/root/share \
    foo33

查看volume

[root@KD111111003041 ~]# docker volume ls
DRIVER              VOLUME NAME
local               app_foo_new
local               foo33
local               portainer_data

查看foo33的详细

[root@KD111111003041 ~]# docker volume inspect foo33
[
    {
        "CreatedAt": "2020-09-17T09:55:08+08:00",
        "Driver": "local",
        "Labels": {},
        "Mountpoint": "/var/lib/docker/volumes/foo33/_data",
        "Name": "foo33",
        "Options": {
            "device": ":/root/share",
            "o": "addr=111.111.3.41,rw",
            "type": "nfs"
        },
        "Scope": "local"
    }
]

发布服务

在manager节点下创建服务

docker service create \
  --name test-nginx-nfs \
  --publish 84:80 \
  --mount type=volume,source=foo33,destination=/app/share \
  --replicas 3 \
  nginx

方式2：使用 docker stack 方发布

编写docker-compose.yml文档

version: '3.8'

services:

  nginx-test5:
    image: nginx:latest
    deploy:
      mode: replicated
      replicas: 3
      restart_policy:
        condition: on-failure
    ports:
      - "88:80"
    networks:
      my-overlay-network:
        aliases:
          - nginx-test5
    volumes:
      - "foo_new:/app/share"


volumes:
  foo_new:
    driver: local
    driver_opts:
      type: "nfs"
      o: "addr=111.111.3.41,rw"
      device: ":/root/share"

networks:
  my-overlay-network:
    driver: overlay

使用docker stack编排方式，无需手动创建Docker volume

发布服务

将编写好的docker-compose.yml上传到是manager的机子上的任意目录下，然后在控制台转到对应的目录下输入命令

[root@KD111111003041 ~]# docker stack deploy -c docker-compose.yml app
Creating network app_my-overlay-network
Creating service app_nginx-test5
[root@KD111111003041 ~]#  docker stack ls
NAME                SERVICES            ORCHESTRATOR
app                 1                   Swarm
[root@KD111111003041 ~]# docker stack services app
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
u02yjjdzt307        app_nginx-test5     replicated          3/3                 nginx:latest        *:88->80/tcp
[root@KD111111003041 ~]# docker service ps app_nginx-test5
ID                  NAME                IMAGE               NODE                               DESIRED STATE       CURRENT STATE           ERROR               PORTS
m246y53tmslk        app_nginx-test5.1   nginx:latest        KD111111003042.ppp-bb.dion.ne.jp   Running             Running 3 minutes ago                       
oxsyphx6qnvh        app_nginx-test5.2   nginx:latest        KD111111003047.ppp-bb.dion.ne.jp   Running             Running 3 minutes ago                       
nj4l0193yxrd        app_nginx-test5.3   nginx:latest        KD111111003044.ppp-bb.dion.ne.jp   Running             Running 3 minutes ago                       
[root@KD111111003041 ~]#

参考文章：Docker Swarm 使用NFS数据共享