docker root dir 迁移方案

docker root dir 迁移方案

Docker Root Dir

1. 存储镜像和容器数据：Docker的镜像和容器数据都存储在"root dir"中。镜像是Docker应用程序的构建块，而容器是基于镜像的运行实例。当您创建、拉取或运行容器时，相关的镜像和容器数据将存储在"root dir"中。
2. 存储容器日志：Docker容器的日志文件通常位于"root dir"中的相应位置。日志记录了容器的输出、错误和其他相关信息，以便您可以进行故障排除和监控。
3. 存储Docker运行时文件：Docker引擎的运行时文件、配置和元数据存储在"root dir"中。这些文件包括Docker守护进程的配置文件、网络设置、容器状态信息等。

一般 我们要把这些文件放到一个大的磁盘下面,防止后续 拉取镜像,或者什么操作 导致 磁盘满了, 导致镜像没有办法拉下来.

所以才有了这次迁移方案. 最近一台服务器 docker pull 出现磁盘空间不够了, 然后默认的 docker root dir 是 /var/lib/docker/

查看了一下 已经占用了 93%. 所以才进行 root dir 的迁移,移到一个 数据盘下面.

操作文档

环境介绍

[xx@xxx data]$ docker info
Client: Docker Engine - Community
 Version:    24.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 82
  Running: 30
  Paused: 0
  Stopped: 52
 Images: 149
 Server Version: 24.0.2

停掉服务

sudo systemctl stop docker
sudo systemctl stop docker.socket
sudo systemctl stop containerd

查看目录所剩余空间

df  -hl   /var/lib/docker/

创建文件夹

mkdir  -p   /data/sdv1/docker-root/

迁移数据服务 ,把数据进行迁移

使用 rsync 进行同步文件夹, 这个过程比较漫长,需要等待

rsync -avxP /var/lib/docker/  /data/sdv1/docker-root/

修改 文件 docker.service , /usr/lib/systemd/system/docker.service

vim /usr/lib/systemd/system/docker.service

找到 ExecStart=/usr/bin/dockerd -H fd:// 类似的这样的一行

修改这行内容

添加 --graph 参数选项 (deprecated)

ExecStart=/usr/bin/dockerd --graph /data/sdv1/docker-root   -H fd:// --containerd=/run/containerd/containerd.sock

reload the systemd configuration for Docker , 启动 docker

systemctl daemon-reload
systemctl start docker

发现启动失败了, 查看日志

Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.

docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Sat 2023-09-09 19:15:24 CST; 15s ago
     Docs: https://docs.docker.com
  Process: 83286 ExecStart=/usr/bin/dockerd --graph /data/sdv1/docker-root -H fd:// --containerd=/run/containerd/containerd.sock (code=exited, status=1/FAILURE)
 Main PID: 83286 (code=exited, status=1/FAILURE)

Sep 09 19:15:22 template systemd[1]: Unit docker.service entered failed state.
Sep 09 19:15:22 template systemd[1]: docker.service failed.
Sep 09 19:15:24 template systemd[1]: docker.service holdoff time over, scheduling restart.
Sep 09 19:15:24 template systemd[1]: Stopped Docker Application Container Engine.
Sep 09 19:15:24 template systemd[1]: start request repeated too quickly for docker.service
Sep 09 19:15:24 template systemd[1]: Failed to start Docker Application Container Engine.
Sep 09 19:15:24 template systemd[1]: Unit docker.service entered failed state.
Sep 09 19:15:24 template systemd[1]: docker.service failed.
[root@template ~]# journalctl -xe
Sep 09 19:15:20 template systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Sep 09 19:15:20 template systemd[1]: Unit docker.service entered failed state.
Sep 09 19:15:20 template systemd[1]: docker.service failed.
Sep 09 19:15:22 template systemd[1]: docker.service holdoff time over, scheduling restart.
Sep 09 19:15:22 template systemd[1]: Stopped Docker Application Container Engine.
-- Subject: Unit docker.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has finished shutting down.
Sep 09 19:15:22 template systemd[1]: Starting Docker Application Container Engine...
-- Subject: Unit docker.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has begun starting up.
Sep 09 19:15:22 template dockerd[83286]: Status: unknown flag: --graph
Sep 09 19:15:22 template dockerd[83286]: See 'dockerd --help'., Code: 125
Sep 09 19:15:22 template systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Sep 09 19:15:22 template systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.
--
-- The result is failed.
Sep 09 19:15:22 template systemd[1]: Unit docker.service entered failed state.
Sep 09 19:15:22 template systemd[1]: docker.service failed.
Sep 09 19:15:24 template systemd[1]: docker.service holdoff time over, scheduling restart.
Sep 09 19:15:24 template systemd[1]: Stopped Docker Application Container Engine.
-- Subject: Unit docker.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has finished shutting down.
Sep 09 19:15:24 template systemd[1]: start request repeated too quickly for docker.service
Sep 09 19:15:24 template systemd[1]: Failed to start Docker Application Container Engine.
-- Subject: Unit docker.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit docker.service has failed.

注意关键点 有一个报错

Status: unknown flag: --graph

通过查文档 graph-flags-on-dockerd 发现 这个 选项已经废弃了.

-g and --graph flags on dockerd

Deprecated In Release: v17.05

Removed In Release: v23.0

The -g or --graph flag for the dockerd or docker daemon command was used to indicate the directory in which to store persistent data and resource configuration and has been replaced with the more descriptive --data-root flag. These flags were deprecated and hidden in v17.05, and removed in v23.0.

这个选项 在 v23.0 以后就已经被移除了. 然后 看到说明文档 使用 --data-root 这个来代替.

重新修改 vim /usr/lib/systemd/system/docker.service

ExecStart=/usr/bin/dockerd --data-root /data/sdv1/docker-root   -H fd:// --containerd=/run/containerd/containerd.sock

添加 data-root 修改 /etc/docker/daemon.json

{
    // 新添加的内容 
   "data-root": "/data/sdv1/docker-root",  
   "registry-mirrors":["xxxxxxxxxxxx"]
}

再次 reload docker

systemctl daemon-reload
systemctl start docker

发现 还是没有起来 ,就继续看日志报错情况.

journalctl -xe 用来查看日志

错误日志 :

-- Unit docker.service has begun starting up.
1191 Sep 09 19:32:24 template dockerd[84272]: unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: data-root: (from flag: /data/sdv1/docker-root, from file: /data/sdv1/docker-root)
1192 Sep 09 19:32:24 template systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
1193 Sep 09 19:32:24 template systemd[1]: Failed to start Docker Application Container Engine.
1194 -- Subject: Unit docker.service has failed
1195 -- Defined-By: systemd
1196 -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
1197 --
1198 -- Unit docker.service has failed.
1199 --
1200 -- The result is failed.
1201 Sep 09 19:32:24 template systemd[1]: Unit docker.service entered failed state.
1202 Sep 09 19:32:24 template systemd[1]: docker.service failed.
1203 Sep 09 19:32:26 template systemd[1]: docker.service holdoff time over, scheduling restart.
1204 Sep 09 19:32:26 template systemd[1]: Stopped Docker Application Container Engine.
1205 -- Subject: Unit docker.service has finished shutting down
1206 -- Defined-By: systemd
1207 -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
1208 --
1209 -- Unit docker.service has finished shutting down.
1210 Sep 09 19:32:26 template systemd[1]: start request repeated too quickly for docker.service
1211 Sep 09 19:32:26 template systemd[1]: Failed to start Docker Application Container Engine.
1212 -- Subject: Unit docker.service has failed
1213 -- Defined-By: systemd
1214 -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
1215 --
1216 -- Unit docker.service has failed.

上面的报错日志 发现了这一条 : /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: data-root

说明 data-root 这个选项 配置重复了. 于是 我又重新修改了 配置文件

vim /usr/lib/systemd/system/docker.service

找到 ExecStart=/usr/bin/dockerd -H fd:// 类似的这样的一行, 删除 data-root 选项, 在 /etc/docker/daemon.json 这个配置文件中 留一个 dat-root 配置

ExecStart=/usr/bin/dockerd  -H fd:// --containerd=/run/containerd/containerd.sock

改完之后 , 再次 reload 重新启动

systemctl daemon-reload
systemctl start docker

终于启动起来了.

查看 docker 服务的启动情况

ps aux |grep dockerd 

查看 docker info root dir

docker info -f '{{ .DockerRootDir}}'

这里 不需要进行任何的配置 , 只要修改 /etc/docker/daemon.json 添加 相关配置即可

修改 daemon.json 文件

vim /etc/docker/daemon.json

最后 确保升级没有问题后,备份数据 防止出现问题.

mv  /var/lib/docker   /var/lib/docker.bak

总结

升级过程 遇到的问题,

第一: 搜索参考文档 比较久远 . 里面是 使用 --graph 来指定 位置 , 这个新版本 24.0 已经废弃了.

第二 : 还有在配置文件的时候 ,进行多项的重复配置 , 在service 中配置了data-root, 然后在 /etc/docker/daemon.json 同时进行了配置,导致 docker 没有办法起来.

不过这些通过查日志 解决了这个问题.

参考文档

https://blog.csdn.net/baidu_23433185/article/details/114398301

https://www.digitalocean.com/community/questions/how-to-move-the-default-var-lib-docker-to-another-directory-for-docker-on-linux

https://tehub.com/a/aRxLqus8aW

https://www.nodinrogers.com/post/2023-02-03-changing-docker-root-directory/

https://docs.docker.com/engine/deprecated/#-g-and—graph-flags-on-dockerd

https://docs.docker.com/engine/deprecated/

分享快乐,留住感动. '2023-09-10 10:56:53' --frank