上午想尝试一下 BLIP-2 在自建图像数据集上的描述生成效果,但由于当前环境的python版本不能够支持最新版本的 transformers 库,于是新建一个 python 3.7 的环境在其上安装较新版本的 transformers 库。
但在使用命令pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
安装torch
过程中,报错如下:
(python3.7) ╭─root@1c113923969c /
╰─# pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
ERROR: Exception:
Traceback (most recent call last):
File "/root/anaconda3/envs/python3.7/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
status = run_func(*args)
File "/root/anaconda3/envs/python3.7/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
return func(self, options, args)
File "/root/anaconda3/envs/python3.7/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 354, in run
build_tracker = self.enter_context(get_build_tracker())
File "/root/anaconda3/envs/python3.7/lib/python3.7/site-packages/pip/_internal/cli/command_context.py", line 27, in enter_context
return self._main_context.enter_context(context_provider)
File "/root/anaconda3/envs/python3.7/lib/python3.7/contextlib.py", line 426, in enter_context
result = _cm_type.__enter__(cm)
File "/root/anaconda3/envs/python3.7/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/root/anaconda3/envs/python3.7/lib/python3.7/site-packages/pip/_internal/operations/build/build_tracker.py", line 46, in get_build_tracker
root = ctx.enter_context(TempDirectory(kind="build-tracker")).path
File "/root/anaconda3/envs/python3.7/lib/python3.7/site-packages/pip/_internal/utils/temp_dir.py", line 125, in __init__
path = self._create(kind)
File "/root/anaconda3/envs/python3.7/lib/python3.7/site-packages/pip/_internal/utils/temp_dir.py", line 164, in _create
path = os.path.realpath(tempfile.mkdtemp(prefix=f"pip-{kind}-"))
File "/root/anaconda3/envs/python3.7/lib/python3.7/tempfile.py", line 356, in mkdtemp
prefix, suffix, dir, output_type = _sanitize_params(prefix, suffix, dir)
File "/root/anaconda3/envs/python3.7/lib/python3.7/tempfile.py", line 126, in _sanitize_params
dir = gettempdir()
File "/root/anaconda3/envs/python3.7/lib/python3.7/tempfile.py", line 294, in gettempdir
tempdir = _get_default_tempdir()
File "/root/anaconda3/envs/python3.7/lib/python3.7/tempfile.py", line 229, in _get_default_tempdir
dirlist)
**FileNotFoundError: [Errno 2] No usable temporary directory found in ['/tmp', '/var/tmp', '/usr/tmp', '/']**
使用cd /
命令切换到根目录,然后使用df
命令查看文件系统的磁盘使用情况:
发现overlay
一行的磁盘100G全部占满,即利用率达到了 100%. 由于我是在/root
目录下新建的环境,所以首先想到的解决办法是先remove
一些不再使用的虚拟环境,使用conda remove -n tf --all
命令,然而发现报错:
(python3.7) ╭─root@1c113923969c ~/.cache
╰─# conda remove -n tf --all
Remove all packages in environment /root/anaconda3/envs/tf:
## Package Plan ##
environment location: /root/anaconda3/envs/tf
The following packages will be REMOVED:
_libgcc_mutex-0.1-main
_openmp_mutex-5.1-1_gnu
ca-certificates-2022.07.19-h06a4308_0
certifi-2021.5.30-py36h06a4308_0
ld_impl_linux-64-2.38-h1181459_1
libffi-3.3-he6710b0_2
libgcc-ng-11.2.0-h1234567_1
libgomp-11.2.0-h1234567_1
libstdcxx-ng-11.2.0-h1234567_1
ncurses-6.3-h5eee18b_3
openssl-1.1.1q-h7f8727e_0
pip-21.2.2-py36h06a4308_0
python-3.6.12-hcff3b4d_2
readline-8.1.2-h7f8727e_1
setuptools-58.0.4-py36h06a4308_0
sqlite-3.39.2-h5082296_0
tk-8.6.12-h1ccaba5_0
wheel-0.37.1-pyhd3eb1b0_0
xz-5.2.5-h7f8727e_1
zlib-1.2.12-h7f8727e_2
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
ERROR conda.core.link:_execute(698): An error occurred while uninstalling package 'defaults/linux-64::_libgcc_mutex-0.1-main'.
Rolling back transaction: done
[Errno 122] Disk quota exceeded: '/root/anaconda3/envs/tf/conda-meta/_libgcc_mutex-0.1-main.json' -> '/root/anaconda3/envs/tf/conda-meta/_libgcc_mutex-0.1-main.json.c~'
()
说明删除操作也因为磁盘占满无法正常运行。于是使用cd /root
命令切换到root
目录,然后使用du -sh *
命令查看root
目录下每个文件夹的大小:
对上述几个文件夹的大小简单计算发现,上述显示出的文件夹占用存储还不到 60G, 怀疑是隐藏文件的原因。于是使用du -sh .[!.]*
命令查看/root
目录下全部隐藏文件夹的大小:
才发现在根目录下有一个隐藏文件夹 /root/.cache/
,其容量竟然达到了 49G(如上图中红框所示)!于是问题变成对该目录下的文件进行清理
。再次使用du -sh *
命令查看.cache
目录下的文件夹大小:
切换到.cache
文件夹下,选择文件进行清理,使用命令rm -rf pip
对pip
目录下的内容进行删除,之后再使用df -h
命令查看文件系统磁盘使用情况:
发现此时overlay
的磁盘占用率降至93%
, 此时再次使用命令pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html
安装torch
:
至此,就可以啦!
下面给出几种**.cache文件清理方法**以供参考:
- 找到日期大于365天的文件,直接删除之,命令:
find ~/.cache/ -type f -atime +365 -delete
- 找到大于10M的文件,命令:
find ~/.cache/ -size +10M
,然后酌情清理之。 - 列出体积大于100M的目录,命令:
du ~/.cache -t 100M
,然后酌情清理之。
参考资料