MobileSAM 项目排坑

任务

把MobileSAM这个项目跑通，明天就可以集中学习SAM、MobileSAM、EfficientSAM和Segformer的论文和代码了。

过程记录

创建环境

老样子：

git clone https://github.com/ChaoningZhang/MobileSAM.git
cd MobileSAM/
conda create -n MOB_SAM python=3.10
conda activate MOB_SAM
pip3 install empy rospkg pyyaml catkin_pkg
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
pip3 install -e.
pip3 install gradio

纯python，没有C++的项目，环境简单多了。

交互式测试

按照readme指示进行：

cd app/
python app.py

报错：

Traceback (most recent call last):
  File "/home/lcy-magic/Segment_TEST/MobileSAM/app/app.py", line 3, in <module>
    import gradio as gr
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/__init__.py", line 3, in <module>
    import gradio._simple_templates
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/_simple_templates/__init__.py", line 1, in <module>
    from .simpledropdown import SimpleDropdown
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/_simple_templates/simpledropdown.py", line 6, in <module>
    from gradio.components.base import FormComponent
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/components/__init__.py", line 1, in <module>
    from gradio.components.annotated_image import AnnotatedImage
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/components/annotated_image.py", line 14, in <module>
    from gradio.components.base import Component
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/components/base.py", line 20, in <module>
    from gradio.blocks import Block, BlockContext
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/blocks.py", line 29, in <module>
    from gradio import (
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/networking.py", line 13, in <module>
    from gradio.routes import App  # HACK: to avoid circular import # noqa: F401
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/gradio/routes.py", line 137, in <module>
    client = httpx.AsyncClient()
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/httpx/_client.py", line 1429, in __init__
    proxy_map = self._get_proxy_map(proxies or proxy, allow_env_proxies)
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/httpx/_client.py", line 217, in _get_proxy_map
    return {
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/httpx/_client.py", line 218, in <dictcomp>
    key: None if url is None else Proxy(url=url)
  File "/home/lcy-magic/anaconda3/envs/MOB_SAM/lib/python3.10/site-packages/httpx/_config.py", line 336, in __init__
    raise ValueError(f"Unknown scheme for proxy URL {url!r}")
ValueError: Unknown scheme for proxy URL URL('socks://127.0.0.1:7890/')

根据参考博客所说，应该把系统代理的环境变量清空。害，不太懂网络，我先把我现在的环境变量备份一下：

echo $all_proxy
socks://127.0.0.1:7890/
echo $all_proxy
socks://127.0.0.1:7890/

然后再：

export all_proxy=''
export ALL_PROXY=''

再运行，报错：

ModuleNotFoundError: No module named 'timm'

安装嘛：

pip3 install timm

然后报错：

WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f219e5f6e30>: Failed to establish a new connection: [Errno 111] 拒绝连接'))': /simple/timm/
ERROR: Could not find a version that satisfies the requirement timm (from versions: none)
ERROR: No matching distribution found for timm

我觉得是网络代理的问题。那好说，因为export只是在当前终端中有效。于是我开一个新终端安装：

conda activate MOB_SAM
pip3 install timm

这会执行又说我没opencv：

ModuleNotFoundError: No module named 'cv2'

于是在新终端安装：

pip3 install opencv-contrib-python

再运行：

python app.py

成功：

ctrl+点击那个ip，就可以在浏览器上看到页面了，可以交互式体验下算法效果：

notebook

因为readme没说哪个是测试脚本，只给了基础用法。那我就先按照Jupyter Notebook来。
装一下：

pip3 install ipykernel ipython

然后跳过enviroment setup部分，执行其他的：
import必要的包，定义一个展示分割后图像的函数：

读取并展示图像：

加载模型和模型参数：

计算mask并可视化：

预处理和后处理参数调整的教学：

predictor_example.ipynb这个脚本也能直接运行，就不废话了。

V2测试

把那个权重文件下载了，解压放到MobileSAMv2/weight里。
按照readme执行：

cd MobileSAMv2
bash ./experiments/mobilesamv2.sh

先后报错：

No module named 'torchpack'
No module named 'onnx'
No module named 'onnxsim'

新终端中安装：

pip3 install torchpack
pip3 install onnx
pip3 install onnxsim

再运行报错：

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.25 GiB. GPU 0 has a total capacity of 7.79 GiB of which 591.62 MiB is free. Process 91813 has 4.39 GiB memory in use. Process 93620 has 646.00 MiB memory in use. Including non-PyTorch memory, this process has 2.06 GiB memory in use. Of the allocated memory 1.79 GiB is allocated by PyTorch, and 44.67 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

没显存了。。。
尝试了下，就算把batch_size改为1也没用。那只能先这样了，看看今晚能不能做到只用核显负责桌面，多让出点独显显存。

欸！不用！我发现我有两个python进程吃了我4G显存，我把他们都kill掉，然后再运行就好了：

watch -n 5 nvidia-smi
kill -15 91813
kill -15 93620
bash ./experiments/mobilesamv2.sh

我现在纯桌面其实占用显存并不多，感觉没必要折腾了：

可惜，这个模型的训练代码同样没有开源。coming soon中。