运行Grounded-Segment-Anything提供的demo:grounded_sam_demo.py时报错如下:
/opt/conda/lib/python3.10/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525552843/work/aten/src/ATen/native/TensorShape.cpp:3190.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
final text_encoder_type: bert-base-uncased
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/opt/conda/lib/python3.10/site-packages/urllib3/util/connection.py", line 95, in create_connection
raise err
File "/opt/conda/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
self._validate_conn(conn)
File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
conn.connect()
File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
self.sock = conn = self._new_conn()
File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7fb023fbf550>: Failed to establish a new connection: [Errno 101] Network is unreachable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/opt/conda/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-uncased/resolve/main/config.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fb023fbf550>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1238, in hf_hub_download
metadata = get_hf_file_metadata(
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1631, in get_hf_file_metadata
r = _request_wrapper(
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 385, in _request_wrapper
response = _request_wrapper(
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 408, in _request_wrapper
response = get_session().request(method=method, url=url, **params)
File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 67, in send
return super().send(request, *args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 565, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-uncased/resolve/main/config.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fb023fbf550>: Failed to establish a new connection: [Errno 101] Network is unreachable'))"), '(Request ID: 628fb20e-209f-4f39-bdd7-1df304219c50)')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/transformers/utils/hub.py", line 389, in cached_file
resolved_file = hf_hub_download(
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1371, in hf_hub_download
raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/appuser/Grounded-Segment-Anything/grounded_sam_demo.py", line 189, in <module>
model = load_model(config_file, grounded_checkpoint, device=device)
File "/home/appuser/Grounded-Segment-Anything/grounded_sam_demo.py", line 50, in load_model
model = build_model(args)
File "/home/appuser/Grounded-Segment-Anything/GroundingDINO/groundingdino/models/__init__.py", line 17, in build_model
model = build_func(args)
File "/home/appuser/Grounded-Segment-Anything/GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py", line 372, in build_groundingdino
model = GroundingDINO(
File "/home/appuser/Grounded-Segment-Anything/GroundingDINO/groundingdino/models/GroundingDINO/groundingdino.py", line 107, in __init__
self.tokenizer = get_tokenlizer.get_tokenlizer(text_encoder_type)
File "/home/appuser/Grounded-Segment-Anything/GroundingDINO/groundingdino/util/get_tokenlizer.py", line 17, in get_tokenlizer
tokenizer = AutoTokenizer.from_pretrained(text_encoder_type)
File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 752, in from_pretrained
config = AutoConfig.from_pretrained(
File "/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1082, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/transformers/configuration_utils.py", line 644, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/transformers/configuration_utils.py", line 699, in _get_config_dict
resolved_config_file = cached_file(
File "/opt/conda/lib/python3.10/site-packages/transformers/utils/hub.py", line 429, in cached_file
raise EnvironmentError(
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like bert-base-uncased is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
重点在最后两句:
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like bert-base-uncased is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
因为远程的服务器无法连接到huggingface,所以需要离线运行。
解决办法是:
1 模型文件下载到本地
2 修改get_tokenlizer.py
1:
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
final text_encoder_type: bert-base-uncased
报错信息中提到的是bert-base-uncased这个模型,所以到huggingface网站去把这个模型相关内容下载下来(pytorch_model.bin、vocab.txt、config.json):
手动下载后同步到远程服务器,还放到Grounded-Segment-Anything这个项目文件夹中:
mat@matcloud:~$ cd /mnt/Grounded-Segment-Anything
mat@matcloud:/mnt/Grounded-Segment-Anything$ ls
assets grounded_sam_3d_box.ipynb groundingdino_swint_ogc.pth
automatic_label_demo.py grounded_sam_colab_demo.ipynb LICENSE
automatic_label_ram_demo.py grounded_sam_demo.py Makefile
automatic_label_simple_demo.py grounded_sam_inpainting_demo.py outputs
automatic_label_tag2text_demo.py grounded_sam.ipynb playground
bert-base-uncased grounded-sam-osx predict.py
bert-base-uncased.rar grounded_sam_osx_demo.py README.md
chatbot.py grounded_sam_simple_demo.py recognize-anything
CITATION.cff grounded_sam_visam.py requirements.txt
cog.yaml grounded_sam_whisper_demo.py sam_vit_h_4b8939.pth
Dockerfile grounded_sam_whisper_inpainting_demo.py segment_anything
EfficientSAM GroundingDINO VISAM
gradio_app.py grounding_dino_demo.py voxelnext_3d_box
mat@matcloud:/mnt/Grounded-Segment-Anything$ ls bert-base-uncased
config.json pytorch_model.bin vocab.txt
2:
由错误信息中提到的这一行:
Anything/GroundingDINO/groundingdino/util/get_tokenlizer.py", line 17, in get_tokenlizer
tokenizer = AutoTokenizer.from_pretrained(text_encoder_type)
找到加载bert模型的部分应该是在/GroundingDINO/groundingdino/util/get_tokenlizer.py这个文件中。打开这个文件,修改相应部分确保使用本地路径。
def get_tokenlizer(text_encoder_type, local_files_only=False):
# ...原有的代码...
# 修改这一行以支持从本地路径加载
tokenizer = AutoTokenizer.from_pretrained(text_encoder_type, local_files_only=local_files_only)
return tokenizer
def get_pretrained_language_model(text_encoder_type, local_files_only=False):
if text_encoder_type == "bert-base-uncased":
return BertModel.from_pretrained(text_encoder_type, local_files_only=local_files_only)
if text_encoder_type == "roberta-base":
return RobertaModel.from_pretrained(text_encoder_type, local_files_only=local_files_only)
raise ValueError("Unknown text_encoder_type {}".format(text_encoder_type))
修改后的代码:
from transformers import AutoTokenizer, BertModel, BertTokenizer, RobertaModel, RobertaTokenizerFast
def get_tokenlizer(text_encoder_type, local_files_only=False): #改这里
if not isinstance(text_encoder_type, str):
# print("text_encoder_type is not a str")
if hasattr(text_encoder_type, "text_encoder_type"):
text_encoder_type = text_encoder_type.text_encoder_type
elif text_encoder_type.get("text_encoder_type", False):
text_encoder_type = text_encoder_type.get("text_encoder_type")
else:
raise ValueError(
"Unknown type of text_encoder_type: {}".format(type(text_encoder_type))
)
print("final text_encoder_type: {}".format(text_encoder_type))
tokenizer = AutoTokenizer.from_pretrained(text_encoder_type, local_files_only=local_files_only) #改这里
return tokenizer
def get_pretrained_language_model(text_encoder_type, local_files_only=False): #改这里
if text_encoder_type == "bert-base-uncased":
return BertModel.from_pretrained(text_encoder_type, local_files_only=local_files_only)
if text_encoder_type == "roberta-base":
return RobertaModel.from_pretrained(text_encoder_type, local_files_only=local_files_only) #改这里
raise ValueError("Unknown text_encoder_type {}".format(text_encoder_type))
完美解决😊
root@b90c7258b1d1:/home/appuser/Grounded-Segment-Anything# export CUDA_VISIBLE_DEVICES=0
root@b90c7258b1d1:/home/appuser/Grounded-Segment-Anything# python3 grounded_sam_demo.py \
> --config GroundingDINO/groundingdino/config/GroundingDINO_SwinT_OGC.py \
> --grounded_checkpoint groundingdino_swint_ogc.pth \
> --sam_checkpoint sam_vit_h_4b8939.pth \
> --input_image assets/demo1.jpg \
> --output_dir "outputs" \
> --box_threshold 0.3 \
> --text_threshold 0.25 \
> --text_prompt "bear" \
> --device "cuda"
/opt/conda/lib/python3.10/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1670525552843/work/aten/src/ATen/native/TensorShape.cpp:3190.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
final text_encoder_type: bert-base-uncased
_IncompatibleKeys(missing_keys=[], unexpected_keys=['label_enc.weight', 'bert.embeddings.position_ids'])
/opt/conda/lib/python3.10/site-packages/transformers/modeling_utils.py:942: FutureWarning: The `device` argument is deprecated and will be removed in v5 of Transformers.
warnings.warn(
/opt/conda/lib/python3.10/site-packages/torch/utils/checkpoint.py:31: UserWarning: None of the inputs have requires_grad=True. Gradients will be None
warnings.warn("None of the inputs have requires_grad=True. Gradients will be None")
root@b90c7258b1d1:/home/appuser/Grounded-Segment-Anything# cd outputs
root@b90c7258b1d1:/home/appuser/Grounded-Segment-Anything/outputs# ls
grounded_sam_output.jpg mask.jpg mask.json raw_image.jpg