- 搞大模型训练的工作不可避免地需要很多库,但是非常讨厌的事情是这些库动不动就不兼容。最近在做文本分类训练的时候又遇到了这个问题,为了避免后面再安装包的时候把我之前的环境破坏了,所以特地来记录一下:
protobuf、streamlit、transformers、icetk、cpm_kernels
版本号
(venv_net) PS D:\mydatapro\myweb> pip show protobuf streamlit transformers icetk cpm_kernels
Name: protobuf
Version: 3.20.3
Summary: Protocol Buffers
Home-page: https://developers.google.com/protocol-buffers/
Author:
Author-email:
License: BSD-3-Clause
Location: D:\mydatapro\venv_net\Lib\site-packages
Requires:
Required-by: googleapis-common-protos, onnxruntime, opentelemetry-proto, streamlit
---
Name: streamlit
Version: 1.17.0
Summary: The fastest way to build data apps in Python
Home-page: https://streamlit.io
Author: Snowflake Inc
Author-email: hello@streamlit.io
License: Apache License 2.0
Location: D:\mydatapro\venv_net\Lib\site-packages
Requires: altair, blinker, cachetools, click, gitpython, importlib-metadata, numpy, packaging, pandas, pillow, protobuf, pyarrow, pydeck, pympler, python-dateutil, requests, rich, semver, toml, tornado, typing-extensions, tzlocal, validators, watchdog
Required-by:
---
Name: transformers
Version: 4.27.1
Summary: State-of-the-art Machine Learning for JAX, PyTorch and TensorFlow
Home-page: https://github.com/huggingface/transformers
Author: The Hugging Face team (past and future) with the help of all our contributors (https://github.com/huggingface/transformers/graphs/contributors)
Author-email: transformers@huggingface.co
License: Apache 2.0 License
Location: D:\mydatapro\venv_net\Lib\site-packages
Requires: filelock, huggingface-hub, numpy, packaging, pyyaml, regex, requests, tokenizers, tqdm
Required-by: sentence-transformers
---
Name: icetk
Version: 0.0.3
Summary: A unified tokenization tool for Images, Chinese and English.
Home-page: https://github.com/THUDM/icetk
Author: Wendi Zheng and Ming Ding
Author-email:
License: MIT license
Location: D:\mydatapro\venv_net\Lib\site-packages
Requires: requests, sentencepiece, torchvision, tqdm
Required-by:
---
Name: cpm-kernels
Version: 1.0.11
Summary: CPM CUDA kernels
Home-page: UNKNOWN
Author:
Author-email:
License: Apache 2.0
Location: D:\mydatapro\venv_net\Lib\site-packages
Requires:
Required-by:
检查是否有冲突:pip check
![在这里插入图片描述](https://i-blog.csdnimg.cn/direct/599a89891a394aeda5bf97c9c0b338c6.png#pic_center)