conda create -n baichuan2 python=3.10
环境:
cuda12.1
torch2.2.1-cu121
bitsandbytes-0.41.1 用windows版本
pip install https://jllllll.github.io/bitsandbytes-windows-webui/bitsandbytes/bitsandbytes-0.41.1-py3-none-win_amd64.whl
accelerate 0.25.0
transformers 4.33.2
运行测试:
python cli_demo.py
运行web:
streamlit run web_demo.py