微信公众号:leetcode_algos_life,代码随想随记
小红书:412408155
CSDN:https://blog.csdn.net/woai8339?type=blog ,代码随想随记
GitHub: https://github.com/riverind
抖音【暂未开始,计划开始】:tian72530,代码随想随记
知乎【暂未开始,计划开始】:happy001
问题
在进行qlora训练demo的时候,参考参考文档里的qlora进行训练。
前期步骤测试时候,出现报错,
报错信息如下:
AttributeError: module 'huggingface_hub.constants' has no attribute 'HF_HUB_CACHE'
测试代码如下:
# coding:utf-8
import json
import os
from pprint import pprint
import bitsandbytes as bnb
import pandas as pd
import torch
import torch.nn as nn
import transformers
from datasets import load_dataset
from huggingface_hub import notebook_login
from peft import (
LoraConfig,
PeftConfig,
PeftModel,
get_peft_model,
prepare_model_for_kbit_training,
)
from transformers import (
AutoConfig,
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
)
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
with open("Ecommerce_FAQ_Chatbot_dataset.json") as json_file:
data = json.load(json_file)
pprint(data["questions"][0], sort_dicts=False)
解决方案
博主目前没有在网上找到现成的解决方案,经过博主测试,如下解决:
pip install tiktoken==0.4.0
pip install transformers==4.36.0
pip install huggingface-hub==0.19.3
参考文档
1、qlora训练参考,https://github.com/curiousily/Get-Things-Done-with-Prompt-Engineering-and-LangChain/blob/master/07.falcon-qlora-fine-tuning.ipynb