不科学上网使用Hugging Face的Transformers库

109702008

于 2024-05-01 10:58:57 发布

阅读量1.8k

点赞数 57

分类专栏：人工智能 # 深度学习 # python 文章标签：人工智能 AIGC

本文链接：https://blog.csdn.net/eidolon_foot/article/details/138349573

版权

人工智能同时被 3 个专栏收录

119 篇文章 2 订阅

订阅专栏

python

92 篇文章 0 订阅

订阅专栏

深度学习

64 篇文章 0 订阅

订阅专栏

参考

Program Synthesis with CodeGen — ROCm Blogs (amd.com)

HF-Mirror - Huggingface 镜像站

https://huggingface.co/docs/transformers/v4.40.1/zh/installation#%E7%A6%BB%E7%BA%BF%E6%A8%A1%E5%BC%8F

准备

apt show rocm-libs -a
pip install transformers
python -m pip install huggingface_hub
export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --resume-download gpt2 --local-dir gpt2
wget https://hf-mirror.com/hfd/hfd.sh
chmod a+x hfd.sh
sudo snap install aria2c
sudo apt install git-lfs
./hfd.sh Salesforce/codegen-350M-mono  --tool aria2c -x 4

使用

(base) user@user-System:~$ python
Python 3.12.1 | packaged by Anaconda, Inc. | (main, Jan 19 2024, 15:51:05) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import time 
>>> from transformers import AutoModelForCausalLM, AutoTokenizer
>>> torch.set_default_device("cuda")
>>> start_time = time.time()
>>> checkpoint = "./codegen-350M-mono"
>>> model = AutoModelForCausalLM.from_pretrained(checkpoint)
>>> tokenizer = AutoTokenizer.from_pretrained(checkpoint)
>>> print(f"Loaded in {time.time() - start_time: .2f} seconds")
Loaded in  70.20 seconds
>>> print(model)
CodeGenForCausalLM(
  (transformer): CodeGenModel(
    (wte): Embedding(51200, 1024)
    (drop): Dropout(p=0.0, inplace=False)
    (h): ModuleList(
      (0-19): 20 x CodeGenBlock(
        (ln_1): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
        (attn): CodeGenAttention(
          (attn_dropout): Dropout(p=0.0, inplace=False)
          (resid_dropout): Dropout(p=0.0, inplace=False)
          (qkv_proj): Linear(in_features=1024, out_features=3072, bias=False)
          (out_proj): Linear(in_features=1024, out_features=1024, bias=False)
        )
        (mlp): CodeGenMLP(
          (fc_in): Linear(in_features=1024, out_features=4096, bias=True)
          (fc_out): Linear(in_features=4096, out_features=1024, bias=True)
          (act): NewGELUActivation()
          (dropout): Dropout(p=0.0, inplace=False)
        )
      )
    )
    (ln_f): LayerNorm((1024,), eps=1e-05, elementwise_affine=True)
  )
  (lm_head): Linear(in_features=1024, out_features=51200, bias=True)
)
>>> def run_inference(raw_input):
...     start_time = time.time()
...     inputs = tokenizer(raw_inputs, return_tensors="pt", return_attention_mask=False)
...     outputs = model.generate(**inputs,max_length=1000)
...     latency = time.time() - start_time
...     throughput = len(outputs[0]) / latency
...     print(f"Latency:    {latency: .2f} seconds")
...     print(f"Throughput: {throughput: .2f} tokens/s")
...     text = tokenizer.batch_decode(outputs)[0]
...     print(text)
... 
>>> raw_inputs = ''' 
... Given an integer array nums, return all the triplets [nums[i], nums[j], nums[k]] such that i != j, i != k, and j != k, and nums[i] + nums[j] + nums[k] == 0.
... 
... Notice that the solution set must not contain duplicate triplets.
... '''
>>> text = run_inference(raw_inputs)
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Latency:     12.16 seconds
Throughput:  42.94 tokens/s
 
Given an integer array nums, return all the triplets [nums[i], nums[j], nums[k]] such that i!= j, i!= k, and j!= k, and nums[i] + nums[j] + nums[k] == 0.

Notice that the solution set must not contain duplicate triplets.

Example 1:

Input: nums = [-1,0,1,2,-1,-4]
Output: [[-1,-1,2],[-1,0,1]]
Explanation:
-1 and -1 are triplets.
-1 and 0 are not triplets.
-1 and 1 are not triplets.
-4 and -1 are not triplets.
-4 and -1 are triplets.
-4 and 0 are not triplets.
-4 and 1 are triplets.
-1 and 2 are not triplets.

Example 2:

Input: nums = []
Output: []

Example 3:

Input: nums = [0]
Output: []

Constraints:

1 <= nums.length <= 104
-104 <= nums[i] <= 104

"""

class Solution:
    def threeSum(self, nums: List[int]) -> List[List[int]]:
        nums.sort()
        res = []
        for i in range(len(nums)):
            if i > 0 and nums[i] == nums[i-1]:
                continue
            l, r = i+1, len(nums)-1
            while l < r:
                if nums[i] + nums[l] + nums[r] == 0:
                    res.append([nums[i], nums[l], nums[r]])
                    while l < r and nums[l] == nums[l+1]:
                        l += 1
                    while l < r and nums[r] == nums[r-1]:
                        r -= 1
                    l += 1
                    r -= 1
                elif nums[i] + nums[l] + nums[r] > 0:
                    r -= 1
                else:
                    l += 1
        return res
<|endoftext|>
>>> raw_inputs = ''' 
... Given a 1-indexed array of integers numbers that is already sorted in non-decreasing order, find two numbers such that they add up to a specific target number. Let these two numbers be numbers[index1] and numbers[index2] where 1 <= index1 < index2 <= numbers.length.
... 
... Return the indices of the two numbers, index1 and index2, added by one as an integer array [index1, index2] of length 2.
... 
... The tests are generated such that there is exactly one solution. You may not use the same element twice.
... 
... Your solution must use only constant extra space.
... '''
>>> text = run_inference(raw_inputs)
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Latency:     8.65 seconds
Throughput:  61.84 tokens/s
 
Given a 1-indexed array of integers numbers that is already sorted in non-decreasing order, find two numbers such that they add up to a specific target number. Let these two numbers be numbers[index1] and numbers[index2] where 1 <= index1 < index2 <= numbers.length.

Return the indices of the two numbers, index1 and index2, added by one as an integer array [index1, index2] of length 2.

The tests are generated such that there is exactly one solution. You may not use the same element twice.

Your solution must use only constant extra space.

Example 1:

Input: numbers = [2,7,11,15], target = 9
Output: [1,2]
Explanation: The sum of 2 and 7 is 9. Therefore index1 = 1, index2 = 2.
Example 2:

Input: numbers = [2,3,4], target = 6
Output: [1,3]
Explanation: The sum of 2 and 3 is 6. Therefore index1 = 1, index2 = 3.
Example 3:

Input: numbers = [2,3,4], target = 18
Output: [1,3]
Explanation: The sum of 2 and 3 is 6. Therefore index1 = 1, index2 = 3.
Example 4:

Input: numbers = [2,3,4], target = 0
Output: [1,2]
Explanation: The sum of 2 and 0 is 0. Therefore index1 = 1, index2 = 2.
Example 5:

Input: numbers = [2,3,4], target = 10
Output: [1,3]
Explanation: The sum of 2 and 3 is 6. Therefore index1 = 1, index2 = 3.

Constraints:

1 <= numbers.length <= 10^4
-10^9 <= numbers[i] <= 10^9
-10^9 <= target <= 10^9

"""

class Solution:
    def twoSum(self, numbers: List[int], target: int) -> List[int]:
        for i in range(len(numbers)):
            for j in range(i+1, len(numbers)):
                if numbers[i] + numbers[j] == target:
                    return [i, j]
        return []
<|endoftext|>
>>> raw_inputs = ''' 
... Implement the cross entropy loss function
... '''
>>> text = run_inference(raw_inputs)
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Latency:     21.40 seconds
Throughput:  46.73 tokens/s
 
Implement the cross entropy loss function

"""

import torch
import torch.nn as nn
import torch.nn.functional as F

class CrossEntropyLoss(nn.Module):
    def __init__(self, ignore_index=255):
        super(CrossEntropyLoss, self).__init__()
        self.ignore_index = ignore_index

    def forward(self, inputs, targets):
        n = inputs.size(0)
        log_probs = F.log_softmax(inputs, dim=1)
        loss = -torch.sum(targets * log_probs, dim=1)
        loss = torch.mean(loss)
        return loss

class DiceLoss(nn.Module):
    def __init__(self, ignore_index=255):
        super(DiceLoss, self).__init__()
        self.ignore_index = ignore_index

    def forward(self, inputs, targets):
        n = inputs.size(0)
        log_probs = F.log_softmax(inputs, dim=1)
        loss = -torch.sum(targets * log_probs, dim=1)
        loss = torch.mean(loss)
        return loss

class DiceLoss2(nn.Module):
    def __init__(self, ignore_index=255):
        super(DiceLoss2, self).__init__()
        self.ignore_index = ignore_index

    def forward(self, inputs, targets):
        n = inputs.size(0)
        log_probs = F.log_softmax(inputs, dim=1)
        loss = -torch.sum(targets * log_probs, dim=1)
        loss = torch.mean(loss)
        return loss

class DiceLoss3(nn.Module):
    def __init__(self, ignore_index=255):
        super(DiceLoss3, self).__init__()
        self.ignore_index = ignore_index

    def forward(self, inputs, targets):
        n = inputs.size(0)
        log_probs = F.log_softmax(inputs, dim=1)
        loss = -torch.sum(targets * log_probs, dim=1)
        loss = torch.mean(loss)
        return loss

class DiceLoss4(nn.Module):
    def __init__(self, ignore_index=255):
        super(DiceLoss4, self).__init__()
        self.ignore_index = ignore_index

    def forward(self, inputs, targets):
        n = inputs.size(0)
        log_probs = F.log_softmax(inputs, dim=1)
        loss = -torch.sum(targets * log_probs, dim=1)
        loss = torch.mean(loss)
        return loss

class DiceLoss5(nn.Module):
    def __init__(self, ignore_index=255):
        super(DiceLoss5, self).__init__()
        self.ignore_index = ignore_index

    def forward(self, inputs, targets):
        n = inputs.size(0)
        log_probs = F.log_softmax(inputs, dim=1)
        loss = -torch.sum(targets * log_probs, dim=1)
        loss = torch.mean(loss)
        return loss

class DiceLoss6(nn.Module):
    def __init__(self, ignore_index=255):
        super(DiceLoss6, self).__init__()
        self.ignore_index = ignore_index

    def forward(self, inputs, targets):
        n = inputs.size(0)
        log_probs = F.log_softmax(inputs, dim=1)
        loss = -torch.sum(targets * log_probs, dim=1)
        loss = torch.mean(loss)
        return loss

class DiceLoss7(nn.Module):
    def __init__(self, ignore_index=255):
        super(DiceLoss7,
>>> 
>>> 
>>> 
>>> raw_inputs = ''' 
... Implement binary search
... '''
>>> text = run_inference(raw_inputs)
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Latency:     2.91 seconds
Throughput:  48.42 tokens/s
 
Implement binary search
"""

def binary_search(arr, target):
    low = 0
    high = len(arr) - 1
    while low <= high:
        mid = (low + high) // 2
        if arr[mid] == target:
            return mid
        elif arr[mid] < target:
            low = mid + 1
        else:
            high = mid - 1
    return -1

arr = [1,2,3,4,5,6,7,8,9,10]
target = 10
print(binary_search(arr, target))<|endoftext|>
>>> exit()

ChatGPT

Hugging Face 的 Transformers 是一个开源的库，它用于自然语言处理（NLP）任务，并利用各种预训练模型。即使没有直接访问 huggingface.co，仍有多种方法可以尝试使用 Transformers 库。
以下是在不科学上网（即在不使用任何特殊网络措施以规避网络限制的情况下）情况下尝试使用 Hugging Face Transformers 的一些建议：
1. PyPI 安装
如果可以正常访问 Python Package Index (PyPI)，可以直接使用 pip 安装 Transformers：

pip install transformers

这将安装最新版本的 Transformers 库以及其依赖。
2. 离线安装
如果无法访问 PyPI，可以尝试从其他有互联网的地方下载 Transformers 库的 wheel 文件或源代码。然后，可以将其拷贝到您的机器上进行安装。对于 wheel 文件，使用命令：

   pip install /path/to/transformers.whl

如果是源代码包，可以解压后在包根目录下运行：

   pip install .

3. 使用镜像站点
某些国家和地区的开发者可能会因为网络限制而使用国内的镜像站点来安装Python包。例如，中国大陆的开发者可以使用清华大学的镜像站点：

   pip install transformers -i https://pypi.tuna.tsinghua.edu.cn/simple

4. 通过GitHub获取
如果还可以访问 GitHub，可以克隆或下载 Transformers 库的仓库：

   git clone https://github.com/huggingface/transformers.git

下载后，可以按照离线安装的步骤在本地安装仓库代码。
请注意，某些功能可能需要下载额外的预训练模型。如果也不能直接下载这些模型，可能需要找到模型文件的离线版本或其他替代的下载方法。一旦拥有这些文件，可以在库中设置环境变量或代码内部变量，让库使用本地路径来加载模型。例如，可以使用以下环境变量：

export TRANSFORMERS_OFFLINE=1

或在 Python 代码中指定模型的本地路径：

from transformers import BertModel

model = BertModel.from_pretrained("/path/to/local/model/directory")

请务必遵守相应的法律法规，合法地使用网络资源。

通义灵码

在没有科学上网的情况下，使用Hugging Face的Transformers库可能会遇到访问Hugging Face Model Hub或更新库时网络连接问题。但是，仍然有几种方法可以尝试：

离线安装库：在能够访问互联网的环境中，可以先下载Transformers库的最新版本，然后在无网络的环境中安装。在有网的机器上运行以下命令：

pip download transformers

这会下载所有必要的文件。然后，将这些文件传输到无网环境，使用pip install命令进行安装：

pip install --no-index --find-links=path/to/downloaded/packages transformers

其中，path/to/downloaded/packages是下载的文件所在的目录。

本地镜像源：如果在中国，可以使用国内的镜像源来加速下载。例如，阿里云提供了Pypi的镜像，可以在有网的时候配置：

   pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
   pip install transformers

安装完成后，在无网络环境中，Transformers库已经可以使用，除非需要更新模型或者库本身。

提前下载模型：如果只需要特定的预训练模型，可以在有网时使用transformers库的save_pretrained方法下载并保存到本地：

   from transformers import AutoModel, AutoTokenizer
   model_name = "bert-base-chinese"
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   model = AutoModel.from_pretrained(model_name)
   tokenizer.save_pretrained("path/to/save/tokenizer")
   model.save_pretrained("path/to/save/model")

然后在无网络环境中，可以使用from_pretrained加载本地的模型和tokenizer。

克隆Git仓库： Hugging Face的Transformers库是开源的，可以直接从GitHub克隆整个库到本地：

git clone https://github.com/huggingface/transformers.git

然后在项目中引用本地的库文件，而不是从PyPI安装。

请注意，即使在本地安装了库，如果需要下载新的模型或者更新库，仍然需要网络连接。如果模型已经下载并保存在本地，那么在无网络环境中使用已有的模型是可以的。

讯飞星火

要在不科学上网的情况下使用Hugging Face的Transformers库，可以通过设置环境变量或使用huggingface-cli命令行工具来实现。具体如下：

设置环境变量：可以设置HF_ENDPOINT环境变量来指定Hugging Face Hub的镜像地址。例如，可以将其设置为https://hf-mirror.com，这是一个常用的Hugging Face Hub的国内镜像地址。
使用huggingface-cli：可以通过安装huggingface-cli命令行工具来下载模型。安装完成后，可以使用huggingface-cli download命令下载模型，并指定--local-dir参数来设置模型的本地存储路径。同时，使用--resume-download参数可以在下载中断后继续下载，而--local-dir-use-symlinks参数用于指定是否使用符号链接。

此外，为了确保安装过程顺利，可以尝试通过清华大学的镜像源来安装所需的依赖包，这样可以加快下载速度并避免潜在的网络问题。

总之，通过上述方法，即使不科学上网，也可以在国内正常使用Hugging Face的Transformers库进行自然语言处理相关的研究和开发工作。

Kimi

如果无法访问互联网或希望在不使用科学上网（即翻墙）的情况下使用Hugging Face的Transformers库，可以通过以下几种方法来实现：

1. 离线模式

启用离线模式，以防止库尝试从Hugging Face的在线Hub下载任何模型或文件。可以通过设置环境变量TRANSFORMERS_OFFLINE=1来实现这一点。

export TRANSFORMERS_OFFLINE=1

2. 本地缓存

如果模型和分词器已经下载到本地，可以直接从本地缓存加载它们。确保有模型的所有文件，包括配置文件、权重文件等。

from transformers import AutoModel, AutoTokenizer

model_path = "./local/path/to/model"
tokenizer_path = "./local/path/to/tokenizer"

tokenizer = AutoTokenizer.from_pretrained(tokenizer_path)
model = AutoModel.from_pretrained(model_path)

3. 提前下载模型

在有网络访问的时候，提前下载所需的模型和分词器，并保存到本地。

from transformers import AutoModel, AutoTokenizer

# 请替换为您想下载的模型名
model_name = "bert-base-uncased"
tokenizer_name = model_name

tokenizer = AutoTokenizer.from_pretrained(tokenizer_name)
model = AutoModel.from_pretrained(model_name)

# 保存到本地路径
save_path = "./local/path/to/save"
tokenizer.save_pretrained(save_path)
model.save_pretrained(save_path)

4. 使用代理或VPN

如果只是临时无法访问某些服务，可以考虑使用代理或VPN来解决网络问题。

5. 使用国内镜像

对于某些依赖，可以尝试使用国内的镜像源来加速下载，比如使用中国大陆的PyPI镜像来安装Python包。

6. 使用Docker

如果使用的是Docker，并且需要拉取Hugging Face的Transformers库的镜像，可能需要配置Docker以使用代理。

7. 手动下载

对于大型模型或数据集，如果Hugging Face Hub提供了直接下载链接，可以手动下载模型文件到本地，然后按照上述方法加载。

注意

在使用离线模式时，将无法访问任何在线资源，包括模型权重、分词器、配置文件等。
如果在没有网络的环境中工作，需要提前准备好所有必要的文件和依赖。
确保遵守所有相关的使用条款和版权法规，不要违反Hugging Face或任何其他资源提供者的服务条款。

通过上述方法，即使在没有互联网连接的情况下，也可以使用Hugging Face的Transformers库。

109702008

关注

57
点赞
踩
21

收藏

觉得还不错? 一键收藏
打赏
2
评论
不科学上网使用Hugging Face的Transformers库

总之，通过上述方法，即使不科学上网，也可以在国内正常使用Hugging Face的Transformers库进行自然语言处理相关的研究和开发工作。如果使用的是Docker，并且需要拉取Hugging Face的Transformers库的镜像，可能需要配置Docker以使用代理。此外，为了确保安装过程顺利，可以尝试通过清华大学的镜像源来安装所需的依赖包，这样可以加快下载速度并避免潜在的网络问题。通过上述方法，即使在没有互联网连接的情况下，也可以使用Hugging Face的Transformers库。
复制链接

扫一扫