huggingface大模型载入

子燕若水

已于 2023-02-10 21:00:31 修改

阅读量3.9k

点赞数 1

分类专栏：深度学习算法机器学习 cuda&深度学习环境文章标签： javascript 开发语言 ecmascript

于 2023-02-07 11:25:27 首次发布

本文链接：https://blog.csdn.net/u010087338/article/details/128915679

版权

深度学习算法同时被 3 个专栏收录

159 篇文章

订阅专栏

cuda&深度学习环境

87 篇文章

订阅专栏

机器学习

85 篇文章

订阅专栏

文章介绍了如何通过HuggingFace的Transformers库进行大模型的加载，特别是在资源有限的情况下。通过使用`device_map=auto`，模型可以自动分布在GPU和CPU上，甚至硬盘，以优化性能。对于内存受限的GPU，可以使用float16精度来降低内存需求。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Large model loading

ModelsWe’re on a journey to advance and democratize artificial intelligence through open source and open science.https://huggingface.co/docs/transformers/main_classes/model#large-model-loading

Moreover, you can directly place the model on different devices if it doesn’t fully fit in RAM (only works for inference for now). With device_map="auto", Accelerate will determine where to put each layer to maximize the use of your fastest devices (GPUs) and offload the rest on the CPU, or even the hard drive if you don’t have enough GPU RAM (or CPU RAM). Even if the model is split across several devices, it will run as you would normally expect.

When passing a device_map, low_cpu_mem_usage is automatically set to True, so you don’t need to specify it:

Copied

from transformers import AutoModelForSeq2SeqLM

t0pp = AutoModelForSeq2SeqLM.from_pretrained("bigscience/T0pp", device_map="auto")

You can inspect how the model was split across devices by looking at its hf_device_map attribute:

Note: If you are limited by GPU memory and have less than 10GB of GPU RAM available, please make sure to load the StableDiffusionPipeline in float16 precision instead of the default float32 precision as done above.

You can do so by loading the weights from the fp16 branch and by telling diffusers to expect the weights to be in float16 precision:

import torch
from diffusers import StableDiffusionPipeline

pipe = StableDiffusionPipeline.from_pretrained("CompVis/stable-diffusion-v1-4", revision="fp16", torch_dtype=to