Python = 3.8.11, cuda=11.8
安装Docker及NVIDIA Docker的教程:https://blog.csdn.net/frontiers/article/details/129732630?spm=1001.2014.3001.5501
FasterTransformer运行GPT的官方教程:https://github.com/NVIDIA/FasterTransformer/blob/main/docs/gpt_guide.md#build-the-fastertransformer
使用pre-build Docker创建Docker:
#tensorflow 模型
nvidia-docker run -ti --shm-size 5g --rm nvcr.io/nvidia/tensorflow:22.09-tf1-py3 bash
#torch模型
nvidia-docker run -ti --shm-size 5g --rm nvcr.io/nvidia/pytorch:22.09-py3 bash
git clone
git clone https://github.com/NVIDIA/FasterTransformer.git
mkdir -p FasterTransformer/build
cd FasterTransformer/build
git submodule init && git submodule update
由于我用的torch模型,下面的步骤只保留torch的,如果需要tensorflow可以去官方guide看
build with torch on A100:
cmake -DSM=80 -DCMAKE_BUILD_TYPE=Release -DBUILD_PYT=ON -DBUILD_MULTI_GPU=ON ..
make -j12
pip install -r ../examples/pytorch/gpt/requirement.txt
wget https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-vocab.json -P ../models
wget https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-merges.txt -P ../models
mkdir -p ../models/openai-gpt-models/
#下载GPT2 model
python ../examples/tensorflow/gpt/utils/download_gpt2_model.py 124M
mv models/124M ../models/openai-gpt-models/
python ../examples/tensorflow/gpt/utils/openai_gpt_ckpt_converter.py -o ../models/openai-gpt-models/c-model/124m/ -i ../models/openai-gpt-models/124M/model.ckpt -g 1
运行GPT2 model
python ../examples/pytorch/gpt/multi_gpu_gpt_example.py
不要使用gpt_example.py, it's deprecated.
这里可以进到multi_gpu_gpt_example.py里面改参数。同样可以在这个文件里看到fake input是怎么生成的。我这里的参数:
=================== Arguments ===================
layer_num.....................: 12
input_len.....................: 512
output_len....................: 32
head_num......................: 12
size_per_head.................: 64
vocab_size....................: 50256
beam_width....................: 1
top_k.........................: 1
top_p.........................: 0.0
temperature...................: 1.0
len_penalty...................: 0.0
beam_search_diversity_rate....: 0.0
tensor_para_size..............: 1
pipeline_para_size............: 1
ckpt_path.....................: ../models/openai-gpt-models/c-model/124m/1-gpu
lib_path......................: ./lib/libth_transformer.so
vocab_file....................: ../models/gpt2-vocab.json
merges_file...................: ../models/gpt2-merges.txt
start_id......................: 50256
end_id........................: 50256
max_batch_size................: 8
repetition_penalty............: 1.0
presence_penalty..............: 0.0
min_length....................: 0
max_seq_len...................: 768
inference_data_type...........: fp16
time..........................: False
sample_input_file.............: None
sample_output_file............: None
enable_random_seed............: False
skip_end_tokens...............: False
detokenize....................: True
use_jieba_tokenizer...........: False
int8_mode.....................: 0
weights_data_type.............: fp32
return_cum_log_probs..........: 0
shared_contexts_ratio.........: 1.0
banned_words..................:
use_gpt_decoder_ops...........: False