NVIDIA Docker运行FasterTransformer

Python = 3.8.11， cuda=11.8

安装Docker及NVIDIA Docker的教程：https://blog.csdn.net/frontiers/article/details/129732630?spm=1001.2014.3001.5501

FasterTransformer运行GPT的官方教程：https://github.com/NVIDIA/FasterTransformer/blob/main/docs/gpt_guide.md#build-the-fastertransformer

使用pre-build Docker创建Docker：

#tensorflow 模型
nvidia-docker run -ti --shm-size 5g --rm nvcr.io/nvidia/tensorflow:22.09-tf1-py3 bash
#torch模型
nvidia-docker run -ti --shm-size 5g --rm nvcr.io/nvidia/pytorch:22.09-py3 bash

git clone

git clone https://github.com/NVIDIA/FasterTransformer.git
mkdir -p FasterTransformer/build
cd FasterTransformer/build
git submodule init && git submodule update

由于我用的torch模型，下面的步骤只保留torch的，如果需要tensorflow可以去官方guide看

build with torch on A100:

cmake -DSM=80 -DCMAKE_BUILD_TYPE=Release -DBUILD_PYT=ON -DBUILD_MULTI_GPU=ON ..
make -j12

pip install -r ../examples/pytorch/gpt/requirement.txt

wget https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-vocab.json -P ../models
wget https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-merges.txt -P ../models

mkdir -p ../models/openai-gpt-models/
#下载GPT2 model
python ../examples/tensorflow/gpt/utils/download_gpt2_model.py 124M
mv models/124M ../models/openai-gpt-models/
python ../examples/tensorflow/gpt/utils/openai_gpt_ckpt_converter.py -o ../models/openai-gpt-models/c-model/124m/ -i ../models/openai-gpt-models/124M/model.ckpt -g 1

运行GPT2 model

python ../examples/pytorch/gpt/multi_gpu_gpt_example.py

不要使用gpt_example.py， it's deprecated.

这里可以进到multi_gpu_gpt_example.py里面改参数。同样可以在这个文件里看到fake input是怎么生成的。我这里的参数：

=================== Arguments ===================

layer_num.....................: 12

input_len.....................: 512

output_len....................: 32

head_num......................: 12

size_per_head.................: 64

vocab_size....................: 50256

beam_width....................: 1

top_k.........................: 1

top_p.........................: 0.0

temperature...................: 1.0

len_penalty...................: 0.0

beam_search_diversity_rate....: 0.0

tensor_para_size..............: 1

pipeline_para_size............: 1

ckpt_path.....................: ../models/openai-gpt-models/c-model/124m/1-gpu

lib_path......................: ./lib/libth_transformer.so

vocab_file....................: ../models/gpt2-vocab.json

merges_file...................: ../models/gpt2-merges.txt

start_id......................: 50256

end_id........................: 50256

max_batch_size................: 8

repetition_penalty............: 1.0

presence_penalty..............: 0.0

min_length....................: 0

max_seq_len...................: 768

inference_data_type...........: fp16

time..........................: False

sample_input_file.............: None

sample_output_file............: None

enable_random_seed............: False

skip_end_tokens...............: False

detokenize....................: True

use_jieba_tokenizer...........: False

int8_mode.....................: 0

weights_data_type.............: fp32

return_cum_log_probs..........: 0

shared_contexts_ratio.........: 1.0

banned_words..................:

use_gpt_decoder_ops...........: False