2.1 ppq量化pytorch-＞onnx

nsq_ai

已于 2023-10-29 11:24:12 修改

阅读量424

点赞数

分类专栏：模型推理文章标签： pytorch 人工智能 python

于 2023-10-29 11:19:44 首次发布

本文链接：https://blog.csdn.net/weixin_45063703/article/details/134101073

版权

模型推理专栏收录该内容

54 篇文章 3 订阅

订阅专栏

前言

torchvision中加载一个模型，转换为 onnx 格式、导出 quantized graph.

code

from typing import Iterable

import torch
import torchvision
from torch.utils.data import DataLoader

from ppq import BaseGraph, QuantizationSettingFactory, TargetPlatform
from ppq.api import export_ppq_graph, quantize_torch_model

BATCHSIZE = 32
INPUT_SHAPE = [3, 224, 224]
DEVICE = 'cuda' # only cuda is fully tested :(  For other executing device there might be bugs.
PLATFORM = TargetPlatform.PPL_CUDA_INT8  # identify a target platform for your network.

def load_calibration_dataset() -> Iterable:
    return [torch.rand(size=INPUT_SHAPE) for _ in range(32)]

def collate_fn(batch: torch.Tensor) -> torch.Tensor:
    return batch.to(DEVICE)

# Load a pretrained mobilenet v2 model
model = torchvision.models.mobilenet.mobilenet_v2(pretrained=True)
model = model.to(DEVICE)

# create a setting for quantizing your network with PPL CUDA.
quant_setting = QuantizationSettingFactory.pplcuda_setting()
quant_setting.equalization = True # use layerwise equalization algorithm.
quant_setting.dispatcher   = 'conservative' # dispatch this network in conservertive way.

# Load training data for creating a calibration dataloader.
calibration_dataset = load_calibration_dataset()
calibration_dataloader = DataLoader(
    dataset=calibration_dataset,
    batch_size=BATCHSIZE, shuffle=True)

# quantize your model.
quantized = quantize_torch_model(
    model=model, calib_dataloader=calibration_dataloader,
    calib_steps=32, input_shape=[BATCHSIZE] + INPUT_SHAPE,
    setting=quant_setting, collate_fn=collate_fn, platform=PLATFORM,
    onnx_export_file='./onnx.model', device=DEVICE, verbose=0)

# Quantization Result is a PPQ BaseGraph instance.
assert isinstance(quantized, BaseGraph)

# export quantized graph.
export_ppq_graph(graph=quantized, platform=PLATFORM,
                 graph_save_to='./quantized(onnx).onnx',
                 config_save_to='./quantized(onnx).json')

# analyse quantization error brought in by every layer
from ppq.quantization.analyse import layerwise_error_analyse, graphwise_error_analyse
graphwise_error_analyse(
    graph=quantized, # ppq ir graph
    running_device=DEVICE, # cpu or cuda
    method='snr',  # the metric is signal noise ratio by default, adjust it to 'cosine' if that's desired
    steps=32, # how many batches of data will be used for error analysis
    dataloader=calibration_dataloader,
    collate_fn=lambda x: x.to(DEVICE)
)
 
layerwise_error_analyse(
    graph=quantized,
    running_device=DEVICE,
    method='snr',  # the metric is signal noise ratio by default, adjust it to 'cosine' if that's desired
    steps=32,
    dataloader=calibration_dataloader,
    collate_fn=lambda x: x.to(DEVICE)
)

结果

加载预训练的mobilenet v2 model
最终生成三个文件信息

# python eaxmple.py 

      ____  ____  __   ____                    __              __
     / __ \/ __ \/ /  / __ \__  ______ _____  / /_____  ____  / /
    / /_/ / /_/ / /  / / / / / / / __ `/ __ \/ __/ __ \/ __ \/ /
   / ____/ ____/ /__/ /_/ / /_/ / /_/ / / / / /_/ /_/ / /_/ / /
  /_/   /_/   /_____\___\_\__,_/\__,_/_/ /_/\__/\____/\____/_/


[07:04:00] PPQ Layerwise Equalization Pass Running ... 2 equalization pair(s) was found, ready to run optimization.
Layerwise Equalization: 100%|█████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 1274.01it/s]
Finished.
[07:04:00] PPQ Quantization Config Refine Pass Running ... Finished.
[07:04:00] PPQ Quantization Fusion Pass Running ...        Finished.
[07:04:00] PPQ Quantize Point Reduce Pass Running ...      Finished.
[07:04:00] PPQ Parameter Quantization Pass Running ...     Finished.
Calibration Progress(Phase 1): 100%|████████████████████████████████████████████████████| 32/32 [01:59<00:00,  3.74s/it]
[07:04:00] PPQ Runtime Calibration Pass Running ...        Finished.
[07:06:00] PPQ Quantization Alignment Pass Running ...     Finished.
[07:06:00] PPQ Passive Parameter Quantization Running ...  Finished.
[07:06:00] PPQ Parameter Baking Pass Running ...           Finished.
--------- Network Snapshot ---------
Num of Op:                    [100]
Num of Quantized Op:          [100]
Num of Variable:              [277]
Num of Quantized Var:         [277]
------- Quantization Snapshot ------
Num of Quant Config:          [386]
BAKED:                        [53]
OVERLAPPED:                   [125]
SLAVE:                        [20]
ACTIVATED:                    [65]
PASSIVE_BAKED:                [53]
FP32:                         [70]
Network Quantization Finished.
Analysing Graphwise Quantization Error(Phrase 1):: 100%|██████████████████████████████████| 1/1 [00:00<00:00,  8.19it/s]
Analysing Graphwise Quantization Error(Phrase 2):: 100%|██████████████████████████████████| 1/1 [00:00<00:00,  6.84it/s]
Layer     | NOISE:SIGNAL POWER RATIO 
Conv_8:   | ████████████████████ | 1.678653
Conv_26:  | ████████████████     | 1.313450
Conv_9:   | █████████████        | 1.087763
Conv_13:  | █████████████        | 1.074564
Conv_55:  | ████████████         | 0.991271
Conv_28:  | ██████████           | 0.857988
Conv_17:  | ████████             | 0.730895
Conv_154: | ████████             | 0.676669
Conv_22:  | ████████             | 0.659212
Conv_152: | ███████              | 0.618322
Conv_142: | ███████              | 0.582268
Conv_133: | ██████               | 0.534554
Conv_45:  | ██████               | 0.520580
Conv_51:  | █████                | 0.464549
Conv_144: | █████                | 0.441523
Conv_41:  | █████                | 0.414612
Conv_36:  | █████                | 0.411636
Conv_57:  | ████                 | 0.387930
Conv_113: | ████                 | 0.368550
Conv_148: | ████                 | 0.351853
Conv_123: | ████                 | 0.333928
Conv_104: | ████                 | 0.331407
Conv_134: | ███                  | 0.319796
Conv_4:   | ███                  | 0.309758
Conv_138: | ███                  | 0.274523
Conv_125: | ███                  | 0.272312
Conv_32:  | ███                  | 0.269519
Conv_94:  | ███                  | 0.255700
Conv_47:  | ███                  | 0.255035
Conv_129: | ███                  | 0.246983
Conv_18:  | ██                   | 0.222586
Conv_65:  | ██                   | 0.205310
Conv_162: | ██                   | 0.190181
Conv_84:  | ██                   | 0.189721
Conv_90:  | ██                   | 0.183772
Conv_96:  | ██                   | 0.181663
Conv_70:  | ██                   | 0.174435
Conv_163: | ██                   | 0.167765
Conv_115: | ██                   | 0.164749
Conv_100: | █                    | 0.152931
Conv_86:  | █                    | 0.150768
Conv_105: | █                    | 0.148656
Conv_80:  | █                    | 0.134689
Conv_109: | █                    | 0.131509
Conv_37:  | █                    | 0.124499
Conv_119: | █                    | 0.122543
Conv_74:  | █                    | 0.096819
Conv_76:  |                      | 0.072862
Conv_61:  |                      | 0.071023
Conv_0:   |                      | 0.067830
Conv_66:  |                      | 0.064776
Gemm_169: |                      | 0.035677
Conv_158: |                      | 0.032427
Analysing Layerwise quantization error:: 100%|██████████████████████████████████████████| 53/53 [00:01<00:00, 34.86it/s]
Layer     | NOISE:SIGNAL POWER RATIO 
Conv_4:   | ████████████████████ | 0.007448
Conv_22:  | ███                  | 0.001254
Conv_133: | ███                  | 0.000973
Conv_142: | █                    | 0.000488
Conv_152: | █                    | 0.000487
Conv_162: | █                    | 0.000420
Conv_104: | █                    | 0.000372
Conv_8:   | █                    | 0.000269
Conv_65:  | █                    | 0.000214
Conv_113: | █                    | 0.000204
Conv_123: | █                    | 0.000190
Conv_13:  |                      | 0.000183
Conv_41:  |                      | 0.000142
Conv_17:  |                      | 0.000136
Conv_26:  |                      | 0.000113
Conv_36:  |                      | 0.000108
Conv_70:  |                      | 0.000096
Conv_94:  |                      | 0.000092
Conv_109: |                      | 0.000078
Conv_100: |                      | 0.000068
Conv_125: |                      | 0.000066
Conv_119: |                      | 0.000065
Gemm_169: |                      | 0.000064
Conv_55:  |                      | 0.000060
Conv_84:  |                      | 0.000058
Conv_138: |                      | 0.000053
Conv_80:  |                      | 0.000051
Conv_28:  |                      | 0.000050
Conv_45:  |                      | 0.000043
Conv_57:  |                      | 0.000036
Conv_90:  |                      | 0.000034
Conv_105: |                      | 0.000032
Conv_32:  |                      | 0.000032
Conv_144: |                      | 0.000031
Conv_74:  |                      | 0.000027
Conv_96:  |                      | 0.000027
Conv_115: |                      | 0.000026
Conv_154: |                      | 0.000026
Conv_51:  |                      | 0.000024
Conv_0:   |                      | 0.000023
Conv_148: |                      | 0.000022
Conv_86:  |                      | 0.000021
Conv_134: |                      | 0.000021
Conv_66:  |                      | 0.000018
Conv_18:  |                      | 0.000015
Conv_76:  |                      | 0.000012
Conv_129: |                      | 0.000010
Conv_47:  |                      | 0.000009
Conv_61:  |                      | 0.000008
Conv_9:   |                      | 0.000006
Conv_163: |                      | 0.000005
Conv_37:  |                      | 0.000004
Conv_158: |                      | 0.000002