先来一段stackoverflow上的一个回答:
https://stackoverflow.com/questions/67943173/onnxruntime-vs-pytorch
ONNX Runtime uses static ONNX graph, so it has full view of the graph and can do a lot of optimizations that are impossible/harder to do with PyTorch. In a sense, it's similar to compiled vs interpreted programming language implementations.
对onnx runtime加速的报道:
实际上,这篇报告中把onnx runtime用到的大部分技术都概括了(当然实际上还是需要从源码看具体实现):
流水线并行化:
https://blog.csdn.net/dQCFKyQDXYm3F8rB0/article/details/102878315
https://pytorch.org/docs/master/pipeline.html
deepspeed:
https://github.com/microsoft/DeepSpeed
https://www.microsoft.com/en-us/research/project/ai-at-scale/
https://zhuanlan.zhihu.com/p/106783111
https://www.oschina.net/news/113328/microsoft-opensource-deepspeed
deepspeed中用到的zero技术论文:
https://arxiv.org/pdf/1910.02054.pdf
https://www.deepspeed.ai/tutorials/zero-offload/
deepspeed使用说明:
https://www.deepspeed.ai/docs/config-json/#sparse-attention
onnx runtime测试报告:
Megatron-LM代码: