OpenMP + MPI 编程（三）Dealing with NUMA and Why Hybrid? Or Why Not?

叶子心情你不懂

于 2020-11-13 16:27:59 发布

阅读量263

点赞数

分类专栏：总结报告文章标签： openmp mpi

本文链接：https://blog.csdn.net/qq_40379678/article/details/109670895

版权

总结报告专栏收录该内容

15 篇文章

订阅专栏

NUMA(非一致性内存访问)在并行编程中带来了内存访问的复杂性，但通过混合编程可以利用共享和分布式内存的优势。混合编程旨在平衡计算负载，减少内存流量，特别是在内存密集型应用中。然而，混合使用SMP和MPI可能会导致性能下降，存在资源浪费的风险。混合编程的主要动机在于优化节点内的计算和内存使用，提高效率。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Dealing with NUMA

How do we deal with NUMA (Non-Uniform Memory Access)?

Standard models for parallel programs assume a uniform architecture

Threads for shared memory
- parent process uses pthreads or OpenMP to fork multiple threads
- threads share the same virtual address space
- also known as SMP = Symmetric MultiProcessing
Message passing for distributed memory
- processes use MPI to pass messages (data) between each other
- each process has its own virtual address space
  If we attempt to combine both types of models -
Hybrid programming
- try to exploit the whole shared/distributed memory hierarchy