tensorflow+MPI多进程报错的解决办法：Signal: Segmentation fault (11)Signal code: Address not mapped (1)

黄埔大道Switch玩家

于 2020-04-27 21:31:28 发布

阅读量8.2k

点赞数 2

分类专栏：到大风大浪中去锻炼文章标签： tensorflow python

本文链接：https://blog.csdn.net/weixin_38909635/article/details/105792556

版权

在使用TensorFlow和MPI进行多进程运算时遇到Segmentation fault (11)错误。问题源于自定义梯度函数在MPI环境中导致的问题，而非函数本身。通过检查发现OpenMPI版本较低，升级到最新版（4.1.4）解决了问题，无需sudo权限，只需通过conda安装和重装。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

tensorflow+MPI跑一个任务。遇到如下错误：

[cdc-24:02776] *** Process received signal ***
[cdc-24:02776] Signal: Segmentation fault (11)
[cdc-24:02776] Signal code: Address not mapped (1)
[cdc-24:02776] Failing at address: 0x55ea3fd8c7f0
[cdc-24:02771] *** Process received signal ***
[cdc-24:02771] Signal: Segmentation fault (11)
[cdc-24:02771] Signal code: Address not mapped (1)
[cdc-24:02771] Failing at address: 0x1d0
[cdc-24:02778] *** Process received signal ***
[cdc-24:02778] Signal: Segmentation fault (11)
[cdc-24:02778] Signal code: Address not mapped (1)
[cdc-24:02778] Failing at address: 0x5569a83df7f0
[cdc-24:02773] *** Process received signal ***
[cdc-24:02776] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7f3fa8e7b390]
[cdc-24:02776] [ 1] [cdc-24:02778] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fd44ecf9390]
[cdc-24:02778] [ 1] /usr/lib/openmpi/lib/openmpi/mca_btl_vader.so(mca_btl_vader_poll_handle_frag+0x154)[0x7fd4185c18d4]