参考:https://github.com/horovod/horovod/commit/ecbda18ee8c3fd5b3ff2b8d319c34873f1f76f9d
注释掉以下两行代码即可:
hvd.broadcast_parameters(model.state_dict(), root_rank=0)
hvd.broadcast_optimizer_state(optimizer, root_rank=0)
参考:https://github.com/horovod/horovod/commit/ecbda18ee8c3fd5b3ff2b8d319c34873f1f76f9d
注释掉以下两行代码即可:
hvd.broadcast_parameters(model.state_dict(), root_rank=0)
hvd.broadcast_optimizer_state(optimizer, root_rank=0)