mindspore和pytorch的比较 -- 网络接口部分（二）

二、网络接口及支持的方法

功能	mindspore Cell接口	torch Module接口
查看模块	cells_and_names()	modules()，named_modules()	功能一致
	cells()	children()	略有差异
	name_cells()	name_children()	略有差异
获取参数	parameters_and_names()，get_parameters()	state_dict(destination=None)	略有差异
	parameters_dict()	/
	/	load_state_dict(state_dict)
需要优化	trainable_params()	named_parameters(), parameters()	略有差异
不需优化	untrainable_params(recurse=True)	buffers(), named_buffers()	略有差异
添加模块	insert_child_to_cell(child_name, child_cell)	add_module(name, module)	功能一致
状态设置	set_train(mode=True)	train(mode=True), eval()	功能一致
	init_parameters_data(auto_parallel_mode=False)	/
	train_net.set_broadcast_flag()	/
	set_param_ps()	/

1. 查看网络中子模块

1）查看外层子模块：功能相同，类型不同

• mindspore: net.name_cells() / net.cells()：OrderedDict / odict_values
• net.named_children() / children()：迭代器

2）查看全部模块：一致

• mindspore: net.cells_and_names()：迭代器
• mindspore: net.named_modules() / modules()：迭代器

# 查看外层子模块
# mindspore
for name, cell in net.name_cells().items():
  print("*** name: ", name)
  print("*** cell: ", cell)
  
# torch
for name, child in net.named_children(): 
    print("*** name: ", name)            
    print("*** child: ", child)         

# mindspore: output
*** name:  build_block1
*** cell:  SequentialCell<
  (0): ConvBNReLU<
    (conv): Conv2d<input_channels=3, output_channels=64, kernel_size=(3, 3),stride=(1, 1),  pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=Falseweight_init=normal, bias_init=zeros, format=NCHW>
    (bn): BatchNorm2d<num_features=64, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block1.0.bn.gamma, shape=(64,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block1.0.bn.beta, shape=(64,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block1.0.bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block1.0.bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)>
    (relu): ReLU<>
    >
  (1): MaxPool2d<kernel_size=2, stride=1, pad_mode=VALID>
  >
*** name:  build_block2
*** cell:  CellList<
  (0): Conv2d<input_channels=64, output_channels=4, kernel_size=(4, 4),stride=(1, 1),  pad_mode=same, padding=0, dilation=(1, 1), group=1, has_bias=Falseweight_init=normal, bias_init=zeros, format=NCHW>
  (1): BatchNorm2d<num_features=4, eps=1e-05, momentum=0.09999999999999998, gamma=Parameter (name=build_block2.1.gamma, shape=(4,), dtype=Float32, requires_grad=True), beta=Parameter (name=build_block2.1.beta, shape=(4,), dtype=Float32, requires_grad=True), moving_mean=Parameter (name=build_block2.1.moving_mean, shape=(4,), dtype=Float32, requires_grad=False), moving_variance=Parameter (name=build_block2.1.moving_variance, shape=(4,), dtype=Float32, requires_grad=False)>
  (2): ReLU<>
  >

# torch: output
*** name:  build_block1
*** child:  Sequential(
  (0): ConvBNReLU(
    (conv): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1))
    (bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (relu): ReLU()
  )
  (pool): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
*** name:  build_block2
*** child:  ModuleList(
  (0): Conv2d(64, 4, kernel_size=(4, 4), stride=(1, 1))
  (1): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): ReLU()
)

# 查看所有子模块
# mindspore
for name, cell in net.cells_and_names():
  print("*** name: ", name)
  
# torch
for name, module in net.named_modules():   
    print("*** name: ", name)              

# output：mindspore
*** name:  
*** name:  build_block1
*** name:  build_block1.0
*** name:  build_block1.0.conv
*** name:  build_block1.0.bn
*** name:  build_block1.0.relu
*** name:  build_block1.1
*** name:  build_block2
*** name:  build_block2.0
*** name:  build_block2.1
*** name:  build_block2.2

# output： torch
*** name:  
*** name:  build_block1
*** name:  build_block1.0
*** name:  build_block1.0.conv
*** name:  build_block1.0.bn
*** name:  build_block1.0.relu
*** name:  build_block1.pool
*** name:  build_block2
*** name:  build_block2.0
*** name:  build_block2.1
*** name:  build_block2.2

1. 获取参数

1）全部参数：

• mindspore: net.parameters_and_names() / net.get_parameters() / net.parameters_dict()：迭代器；迭代器；OrderedDict
• torch: net.state_dict()：OrderedDict

2）查看需要优化的参数：

• mindspore: net.trainable_params()：迭代器
• torch: net.named_parameters(), net.parameters()：迭代器

3）查看不需要优化的参数：

• mindspore: net.untrainable_params()：迭代器
• torch: net.buffers(), net.named_buffers()：迭代器

# mindspore: 查看所有参数
for name, params in net.parameters_and_names():
  print("*** name: ", name, "     *** params: ", params)
  
# torch: 查看所有参数
for name, state in net.state_dict().items():                      
    print("*** name: ", name, "     *** state: ", type(state))    

# mindspore output
*** name:  build_block1.0.conv.weight      *** params:  Parameter (name=build_block1.0.conv.weight, shape=(64, 3, 3, 3), dtype=Float32, requires_grad=True)
*** name:  build_block1.0.bn.moving_mean      *** params:  Parameter (name=build_block1.0.bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False)
*** name:  build_block1.0.bn.moving_variance      *** params:  Parameter (name=build_block1.0.bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)
*** name:  build_block1.0.bn.gamma      *** params:  Parameter (name=build_block1.0.bn.gamma, shape=(64,), dtype=Float32, requires_grad=True)
*** name:  build_block1.0.bn.beta      *** params:  Parameter (name=build_block1.0.bn.beta, shape=(64,), dtype=Float32, requires_grad=True)
*** name:  build_block2.0.weight      *** params:  Parameter (name=build_block2.0.weight, shape=(4, 64, 4, 4), dtype=Float32, requires_grad=True)
*** name:  build_block2.1.moving_mean      *** params:  Parameter (name=build_block2.1.moving_mean, shape=(4,), dtype=Float32, requires_grad=False)
*** name:  build_block2.1.moving_variance      *** params:  Parameter (name=build_block2.1.moving_variance, shape=(4,), dtype=Float32, requires_grad=False)
*** name:  build_block2.1.gamma      *** params:  Parameter (name=build_block2.1.gamma, shape=(4,), dtype=Float32, requires_grad=True)
*** name:  build_block2.1.beta      *** params:  Parameter (name=build_block2.1.beta, shape=(4,), dtype=Float32, requires_grad=True)

# torch output
*** name:  build_block1.0.conv.weight      *** state:  <class 'torch.Tensor'>
*** name:  build_block1.0.conv.bias      *** state:  <class 'torch.Tensor'>
*** name:  build_block1.0.bn.weight      *** state:  <class 'torch.Tensor'>
*** name:  build_block1.0.bn.bias      *** state:  <class 'torch.Tensor'>
*** name:  build_block1.0.bn.running_mean      *** state:  <class 'torch.Tensor'>
*** name:  build_block1.0.bn.running_var      *** state:  <class 'torch.Tensor'>
*** name:  build_block1.0.bn.num_batches_tracked      *** state:  <class 'torch.Tensor'>
*** name:  build_block2.0.weight      *** state:  <class 'torch.Tensor'>
*** name:  build_block2.0.bias      *** state:  <class 'torch.Tensor'>
*** name:  build_block2.1.weight      *** state:  <class 'torch.Tensor'>
*** name:  build_block2.1.bias      *** state:  <class 'torch.Tensor'>
*** name:  build_block2.1.running_mean      *** state:  <class 'torch.Tensor'>
*** name:  build_block2.1.running_var      *** state:  <class 'torch.Tensor'>
*** name:  build_block2.1.num_batches_tracked      *** state:  <class 'torch.Tensor'>

# mindspore: 查看需要优化的参数
for params in net.trainable_params():
  print("*** params: ", params)
  
# torch: 查看需要优化的参数
for name, params in net.named_parameters():                          
    print("*** name: ", name, "     *** params: ", type(params))     

# mindspore output
*** params:  Parameter (name=build_block1.0.conv.weight, shape=(64, 3, 3, 3), dtype=Float32, requires_grad=True)
*** params:  Parameter (name=build_block1.0.bn.gamma, shape=(64,), dtype=Float32, requires_grad=True)
*** params:  Parameter (name=build_block1.0.bn.beta, shape=(64,), dtype=Float32, requires_grad=True)
*** params:  Parameter (name=build_block2.0.weight, shape=(4, 64, 4, 4), dtype=Float32, requires_grad=True)
*** params:  Parameter (name=build_block2.1.gamma, shape=(4,), dtype=Float32, requires_grad=True)
*** params:  Parameter (name=build_block2.1.beta, shape=(4,), dtype=Float32, requires_grad=True)

# torch output
*** name:  build_block1.0.conv.weight      *** params:  <class 'torch.nn.parameter.Parameter'>
*** name:  build_block1.0.conv.bias      *** params:  <class 'torch.nn.parameter.Parameter'>
*** name:  build_block1.0.bn.weight      *** params:  <class 'torch.nn.parameter.Parameter'>
*** name:  build_block1.0.bn.bias      *** params:  <class 'torch.nn.parameter.Parameter'>
*** name:  build_block2.0.weight      *** params:  <class 'torch.nn.parameter.Parameter'>
*** name:  build_block2.0.bias      *** params:  <class 'torch.nn.parameter.Parameter'>
*** name:  build_block2.1.weight      *** params:  <class 'torch.nn.parameter.Parameter'>
*** name:  build_block2.1.bias      *** params:  <class 'torch.nn.parameter.Parameter'>

# mindspore: 查看不需要优化的参数
for params in net.untrainable_params():
  print("*** params: ", params)
  
# torch: 查看不需要优化的参数
for name, buffer in net.named_buffers():                                
    print("*** name: ", name, "     *** buffer: ", type(buffer))        

# mindspore output
*** params:  Parameter (name=build_block1.0.bn.moving_mean, shape=(64,), dtype=Float32, requires_grad=False)
*** params:  Parameter (name=build_block1.0.bn.moving_variance, shape=(64,), dtype=Float32, requires_grad=False)
*** params:  Parameter (name=build_block2.1.moving_mean, shape=(4,), dtype=Float32, requires_grad=False)
*** params:  Parameter (name=build_block2.1.moving_variance, shape=(4,), dtype=Float32, requires_grad=False)

# torch output
*** name:  build_block1.0.bn.running_mean      *** buffer:  <class 'torch.Tensor'>
*** name:  build_block1.0.bn.running_var      *** buffer:  <class 'torch.Tensor'>
*** name:  build_block1.0.bn.num_batches_tracked      *** buffer:  <class 'torch.Tensor'>
*** name:  build_block2.1.running_mean      *** buffer:  <class 'torch.Tensor'>
*** name:  build_block2.1.running_var      *** buffer:  <class 'torch.Tensor'>
*** name:  build_block2.1.num_batches_tracked      *** buffer:  <class 'torch.Tensor'>