pytorch学习笔记（七）——Broadcasting自动扩展

Broadcasting自动扩展 

能够自动扩展且不copy数据，节省数据

Insert 1 dim ahead

Expand dims with size 1 to same size

Feature maps: [4, 32, 14, 14]

Bias: [32, 1, 1] => [1, 32, 1, 1] => [4, 32, 14, 14]

broadcasting的意义

使[4,28,8]的tensor也能与[1]的标量相加（做运算）同时会节省很大的内存

1. for actual demanding

[class, students, scores]

Add bias for every students: +5 score

[4, 32, 8] + [4, 32, 8] ▪ [4, 32, 8] + [5.0]

2. memory consumption

[4, 32, 8] => 1024

[5.0] => 1

使用broadcasting的情况

1.If current dim=1, expand to same

A：[4, 32, 14, 14]

B：[1, 32, 1, 1] => [4, 32, 14, 14]

2.If either has no dim, insert one dim and expand to same

A：[4, 32, 14, 14]

B：[14, 14] => [1, 1, 14, 14] => [4, 32, 14, 14]

3.otherwise, NOT broadcasting-able

A：[4, 32, 14, 14]

B：[2, 32, 14, 14]

此时不能用broadcasting