Tensorflow2.0——3、Broadcasting

Broadcasting
■ expand
■ without copying data
■ VS tf.tile
■ tf. broadcast to

key idea
■ Insert 1 dim ahead if needed
■ Expand dims with size 1 to same size

Feature maps: [4, 32, 32, 3]
Bias: [3]→[1,1,1, 32]→[4, 32, 32, 3]
//即wx+b中b的偏置

---

注意*********

[1,3]
[1,4]
这两者不可以进行broadcast！！！
从右到左最右的数据3,4不一致，最右数据叫小维度，最左叫做大维度

---

Why broadcasting?

1、for real demanding

• [classes, students, scores]
• Add bias for every student: + 5 score
  . [4,32,8] + [4, 32, 8]
  ■[4, 32, 8] + [5.0]

2、memory consumption

• [4,32, 8]→1024
• bias=[8]: [5.0,5.0,5.0…]→8
  (节省内存，1024 * 4/8 * 4=2^7)

---

Situation 1:（√）

[4, 32, 14, 14]
[1, 32, 1,1]→[4, 32, 14, 14]

Situation 2（√）

[4, 32, 14, 14]
[14, 14]→[1, 1,14, 14]→[4, 32, 14, 14]

Situation 3（×）

[4, 32, 14, 14]
[2, 32, 14, 14]

• Dim 0 has dim, can NOT insert and expand to same
• Dim 0 has distinct dim, NOT size 1
• NOT broadcasting-able

---

eg：

In [25]: x=tf.random. normal([4,32,32,3])
In [27]: (x+tf.r andom. normal([3])). shape
Out[27]: Tensor Shape([4, 32， 32, 3])

In [28]: (x+tf.r andom. normal([32,32, 1])). shape
Out[28]: Tensor Shape([4, 32，32,
3])

In [29]: (x+tf.r andom. normal([4,1,1, 1])). shape 
Out[29]: Tensor Shape([4, 32，32，3])

In [31]: (x+tf.random. normal([1,4,1, 1])). shape 
InvalidAr gumentError: Incompatible shapes: [4,32,32,3] Vs. [1,4,1,1] [Op:Add]
name: add/

tf. broadcast_ to

In [37]: b=tf. broadcast_ to(tf. random. normal([4,1,1,1])，[4,32,32,3])
In [38]: b. shape
Out[38]: Tensor. Shape([4, 32, 32，3])