07. Tensorflow2.0中的broadcasting

1. Broadcasting

1.1. Outline

• expand
• without copying data
  • VS tf.tile
• tf.broadcast_to

1.2. Key idea

• Insert 1 dim ahead if needed
• Expand dims with size 1 to same size
• Feature maps: [4, 32, 32, 3]
• Bias: [3] → [1, 1, 1, 3] → [4, 32, 32, 3]

1.3. How to understand?

• When it has no axis

  • Create a new concept
  • [classes, students, scores] + [scores]

• When it has dim of size 1

  • Treat it shared by all
  • [classes, students, scores] + [students, 1]

1.4. Why broadcasting?

• for real demanding

  • [classes, students, scores]
  • Add bias for every student: + 5 score
  • [4, 32, 8] + [4, 32, 8]
  • [4, 32, 8] + [5.0]

• memory consumption

  • [4, 32, 8] --> 1024
  • bias=[8]: [5.0, 5.0…] --> 8

2. Broadcastable?

2.1. Match from Last dim!

- if current dim = 1, expand to same
- if either has no dim, insert one dim and expand to same
- otherwise, NOT broadcastable

2.1.1. situation1

• [4, 32, 14, 14]
• [1, 32, 1, 1] → [4, 32, 14, 14]

2.1.2. situation2

• [4, 32, 14, 14]
• [14, 14] → [1, 1, 14, 14] → [4, 32, 14, 14]

2.1.3. situation3

• [4, 32, 14, 14]
• [2, 32, 14, 14]
• Dim 0 has dim, can NOT insert and expand to same
• Dim 0 has distinct dim, NOT size 1 Not broadcasting-able

3. Code

3.1. Broadcasting（隐式）

3.2. tf.broadcast_to（显示）

3.3. Broadcast VS Tile

4. 需要全套课程视频+PPT+代码资源可以私聊我-

• 方式1：CSDN私信我!
• 方式2：QQ邮箱:594042358@qq.com或者直接加我QQ: 594042358!