broadcast自动扩展/合并与分割/基本运算

一、broadcast

Key idea

▪ Insert 1 dim ahead
▪ Expand dims with size 1 to same size
▪ Feature maps: [4, 32, 14, 14]
▪ Bias: [32, 1, 1] => [1, 32, 1, 1] => [4, 32, 14, 14]
Situation 1:
▪ [4, 32, 14, 14]
▪ [1, 32, 1, 1] => [4, 32, 14, 14]
Situation 2
▪ [4, 32, 14, 14]
▪ [14, 14] => [1, 1, 14, 14] => [4, 32, 14, 14]
Situation 3
▪ [4, 32, 14, 14]
▪ [2, 32, 14, 14]
▪ Dim 0 has dim, can NOT insert and expand to same
▪ Dim 0 has distinct dim, NOT size 1
▪ NOT broadcasting-able

match from last dim

When it has no dim
▪ treat it as all own the same
▪ [class, student, scores] + [scores]
▪ When it has dim of size 1
▪ Treat it shared by all
▪ [class, student, scores] + [student, 1]

It’s effective and critically, intuitive
▪ [4, 3, 32, 32]
▪ + [32, 32]
▪ + [3, 1, 1]
▪ + [1, 1, 1, 1]

二、合并

cat

stack

create new dim

三、拆分

split：

## chunk

四、基本运算

matmul

An example

matmul

power

exp log

approximation

floor() ceil() round() trunc() ,frac()

clamp

五、统计属性

总结