Broadcasting机制

一、Broadcasting机制介绍

1. Broadcasting(广播)机制是numpy对不同shape的数组进行数值计算的一种方式。
2. Numpy的通用函数中要求输入的两个tensor的shape一致。当两个tensor的shape不一致时，会使用Broadcasting机制。

二、Broadcast机制规则

1. 规则1:让所有输入的tensor向dim最大的tensor看齐，dim不足，前补1
2. 规则2:将输入的tensor的某一个shape为1的维度拉伸。
3. 规则3:如果存在两个tensor的shape在任何维度均不匹配，且均没有等于1的维度，则会报错。

三、实例说明

1. Broadcasting理解
   1. 三部分代码分别对应下图的三个实例

a = torch.tensor([ [0,0,0],[10,10,10],[20,20,20],[30,30,30]]) # torch.Size([4, 3])
b = torch.tensor([ [0,1,2] , [0,1,2],[0,1,2],[0,1,2]]) # torch.Size([4, 3])
print("a + b == \n",a + b)
# 由于a和b的shape相同，不需要使用Broadcasting机制
# ---------------------------------------------------------------------
a = torch.tensor([ [0,0,0],[10,10,10],[20,20,20],[30,30,30]]) # torch.Size([4, 3])
b = torch.tensor([0,1,2]) # torch.Size([3])
print("a + b == \n",a + b)
# a.shape = [4,3] a.dim = 2; b.shape = [3], b.dim = 1, shape不同，需要使用Broadcasting机制
# 由于b.dim = 1 < a.dim = 2 , b向a看齐
# 相当于b.unsqueeze(0),使其shape变为[1,3],然后再使用expand(4,-1)使其shape变为[4,3]

# ---------------------------------------------------------------------
a = torch.tensor([[0],[10],[20],[30]]) # torch.Size([4, 1])
b = torch.tensor([0,1,2]) # torch.Size([3])
print("a + b == \n",a + b)
# a.shape = [4,1] a.dim = 2; b.shape = [3], b.dim = 1,  shape不同，需要使用Broadcasting机制
# 由于b.dim = 1 < a.dim = 2 , b向a看齐
# 如果dim不同需要补齐所以b的shape变换为 [3] => [1,3]
# dim变换后，将shape为1的拉伸: a.shape变换为: [4,1] => [4,3] ; b.shape变换为: [1,3] => [4,3]

2. 简单实际应用
1. 给你一个tensor，其shape含义为[class,students,scores]
2. 先需要将学生的所有成绩+5分

a = torch.rand(4,32,8) # torch.Size([4, 32, 8])
b = torch.tensor(5) # torch.Size([])
c = a + b

3. 错误实例

a = torch.rand(4,32,14,14)
b = torch.rand(2,32,14,14)
c = a + b
# RuntimeError: The size of tensor a (4) must match the size of tensor b (2) at non-singleton dimension 0
# 这里出错原因: 
# Dim 0 has dim，can NOT insert and expand to same
# Dim 0 has distinct dim, NOT size 1
# NOT broadcasting-able

4. 多种方式理解
   1. 一下例子我们用[b,c,h,w]
   2. [4,3,32,32] + [32,32] : 对于所有的batch,channels，都叠加相同的picture_map,相当于叠加相同的base，使其平移。
   3. [4,3,32,32] + [3,1,1] :对于RGB来说，增加值
   4. [4,3,32,32] + [1,1,1,1] :对于所有像素点都增加一个值