HRNet 源码分析

论文 Deep High-Resolution Representation Learning for Human Pose Estimation

也是 一个 top-down得对于人体姿态估计得检测方法。和Stack hourglass netword, CPN, MSPN等都大致一样。需要先学习一个人体检测器，将每个人都检测出来，然后在送进单个人体姿态估计模型。

从论文名字可以看出，HIgh-Resolution 高分辨率。

Stack hourglass netword, CPN, MSPN 模型结构都有一定得相似性，类型和Unet结构相似，加上一些残差。都经历一个下采样然后在进行上采样得过程。

然后HRNet 稍微有点不同，保持相同大小进行特征传递。每经过一个Transition是多出一个下采样得分支

如图：

 模型得代码如下：

class PoseHighResolutionNet(nn.Module):

    def __init__(self, cfg, **kwargs):
        pass

    def forward(self, x):
        # 下面将x缩小了4倍  两个conv得s=2
        x = self.conv1(x)
        x = self.bn1(x)
        x = self.relu(x)
        x = self.conv2(x)
        x = self.bn2(x)
        x = self.relu(x)
        x = self.layer1(x)

        x_list = []
        # 每经过一个transition 产生一个下采样分支
        for i in range(self.stage2_cfg['NUM_BRANCHES']):
            if self.transition1[i] is not None:
                x_list.append(self.transition1[i](x))
            else:
                x_list.append(x)
        y_list = self.stage2(x_list)

        x_list = []
        # 每经过一个transition 产生一个下采样分支
        for i in range(self.stage3_cfg['NUM_BRANCHES']):
            if self.transition2[i] is not None:
                x_list.append(self.transition2[i](y_list[-1]))
            else:
                x_list.append(y_list[i])
        y_list = self.stage3(x_list)

        x_list = []
        # 每经过一个transition 产生一个下采样分支
        for i in range(self.stage4_cfg['NUM_BRANCHES']):
            if self.transition3[i] is not None:
                x_list.append(self.transition3[i](y_list[-1]))
            else:
                x_list.append(y_list[i])
        y_list = self.stage4(x_list)

        # 最终模型只取 最后一个stage得 第一层得输出来
        x = self.final_layer(y_list[0])

        return x

 对于heatmaplabel的生成 和其他网络一样采用 2D高斯函数生成

    def generate_target(self, joints, joints_vis):
        '''
        :param joints:  [num_joints, 3]
        :param joints_vis: [num_joints, 3]
        :return: target, target_weight(1: visible, 0: invisible)
        '''
        target_weight = np.ones((self.num_joints, 1), dtype=np.float32)
        target_weight[:, 0] = joints_vis[:, 0]

        assert self.target_type == 'gaussian', \
            'Only support gaussian map now!'

        if self.target_type == 'gaussian':
            target = np.zeros((self.num_joints,
                               self.heatmap_size[1],
                               self.heatmap_size[0]),
                              dtype=np.float32)

            tmp_size = self.sigma * 3

            for joint_id in range(self.num_joints):
                feat_stride = self.image_size / self.heatmap_size
                mu_x = int(joints[joint_id][0] / feat_stride[0] + 0.5)
                mu_y = int(joints[joint_id][1] / feat_stride[1] + 0.5)
                # Check that any part of the gaussian is in-bounds
                ul = [int(mu_x - tmp_size), int(mu_y - tmp_size)]
                br = [int(mu_x + tmp_size + 1), int(mu_y + tmp_size + 1)]
                if ul[0] >= self.heatmap_size[0] or ul[1] >= self.heatmap_size[1] \
                        or br[0] < 0 or br[1] < 0:
                    # If not, just return the image as is
                    target_weight[joint_id] = 0
                    continue

                # # Generate gaussian
                # 生成高斯函数进行赋值
                size = 2 * tmp_size + 1
                x = np.arange(0, size, 1, np.float32)
                y = x[:, np.newaxis]
                x0 = y0 = size // 2
                # The gaussian is not normalized, we want the center value to equal 1
                g = np.exp(- ((x - x0) ** 2 + (y - y0) ** 2) / (2 * self.sigma ** 2))

                # Usable gaussian range
                g_x = max(0, -ul[0]), min(br[0], self.heatmap_size[0]) - ul[0]
                g_y = max(0, -ul[1]), min(br[1], self.heatmap_size[1]) - ul[1]
                # Image range
                img_x = max(0, ul[0]), min(br[0], self.heatmap_size[0])
                img_y = max(0, ul[1]), min(br[1], self.heatmap_size[1])

                v = target_weight[joint_id]
                if v > 0.5:
                    target[joint_id][img_y[0]:img_y[1], img_x[0]:img_x[1]] = \
                        g[g_y[0]:g_y[1], g_x[0]:g_x[1]]

        if self.use_different_joints_weight:
            target_weight = np.multiply(target_weight, self.joints_weight)

        return target, target_weight

 对于人体姿态估计网络的损失函数 基本都是一样的 MSE

class JointsMSELoss(nn.Module):
    def __init__(self, use_target_weight):
        super(JointsMSELoss, self).__init__()
        # 采用 MSE
        self.criterion = nn.MSELoss(reduction='mean')
        self.use_target_weight = use_target_weight

    def forward(self, output, target, target_weight):
        batch_size = output.size(0)
        num_joints = output.size(1)
        heatmaps_pred = output.reshape((batch_size, num_joints, -1)).split(1, 1)
        heatmaps_gt = target.reshape((batch_size, num_joints, -1)).split(1, 1)
        loss = 0

        for idx in range(num_joints):
            heatmap_pred = heatmaps_pred[idx].squeeze()
            heatmap_gt = heatmaps_gt[idx].squeeze()
            if self.use_target_weight:
                # 损失计算
                loss += 0.5 * self.criterion(
                    heatmap_pred.mul(target_weight[:, idx]),
                    heatmap_gt.mul(target_weight[:, idx])
                )
            else:
                loss += 0.5 * self.criterion(heatmap_pred, heatmap_gt)

        return loss / num_joints

从代码上看 其实和其他的top-down网络基本相似，貌似仅仅是在网络结构上进行了一定调整。

网络结构采用并行的传递方式。

基于inference代码 基本上和其他top-down网络代码相似。

有兴趣可以看看其他几篇top-down网络的源码分析

Rethinking on Multi-Stage Networks for Human Pose Estimation 源码分析_那时那月那人的博客-CSDN博客Rethinking on Multi-Stage Networks for Human Pose Estimation 源码分析https://blog.csdn.net/xiaoxu1025/article/details/127840623CPN-Cascaded Pyramid Network for Multi-Person Pose Estimation 源码分析_那时那月那人的博客-CSDN博客CPN 源码分析https://blog.csdn.net/xiaoxu1025/article/details/127838074Stacked Hourglass Networks for Human Pose Estimation 源码分析_那时那月那人的博客-CSDN博客Stacked Hourglass Networks for Human Pose Estimation 源码分析从源码分析 Stacked Hourglass Networks 在人体检测方向得具体实现https://blog.csdn.net/xiaoxu1025/article/details/127835690

到此 基于 top-down 方法的人体姿态检测 网络模型告一段落。

如果对采用bottom-up方式的HigherHRNet感兴趣可以移步下面链接

HigherHRNet 源码分析_那时那月那人的博客-CSDN博客