语义分割自己收集的一些tricks

https://www.bilibili.com/video/BV11E411y7Dr/?spm_id_from=333.788.videocard.12

ResNeSt: Split-Attention Networks（ResNet改进版本）
（1）大的min batch，使用cosine学习率衰减策略。warm up。BN层参数设置。
（2）标签平滑
（3）自动增强
（4）mixup训练
（5）大的切割设置
（6）正则化

Bisenet
https://github.com/CoinCheung/BiSeNet
These are the tricks that I find might be useful:
use online hard example mining loss. This let the model be trained more efficiently.
do not add weight decay when bn parameters and bias parameters of nn.Conv2d or nn.Linear are tuned.
use a 10 times larger lr at the model output layers.
use crop evaluation. We do not want the eval scale to be too far away from the train scale, so we crop the chips from the images to do evaluation and then combine the results to make the final prediction.
multi-scale training and multi-scale-flip evaluating. On each scale, the scores of the original image and its flipped version are summed up, and then the exponential of the sum is computed to be the prediction of this scale.
warmup of 1000 iters to make sure the model better initialized.

torch-cv
# Zero-initialize the last BN in each residual branch,
# so that the residual branch starts with zeros, and each residual block behaves like an identity.
# This improves the model by 0.2~0.3% according to https://arxiv.org/abs/1706.02677
if zero_init_residual:
    for m in self.modules():
        if isinstance(m, Bottleneck):
            nn.init.constant_(m.bn3.weight, 0)
        elif isinstance(m, BasicBlock):
            nn.init.constant_(m.bn2.weight, 0)

图像分类常使用的trick：https://mp.weixin.qq.com/s?__biz=MzU2NTc5MjIwOA==&mid=2247497397&idx=2&sn=54a27c6c60637b280d0c2442db8fb0df&chksm=fcb4ea0acbc3631ca7277505585478f1fc1c342fcbfebc33dd5ad9ce13f335eeec33f61be6ce&scene=126&sessionid=1600855122&key=461d0d4ae0561348c3fd7bcc195eb51d6f250dbeaf26aaedf745a8a001895b7d555f9be8ba988c1e8fedcbac3e774b35f583cb28633d88ce18f17002b3f554806dc08d23352ffc7a3474f29857bd4f87655eb7c70e0e5fab25d3f0baab77bdd67f15c9fac6d782cd83e6a34e6033da6c64091182ed0f39f9835e7b559c594ca2&ascene=1&uin=MjM3NDE0MTk3NA%3D%3D&devicetype=Windows+10&version=62080079&lang=zh_CN&exportkey=ARMcy9aXVRc0tpxWJHLVnMU%3D&pass_ticket=M9diu1GtzxssCtMaW3vXAOgAzUKchtQ3CFSN97Hxr4vRbV3jeKM2QEUVK%2Fyr1XCr&wx_header=0