问题描述
目标检测模型输出的检测框存在内嵌情况。
原因分析与解决方法:
根据经验,第一感觉是后处理nms
部分出了问题。来看下对应的代码:
static float CalcIou(const vector<float> &box1, const vector<float> &box2)
{
float area1 = box1[6];
float area2 = box2[6];
float xx1 = max(box1[0], box2[0]);
float yy1 = max(box1[1], box2[1]);
float xx2 = min(box1[2], box2[2]);
float yy2 = min(box1[3], box2[3]);
float w = max(0.0f, xx2 - xx1 + 1);
float h = max(0.0f, yy2 - yy1 + 1);
float inter = w * h;
float ovr = inter /(area1 + area2 - inter);
return ovr;
}
static void MulticlassNms(vector<vector<float>>& bboxes, const vector<vector<float>>& vaildBox, float nmsThr)
{
for (auto &item : vaildBox) { /* score, xcenter, ycenter, w, h, classId */
float boxXCenter = item[XCENTER_IDX];
float boxYCenter = item[YCENTER_IDX];
float boxWidth = item[W_IDX];
float boxHeight = item[H_IDX];
float x1 = (boxXCenter - boxWidth / 2);
float y1 = (boxYCenter - boxHeight / 2);
float x2 = (boxXCenter + boxWidth / 2);
float y2 = (boxYCenter + boxHeight / 2);
float area = (x2 - x1 + 1) * (y2 - y1 + 1);
bool keep = true;
/* lx, ly, rx, ry, score, class id, area */
vector<float> bbox {x1, y1, x2, y2, item[SCORE_IDX], item[CLSAA_ID_IDX], area};
for (size_t j = 0; j < bboxes.size(); j++) {
if (CalcIou(bbox, bboxes[j]) > nmsThr) {
keep = false;
break;
}
}
if (keep) {
bboxes.push_back(bbox);
}
}
}
目前分析最可能的原因是nms
的nmsThr
设置过大,没能滤除重叠检测框,原来nmsThr
设置的为0.45
,现调整为0.1
。
检测框内嵌情况基本消失:
后续及思考
关于目标检测中的损失函数:IOU_Loss、GIOU_Loss、DIOU_Loss和CIOU_Loss,我也做了一篇笔记,有需要的可以了解下。
先给个结论,综合的看下各个Loss函数的不同点::
IOU_Loss:主要考虑检测框和目标框重叠面积。
GIOU_Loss:在IOU的基础上,解决边界框不重合时的问题。
DIOU_Loss:在IOU和GIOU的基础上,考虑边界框中心点距离的信息。
CIOU_Loss:在DIOU的基础上,考虑边界框宽高比的尺度信息。
此项目中用的是基本的IOU
,在推理性能足够的情况下,可以考虑使用DIOU
,下面也给出使用DIOU
的nms
代码:
static float CalcDiou(const vector<float>& box1, const vector<float>& box2) {
float x1 = min(box1[0], box2[0]);
float y1 = min(box1[1], box2[1]);
float x2 = max(box1[2], box2[2]);
float y2 = max(box1[3], box2[3]);
float c_x1 = (box1[0] + box1[2]) / 2.0;
float c_y1 = (box1[1] + box1[3]) / 2.0;
float c_x2 = (box2[0] + box2[2]) / 2.0;
float c_y2 = (box2[1] + box2[3]) / 2.0;
float dist_center = sqrt((c_x1 - c_x2) * (c_x1 - c_x2) + (c_y1 - c_y2) * (c_y1 - c_y2));
float w = max(0.0f, x2 - x1);
float h = max(0.0f, y2 - y1);
float intersection = w * h;
float area1 = (box1[2] - box1[0]) * (box1[3] - box1[1]);
float area2 = (box2[2] - box2[0]) * (box2[3] - box2[1]);
float union_area = area1 + area2 - intersection;
float diou = intersection / union_area - dist_center * dist_center / (union_area * union_area);
return diou;
}
static void MulticlassNms(vector<vector<float>>& bboxes, const vector<vector<float>>& vaildBox, float nmsThr)
{
for (auto &item : vaildBox) { /* score, xcenter, ycenter, w, h, classId */
float boxXCenter = item[XCENTER_IDX];
float boxYCenter = item[YCENTER_IDX];
float boxWidth = item[W_IDX];
float boxHeight = item[H_IDX];
float x1 = (boxXCenter - boxWidth / 2);
float y1 = (boxYCenter - boxHeight / 2);
float x2 = (boxXCenter + boxWidth / 2);
float y2 = (boxYCenter + boxHeight / 2);
float area = (x2 - x1 + 1) * (y2 - y1 + 1);
bool keep = true;
vector<float> bbox {x1, y1, x2, y2, item[SCORE_IDX], item[CLSAA_ID_IDX], area};
for (size_t j = 0; j < bboxes.size(); j++) {
if (CalcDiou(bbox, bboxes[j]) > nmsThr) {
keep = false;
break;
}
}
if (keep) {
bboxes.push_back(bbox);
}
}
}
有读者会有疑问,这里为什么不用CIOU_nms
,而用DIOU_nms
?
答:因为CIOU_loss
,是在DIOU_loss
的基础上,添加的影响因子,包含groundtruth
标注框的信息,在训练时用于回归。
但在测试过程中,并没有groundtruth
的信息,不用考虑影响因子,因此直接用DIOU_nms
即可。
参考文档
https://blog.csdn.net/nan355655600/article/details/106246625