基于HyperLPR的车牌识别（十一）

2021SC@SDUSC

源代码下载地址：HyperLPR: HyperLRP是一个开源的、基于深度学习高性能中文车牌识别库，由北京智云视图科技有限公司开发，支持PHP、C/C++、Python语言，Windows/Mac/Linux/Android/IOS 平台。

源码配置的详情见第一篇分析

这一次的分析是pipline.py的最后几行，主要是对于任务的收尾工作。以下是代码的实现：

# image_gray = horizontalSegmentation(image_gray)
        cv2.imshow("image_gray",image_gray)
        # cv2.waitKey()

        cv2.imwrite("./"+str(j)+".jpg",image_gray)
        # cv2.imshow("image",image_gray)
        # cv2.waitKey(0)
        print("校正",time.time() - t1,"s")
        # cv2.imshow("image,",image_gray)
        # cv2.waitKey(0)
        t2 = time.time()
        val = segmentation.slidingWindowsEval(image_gray)
        # print val
        print("分割和识别",time.time() - t2,"s")
        if len(val)==3:
            blocks, res, confidence = val
            if confidence/7>0.7:
                image =  drawRectBox(image,rect,res)
                res_set.append(res)
                for i,block in enumerate(blocks):

                    block_ = cv2.resize(block,(25,25))
                    block_ = cv2.cvtColor(block_,cv2.COLOR_GRAY2BGR)
                    image[j * 25:(j * 25) + 25, i * 25:(i * 25) + 25] = block_
                    if image[j*25:(j*25)+25,i*25:(i*25)+25].shape == block_.shape:
                        pass


            if confidence>0:
                print("车牌:",res,"置信度:",confidence/7)
            else:
                pass

首先是使用了cv2的函数来对图片进行一系列的处理，函数我在之前的文章中提到过，就不再多说了。

最主要的是：

val = segmentation.slidingWindowsEval(image_gray)

源代码是：

def slidingWindowsEval(image):
    windows_size = 16;
    stride = 1
    height= image.shape[0]
    t0 = time.time()
    data_sets = []

    for i in range(0,image.shape[1]-windows_size+1,stride):
        data = image[0:height,i:i+windows_size]
        data = cv2.resize(data,(23,23))
        # cv2.imshow("image",data)
        data = cv2.equalizeHist(data)
        data = data.astype(np.float)/255
        data=  np.expand_dims(data,3)
        data_sets.append(data)

    res = model2.predict(np.array(data_sets))
    print("分割",time.time() - t0)

    pin = res
    p = 1 -  (res.T)[1]
    p = f.gaussian_filter1d(np.array(p,dtype=np.float),3)
    lmin = l.argrelmax(np.array(p),order = 3)[0]
    interval = []
    for i in range(len(lmin)-1):
        interval.append(lmin[i+1]-lmin[i])

    if(len(interval)>3):
        mid  = get_median(interval)
    else:
        return []
    pin = np.array(pin)
    res =  searchOptimalCuttingPoint(image,pin,0,mid,3)

    cutting_pts = res[1]
    last =  cutting_pts[-1] + mid
    if last < image.shape[1]:
        cutting_pts.append(last)
    else:
        cutting_pts.append(image.shape[1]-1)
    name = ""
    confidence =0.00
    seg_block = []
    for x in range(1,len(cutting_pts)):
        if x != len(cutting_pts)-1 and x!=1:
            section = image[0:36,cutting_pts[x-1]-2:cutting_pts[x]+2]
        elif  x==1:
            c_head = cutting_pts[x - 1]- 2
            if c_head<0:
                c_head=0
            c_tail = cutting_pts[x] + 2
            section = image[0:36, c_head:c_tail]
        elif x==len(cutting_pts)-1:
            end = cutting_pts[x]
            diff = image.shape[1]-end
            c_head = cutting_pts[x - 1]
            c_tail = cutting_pts[x]
            if diff<7 :
                section = image[0:36, c_head-5:c_tail+5]
            else:
                diff-=1
                section = image[0:36, c_head - diff:c_tail + diff]
        elif  x==2:
            section = image[0:36, cutting_pts[x - 1] - 3:cutting_pts[x-1]+ mid]
        else:
            section = image[0:36,cutting_pts[x-1]:cutting_pts[x]]
        seg_block.append(section)
    refined = refineCrop(seg_block,mid-1)

    t0 = time.time()
    for i,one in enumerate(refined):
        res_pre = cRP.SimplePredict(one, i )
        # cv2.imshow(str(i),one)
        # cv2.waitKey(0)
        confidence+=res_pre[0]
        name+= res_pre[1]
    print("字符识别",time.time() - t0)

    return refined,name,confidence

slidingWindowsEval是基于滑动窗口的字符分割与识别。
里介绍一篇较为经典的论文(End-to-end text recognition with convolutional neural networks)在Cousera上Andrew Ng的Machine Learning课程中也提到了这种方法。在OpenCV的text模块中也有对应的实现。

它的主要思想是利用一个训练好的正负样本分类器来进行在图像上滑动然后产生概率response图，然后对raw response进行nms(非极大值抑制)。在确定字符bdbox,数目之后使用类似于viterbi算法来获取最佳分割路径。

模板匹配法根据车牌的标准样式，结合字符的结构、尺寸和间距特征，设计出一个固定的模板和一个用来度量模板匹配程度的评价函数，然后将该模板在归一化后的图像中从左至右滑动，每滑动一次计算对应的评价值，最终选出匹配程度最高的模板滑动位置作为字符分割的位置。模板匹配法的缺点是它对车牌图像要求较高，当车牌倾斜角度很小、边框去除干净、图像噪声较少，字符尺寸和字符间距比较标准时，该算法速度快、分割效果好；而当车牌图像质量较为一般时，该算法的分割效果常常不尽人意。
HyperLpr中在模板匹配的基础上，同样尝试了投影法解决字符分割的问题，大致分为如下几步：
1.在图像滤波的基础上，获得投影

2.利用获得的投影值寻找分割点