微信二维码识别研究

微信二维码识别研究

---

---

前言

由于工作原因，需要熟悉二维码解析流程，正好微信在Opencv4.5.2中开源了wechat_qrcode源码，现对第一阶段调研做下记录。

---

提示：以下是本篇文章正文内容，下面案例可供参考

一、在Win10下编译OpenCV4.5.2（含opencv_contrib）

本人使用的是vs2019编译，感谢如下博文：
win10 下编译用于 Visual Studio 2019 的 OpenCV4.5.2（含 opencv_contrib 扩展模块）附编译好的OpenCV（图文）

二、调试步骤

1.链接OpenCV

编译opencv生成解决方案后，右键CMakeTargets下的INSTALL->仅用于项目->仅生成INSTALL,顺利的话，install目录如下：

2.编写Demo

另起VS的Demo工程，代码如下（本人根据自己需要修改了opencv源码，接口增加了参数用于对比测试）：

#include <iostream>
#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>

#include <opencv2/wechat_qrcode.hpp>

using namespace std;
using namespace cv;

int main(int argc, char* argv[])
{
    for (size_t k = 0; k < 84; k++)
    {
        for (size_t i = 1; i < 3; i++)
        {
            string imgPath = "D:\\aaWorkspace\\WeChatQRCodeDemo\\QRCodeDemo\\qrcode_compare\\" + to_string(k) + "-" + to_string(i) + ".jpg";
            Mat img = imread(imgPath);
            if (!img.data)
            {
                cout << "Failed to read image!" << endl;
                continue;
            }
            string path = "D:\\aaWorkspace\\WeChatQRCodeDemo\\QRCodeDemo\\steps\\img" + to_string(k * 10 + i) + "\\0-src.jpg";
            imwrite(path, img);

            Ptr<wechat_qrcode::WeChatQRCode> detector;

            try {
                detector = makePtr<wechat_qrcode::WeChatQRCode>(
                    "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\detect.prototxt",
                    "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\detect.caffemodel",
                    "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\sr.prototxt",
                    "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\sr.caffemodel");
            }
            catch (const std::exception& e) {
                cout <<
                    "\n---------------------------------------------------------------\n"
                    "Failed to initialize WeChatQRCode.\n"
                    "Please, download 'detector.*' and 'sr.*' from\n"
                    "https://github.com/WeChatCV/opencv_3rdparty/tree/wechat_qrcode\n"
                    "and put them into the current directory.\n"
                    "---------------------------------------------------------------\n";
                cout << e.what() << endl;
                return 0;
            }
            string prevstr = "";
            vector<Mat> points;
            auto res = detector->detectAndDecode(img, points, k * 10 + i);
            int j = 0;
            for (const auto& t : res)
            {
                cout << endl << "qrcode: " << t << endl;
                putText(img, t, cv::Point(points[j].at<float>(0, 0), points[j].at<float>(0, 1)), FONT_HERSHEY_COMPLEX, 2, Scalar(0, 255, 0), 2);
                j++;
            }
            for (const auto& point : points)
            {
                rectangle(img, cv::Point(point.at<float>(0, 0), point.at<float>(0, 1)), cv::Point(point.at<float>(2, 0), point.at<float>(2, 1)), Scalar(0, 0, 255));
            }
            path = "D:\\aaWorkspace\\WeChatQRCodeDemo\\QRCodeDemo\\steps\\img" + to_string(k * 10 + i) + "\\9-result.jpg";
            imwrite(path, img);
        }
    }
    waitKey(0);
    return 0;
}

3.为Demo配置环境（Debug环境）

Visual Studio配置第三方库的常规操作，如下：

（1）项目右键属性->VC++目录->包含目录：

D:\opencv-4.5.2-contrib\opencv\newbuild\install\include
D:\opencv-4.5.2-contrib\opencv\newbuild\install\include\opencv2

（2）项目右键属性->VC++目录->库目录：

D:\opencv-4.5.2-contrib\opencv\newbuild\install\x64\vc16\lib

（3）项目右键属性->链接器->输入：

opencv_world452d.lib

（4）将步骤1目录中生成的opencv_world452d.dll（\install\x64\vc16\bin）放置到demo.exe同级

（5）在opencv工程的INSTALL项目右键属性->调试->命令，输入demo.exe位置：

D:\aaWorkspace\WeChatQRCodeDemo\QRCodeDemo\x64\Debug\demo.exe

然后就可以断点调试opencv中相关源码了

4.修改OpenCV源码

比如我们需要保存检测的二维码的候选区域图片，添加如下代码：

（1）修改之后保存文件，对opencv工程右键->生成解决方案，注意不要点击重新生成解决方案
（2）CMakeTargets下的INSTALL右键->仅用于项目->仅生成INSTALL，重新生成opencv_world452d.dll，替换demo.exe同级目录下文件

---

三、微信二维码解码流程，浅见

1.二维码定位

try {
        detector = makePtr<wechat_qrcode::WeChatQRCode>(
           "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\detect.prototxt",
           "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\detect.caffemodel",
           "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\sr.prototxt",
           "D:\\opencv-4.5.2-contrib\\opencv\\newbuild\\downloads\\wechat_qrcode\\sr.caffemodel");
    }

int WeChatQRCode::Impl::applyDetector(const Mat& img, vector<Mat>& points) {
    int img_w = img.cols;
    int img_h = img.rows;

    // hard code input size
    int minInputSize = 400;
    float resizeRatio = sqrt(img_w * img_h * 1.0 / (minInputSize * minInputSize));
    int detect_width = img_w / resizeRatio;
    int detect_height = img_h / resizeRatio;

    points = detector_->forward(img, detect_width, detect_height);

    return 0;
}

微信使用训练好的深度学习的SSD目标检测模型推理出二维码矩形候选区域，又快又准，模型也很小

2.二维码解码

（1）解码前对候选区域做padding，宽高扩大10%

Mat WeChatQRCode::Impl::cropObj(const Mat& img, const Mat& point, Align& aligner) {
    // make some padding to boost the qrcode details recall.
    float padding_w = 0.1f, padding_h = 0.1f;
    auto min_padding = 15;
    auto cropped = aligner.crop(img, point, padding_w, padding_h, min_padding);
    return cropped;
}

（2）缩放尺寸，多次尝试

// empirical rules
vector<float> WeChatQRCode::Impl::getScaleList(const int width, const int height) {
    if (width < 320 || height < 320) return {1.0, 2.0, 0.5};
    if (width < 640 && height < 640) return {1.0, 0.5};
    return {0.5, 1.0};
    //yqg
    //return { 1.0, 2.0, 0.5 };
}

（3）在ZXing的基础上扩展，采用4种二值化方式轮流尝试解码

enum BINARIZER {
        Hybrid = 0,
        FastWindow = 1,
        SimpleAdaptive = 2,
        AdaptiveThreshold = 3
    };

// Four Binarizers
    int tryBinarizeTime = 4;
    for (int tb = 0; tb < tryBinarizeTime; tb++) {
        if (source == NULL || height * width > source->getMaxSize()) {
            source = ImgSource::create(scaled_img_zx.data(), width, height);
        } else {
            source->reset(scaled_img_zx.data(), width, height);
        }
        int ret = TryDecode(source, zx_result, scaled);
        if (!ret) {
            result = zx_result->getText()->getText();
            return ret;
        }
        else//yqg
        {
            string scaledPath = "D:\\aaWorkspace\\WeChatQRCodeDemo\\QRCodeDemo\\steps\\img" + to_string(index) + "\\5-" + to_string(qr) + "-scaled-" + to_string(scaled) + "-try-Binarize-" + to_string(tb) + "-failed.jpg";
            imwrite(scaledPath, src);
        }
        // try different binarizers
        binarizer_mgr_.SwitchBinarizer();
    }
}

四种二值化器下显示效果：

（4）得到合适的二值化图片后开始解码，寻找三个定位点，透视变换等，又要忙别的了，抽空再看吧。。。