《Master Opencv...读书笔记》图像特征点匹配

这是本书的第三章，本文主要关注其中的特征点匹配及去除失配点的方法。

主要功能：对统一物体拍了两张照片，只是第二张图片有选择和尺度的变化。现在要分别对两幅图像提取特征点，然后将这些特征点匹配，使其尽量相互对应

下面，本文通过采用surf特征，分别使用Brute-force matcher和Flann-based matcher对特征点进行相互匹配：

第一段代码摘自opencv官网的教程：

// test2.cpp : 定义控制台应用程序的入口点。
//

#include "stdafx.h"
#include <iostream>
#include "opencv2/core/core.hpp"
#include "opencv2/features2d/features2d.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/nonfree/features2d.hpp"
#include "opencv2/calib3d/calib3d.hpp"
#include "opencv2/imgproc/imgproc.hpp"

using namespace cv;
using namespace std;


int _tmain(int argc, _TCHAR* argv[])
{
	Mat img_1 = imread( "haha1.jpg", CV_LOAD_IMAGE_GRAYSCALE );
	Mat img_2 = imread( "haha2.jpg", CV_LOAD_IMAGE_GRAYSCALE );

	if( !img_1.data || !img_2.data )
	{ return -1; }

	//-- Step 1: Detect the keypoints using SURF Detector
	//Threshold for hessian keypoint detector used in SURF
	int minHessian = 15000;

	SurfFeatureDetector detector( minHessian );

	std::vector<KeyPoint> keypoints_1, keypoints_2;

	detector.detect( img_1, keypoints_1 );
	detector.detect( img_2, keypoints_2 );

	//-- Step 2: Calculate descriptors (feature vectors)
	SurfDescriptorExtractor extractor;

	Mat descriptors_1, descriptors_2;

	extractor.compute( img_1, keypoints_1, descriptors_1 );
	extractor.compute( img_2, keypoints_2, descriptors_2 );

	//-- Step 3: Matching descriptor vectors with a brute force matcher
	BFMatcher matcher(NORM_L2,false);
	vector< DMatch > matches;
	matcher.match( descriptors_1, descriptors_2, matches );
	
	//-- Draw matches
	Mat img_matches;
	drawMatches( img_1, keypoints_1, img_2, keypoints_2, matches, img_matches );

	//-- Show detected matches
	imshow("Matches", img_matches );

	waitKey(0);

	return 0;
}

    Brute-forcedescriptor matcher. For each descriptor in the first set, this matcher findsthe closest descriptor in the second set by trying each one. This descriptormatcher supports masking permissible matches of descriptor sets.

   上面是那个bfmatcher的介绍，各位自己体会。我上面代码把surf的阈值故意设置的很大，否则图片全是线，没法看。上面代码的运行结果：

   

如图，有很多匹配失误。书中对匹配失误有两种定义：

False-positivematches：特征点健全，只是对应关系错误；

False-negativematches：特征点消失，导致对应关系错误；

我们只关心第一种情况，解决方案有两种，一种是将BFMatcher构造函数的第二个参数设置为true，作为cross-match filter。

BFMatcher matcher(NORM_L2,true);

他的思想是：to match train descriptors with the query set and viceversa.Only common matches for these two matches are returned. Such techniquesusually produce best results with minimal number of outliers when there areenough matches

效果图：

可以看到匹配错误的线段比第一副图少了。

第二种Flann-based matcher：uses the fastapproximate nearest neighbor search algorithm to find correspondences (it usesfast third-party library for approximate nearest neighbors library for this).

用法：

FlannBasedMatcher matcher1;
matcher1.match(descriptors_1, descriptors_2, matches );

效果图：

下面介绍第二种去除匹配错误点方法，KNN-matching：We performKNN-matching first with K=2. Two nearest descriptors are returned for eachmatch.The match is returned only if the distance ratio between the first andsecond matches is big enough (the ratio threshold is usually near two).

// test2.cpp : 定义控制台应用程序的入口点。
//

#include "stdafx.h"
#include <iostream>
#include "opencv2/core/core.hpp"
#include "opencv2/features2d/features2d.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/nonfree/features2d.hpp"
#include "opencv2/calib3d/calib3d.hpp"
#include "opencv2/imgproc/imgproc.hpp"

using namespace cv;
using namespace std;


int _tmain(int argc, _TCHAR* argv[])
{
	Mat img_1 = imread( "test.jpg", CV_LOAD_IMAGE_GRAYSCALE );
	Mat img_2 = imread( "test1.jpg", CV_LOAD_IMAGE_GRAYSCALE );

	if( !img_1.data || !img_2.data )
	{ return -1; }

	//-- Step 1: Detect the keypoints using SURF Detector
	//Threshold for hessian keypoint detector used in SURF
	int minHessian = 1500;

	SurfFeatureDetector detector( minHessian );

	std::vector<KeyPoint> keypoints_1, keypoints_2;

	detector.detect( img_1, keypoints_1 );
	detector.detect( img_2, keypoints_2 );

	//-- Step 2: Calculate descriptors (feature vectors)
	SurfDescriptorExtractor extractor;

	Mat descriptors_1, descriptors_2;

	extractor.compute( img_1, keypoints_1, descriptors_1 );
	extractor.compute( img_2, keypoints_2, descriptors_2 );

	//-- Step 3: Matching descriptor vectors with a brute force matcher
	BFMatcher matcher(NORM_L2,false);
	//FlannBasedMatcher matcher1;
	vector< DMatch > matches;
	vector<vector< DMatch >> matches2;
	matcher.match( descriptors_1, descriptors_2, matches );
	
	//matcher1.match(descriptors_1, descriptors_2, matches );

	const float minRatio = 1.f / 1.5f;
	matches.clear();
	matcher.knnMatch(descriptors_1, descriptors_2,matches2,2);
	for (size_t i=0; i<matches2.size(); i++)
	{
		const cv::DMatch& bestMatch = matches2[i][0];
		const cv::DMatch& betterMatch = matches2[i][1];
		float distanceRatio = bestMatch.distance /betterMatch.distance;
		// Pass only matches where distance ratio between
		// nearest matches is greater than 1.5
		// (distinct criteria)
		if (distanceRatio < minRatio)
		{
			matches.push_back(bestMatch);
		}
	}

	//-- Draw matches
	Mat img_matches;
	drawMatches( img_1, keypoints_1, img_2, keypoints_2, matches, img_matches );

	//-- Show detected matches
	imshow("Matches", img_matches );

	waitKey(0);

	return 0;
}

这里，我把surf阈值设为1500了，效果图：

最后，老外书中又提到了使用单应性矩阵变换来进一步细化结果：

        //refine
	const int minNumberMatchesAllowed = 8;

	if (matches.size() < minNumberMatchesAllowed)
		return false;
	// Prepare data for cv::findHomography
	std::vector<cv::Point2f> srcPoints(matches.size());
	std::vector<cv::Point2f> dstPoints(matches.size());

	for (size_t i = 0; i < matches.size(); i++)
	{
		//cout<<i<<' '+matches[i].trainIdx<<' '+matches[i].queryIdx<<endl;
		srcPoints[i] = keypoints_1[matches[i].trainIdx].pt;
		dstPoints[i] = keypoints_2[matches[i].queryIdx].pt;
		
	}

	// Find homography matrix and get inliers mask
	std::vector<unsigned char> inliersMask(srcPoints.size());
	Mat homography = findHomography(srcPoints, dstPoints, CV_FM_RANSAC, 3.0f, inliersMask);

	std::vector<cv::DMatch> inliers;
	for (size_t i=0; i<inliersMask.size(); i++)
	{
		if (inliersMask[i])
			inliers.push_back(matches[i]);
	}

	matches.swap(inliers);

这段代码直接承接上一段代码即可。效果图：

这次文章写得太没劲了，主要老外书上这章本身就没意思，随书代码每次都一大堆，第三章还要用到opengl，不玩了！

以后一段日子，专心图像基本特征提取的实现和android基础功能的实现。opencv先放一边了