Template Matching with OpenCV and C#

http://www.prodigyproductionsllc.com/articles/programming/template-matching-with-opencv-and-c/

Apparently, a lot of people are interested in my OpenCV articles. In fact, my OpenCV articles receive more visits than any other page on this site. The OpenCV articles receive about 100 hits each every day. Along with those hits come a lot of emails asking for help with the reader’s own projects. One of the most commonly sought out projects is one that uses OpenCV and C# to locate specific objects within a video feed. For instance, many readers want to know how to do things like augmented reality or how to detect road signs within a video feed. So, I’m going to take a few minutes and show all of you how to do this very thing.

The first thing you are going to need is an image of the object you want to locate within a video feed. For this example, I am going to locate a typical U.S. road sign (Speed Limit 55). If you want to follow along with this example, here is the image I used. Just right click on the image and select “Save As…” and save it in your application’s bin > Debug folder. You’ll also want to go ahead and print the image while you’re at it since you’ll be needing it to test with.

Now that you have an image of the object you want to track in the video feed, you will need to load the image into a new IplImage object which will be your template.

IplImage tpl = Cv.LoadImage(“speedlimit55.jpg”, LoadMode.Color);

Next, you’ll need to go ahead and setup your CvCapture device and a CvWindow to display your results in.

CvCapture cap = CvCapture.FromCamera(1);
CvWindow w = new CvWindow(“Template Matching”);

At this point, you’ll also need to setup a few variables as place holders which we’ll use in just a minute.

CvPoint minloc, maxloc;
double minval, maxval;

Ok. So far so good. The next thing you’re going to need to do is to create a while-loop so that your application continues watching for input from the camera instead of closing immediately. If you’re working with a CvWindow, it’s best if you setup your while-loop to watch for a key press so that the loop is exited cleanly and the app will close.

while (CvWindow.WaitKey(10) < 0)

This is where the fun begins. Inside of your while-loop, setup a new IplImage to hold the frames from your camera and populate the variable using the QueryFrame() method from your capture device. Also, go ahead and create a new IplImage to house the result from OpenCVs MatchTemplate method. This result image needs to have a size that is the width & height of your video feed minus the width and height of your template plus 1. It will also need a bit depth of F32. Once you have those images setup, go ahead and pass them to the Cv.MatchTemplate method along with a reference to your template from above. The MatchTemplate takes 4 parameters. The first parameter is the image that holds your video feed frame. The second parameter is the template. The third parameter is your result output image from earlier. And, the fourth parameter is the method that you want to use to do your template matching with. Feel free to change the last parameter around to find what best fits your needs. I know that seems like a lot, but it isn’t bad. Here is the code for all of that.

IplImage img = cap.QueryFrame();
IplImage res = Cv.CreateImage(Cv.Size(img.Width – tpl.Width + 1, img.Height – tpl.Height + 1),BitDepth.F32, 1);
Cv.MatchTemplate(img, tpl, res, MatchTemplateMethod.CCoeff);

Are you still with me? If so, the next thing you’re gonna need is the actual location of the image inside of your video feed. With these coordinates, you can do whatever you want with the template as it appears in your video feed. To keep things simple, I’m just gonna draw a red box around the image inside my video feed.

Cv.MinMaxLoc(res, out minval, out maxval, out minloc, out maxloc, null);
Cv.Rectangle(img, Cv.Point(minloc.X, minloc.Y), Cv.Point(minloc.X + tpl.Width, minloc.Y + tpl.Height),CvColor.Red, 1, 0, 0);

That’s it. The only thing you have left to do is to draw the image back to your CvWindow by assigning it to the window Image property like so:

w.Image = img;

Oh yeah, you’ll also probably want to call Cv.ReleaseImage for input and output images. This helps free up memory and keeps your app running smoothly. If everything went according to plan, you should see something like this:

Also, if you want to take things one step further, you can implement some VRML images or other animated images to spice things up a bit. All you have to do is locate the template within the input image and draw your animated images over the top of your template. Here is a quick example I threw together a few weeks back for one of my readers. I’ll upload a video when I have more time. It’s actually pretty cool as the flower moves while the bee flies around in circles.

For those of you that are interested, here is the entire code for this article (minus the VRML loading).

view source print ?

01	using System;

02	using OpenCvSharp;

03

04	namespace EdgeDetect

05

{

06	class Template

07

{

08	public Template()

09

{

10	CvCapture cap = CvCapture.FromCamera(1);

11	CvWindow w = new CvWindow("Template Matching");

12

13	IplImage tpl = Cv.LoadImage("speedlimit55.jpg", LoadMode.Color);

14

15	CvPoint minloc, maxloc;

16	double minval, maxval;

17

18	while (CvWindow.WaitKey(10) < 0)

19

{

20	IplImage img = cap.QueryFrame();

21	IplImage res = Cv.CreateImage(Cv.Size(img.Width - tpl.Width + 1, img.Height - tpl.Height + 1), BitDepth.F32, 1);

22	Cv.MatchTemplate(img, tpl, res, MatchTemplateMethod.CCoeff);

23

24	Cv.MinMaxLoc(res, out minval, out maxval, out minloc, outmaxloc, null);

25	Cv.Rectangle(img, Cv.Point(minloc.X, minloc.Y), Cv.Point(minloc.X + tpl.Width, minloc.Y + tpl.Height), CvColor.Red, 1, 0, 0);

26

27	w.Image = img;

28

29	Cv.ReleaseImage(res);

30	Cv.ReleaseImage(img);

31

}

32

}

33

}

34

}