practical python and opencv_Practical Python and OpenCV + Case Studies

conference：

• CVPR - Computer Vision and Pattern Recognition

• ICCV - International Conference on Computer Vision

• ECCV - European Conference on Computer Vision

• BMVC - British Machine Vision Conference

• ICIP - IEEE International Conference on Image Processing

Beginner Books：

• Programming Computer Vision with Python: Tools and algorithms for analyzing images by Jan Erik Solem

• Practical Computer Vision with SimpleCV : The Simple Way to Make Technology See by Kurt Demaagd, Anthony Oliver, Nathan Oostendorp, and Katherine Scott

• OpenCV Computer Vision with Python by Joseph Howse

• Learning OpenCV: Computer Vision with the OpenCV Library by Gary Bradski and Adrian Kaehler

• OpenCV 2 Computer Vision Application Programming Cookbook by Robert Laganière

• Mastering OpenCV with Practical Computer Vision Projects by Daniel Lélis Baggio, Shervin Emami,

David Millán Escrivá, Khvedchenia Ievgen, Jasonl Saragih, and Roy Shilkrot

• SciPy and NumPy: An Overview for Developers by Eli Bressert

Textbooks：

• Computer Vision: A Modern Approach (2nd Edition) by David A. Forsyth and Jean Ponce

• Computer Vision by Linda G. Shapiro and George C. Stockman

• Computer Vision: Algorithms and Applications by Richard Szeliski

• Algorithms for Image Processing and Computer Vision by J. R. Parker

• Computer Vision: Models, Learning, and Inference by Dr Simon J. D. Prince

• Computer and Machine Vision, Fourth Edition: Theory, Algorithms, Practicalities by E. R. Davies

Python Libraries

When I first became interested in computer vision and image search engines over eight

years ago, I had no idea where to start. I didn’t know which language to use, I didn’t

know which libraries to install, and the libraries I found I didn’t know how to use. I WISH

there had been a list like this one, detailing the best Python libraries to use for image

processing, computer vision, and image search engines.

This list is by no means complete or exhaustive. It’s just my favorite Python libraries that

I use each and everyday for computer vision and image search engines. If you think that

I’ve left an important one out, please leave me an email at adrian@pyimagesearch.com.

NumPy

NumPy is a library for the Python programming language that (among other things)

provides support for large, multi-dimensional arrays. Why is that important? Using

NumPy, we can express images as multi-dimensional arrays. Representing images as

NumPy arrays is not only computational and resource efficient, but many other image

processing and machine learning libraries use NumPy array representations as well.

Furthermore, by using NumPy’s built-in high-level mathematical functions, we can

quickly perform numerical analysis on an image.

SciPy

Going hand-in-hand with NumPy, we also have SciPy. SciPy adds further support for

scientific and technical computing. One of my favorite sub-packages of SciPy is the

spatial package which includes a vast amount of distance functions and a kd-tree

implementation. Why are distance functions important? When we “describe” an image,

we perform feature extraction. Normally after feature extraction an image is represented

by a vector (a list) of numbers. In order to compare two images, we rely on distance

functions, such as the Euclidean distance. To compare two arbitrary feature vectors, we

simply compute the distance between their feature vectors. In the case of the Euclidean

distance, the smaller the distance the more “similar” the two images are.

matplotlib

Simply put, matplotlib is a plotting library. If you’ve ever used MATLAB before, you’ll

probably feel very comfortable in the matplotlib environment. When analyzing images,

we’ll make use of matplotlib, whether plotting the overall accuracy of search systems or

simply viewing the image itself, matplotlib is a great tool to have in your toolbox.

PIL and Pillow

These two packages are good and what they do: simple image manipulations, such as

resizing, rotation, etc. If you need to do some quick and dirty image manipulations

definitely check out PIL and Pillow, but if you’re serious about learning about image

processing, computer vision, and image search engines, I would highly recommend that

you spend your time playing with OpenCV and SimpleCV instead.

OpenCV

If NumPy’s main goal is large, efficient, multi-dimensional array representations, then,

by far, the main goal of OpenCV is real-time image processing. This library has been

around since 1999, but it wasn’t until the 2.0 release in 2009 did we see the incredible

NumPy support. The library itself is written in C/C++, but Python bindings are provided

when running the installer. OpenCV is hands down my favorite computer vision library,

but it does have a learning curve. Be prepared to spend a fair amount of time learning

the intricacies of the library and browsing the docs (which have gotten substantially

better now that NumPy support has been added). If you are still testing the computer

vision waters, you might want to check out the SimpleCV library mentioned below,

which has a substantially smaller learning curve.

SimpleCV

The goal of SimpleCV is to get you involved in image processing and computer vision

as soon as possible. And they do a great job at it. The learning curve is substantially

smaller than that of OpenCV, and as their tagline says, “it’s computer vision made

easy”. That all said, because the learning curve is smaller, you don’t have access to as

many of the raw, powerful techniques supplied by OpenCV. If you’re just testing the

waters, definitely try this library out.

mahotas

Mahotas, just as OpenCV and SimpleCV, rely on NumPy arrays. Much of the

functionality implemented in Mahotas can be found in OpenCV and/or SimpleCV, but in

some cases, the Mahotas interface is just easier to use, especially when it comes to

their features package.

scikit-learn

Alright, you got me, Scikit-learn isn’t an image processing or computer vision library —

it’s a machine learning library. That said, you can’t have advanced computer vision

techniques without some sort of machine learning, whether it be clustering, vector

quantization, classification models, etc. Scikit-learn also includes a handful of image

feature extraction functions as well.

scikit-image

Scikit-image is fantastic, but you have to know what you are doing to effectively use this

library -- and I don’t mean this in a “there is a steep learning curve” type of way. The

learning curve is actually quite low, especially if you check out their gallery. The

algorithms included in scikit-image (I would argue) follow closer to the state-of-the-art in

computer vision. New algorithms right from academic papers can be found in scikit-

image, but in order to (effectively) use these algorithms, you need to have developed

some rigor and understanding in the computer vision field. If you already have some

experience in computer vision and image processing, definitely check out scikit-image;

otherwise, I would continue working with OpenCV and SimpleCV to start.

ilastik

I’ll be honest. I’ve never used ilastik. But through my experiences at computer vision

conferences, I’ve met a fair amount of people who do, so I felt compelled to put it in this

list. Ilastik is mainly for image segmentation and classification and is especially geared

towards the scientific community.

pprocess

Extracting features from images is inherently a parallelizable task. You can reduce the

amount of time it takes to extract features from an entire dataset by using a

multithreading/multitasking library. My favorite is pprocess, due to the simple nature I

need it for, but you can use your favorite.

h5py

The h5py library is the de-facto standard in Python to store large numerical datasets.

The best part? It provides support for NumPy arrays. So, if you have a large dataset

represented as a NumPy array, and it won’t fit into memory, or if you want efficient,

persistent storage of NumPy arrays, then h5py is the way to go. One of my favorite

techniques is to store my extracted features in a h5py dataset and then apply scikit-

learn’s MiniBatchKMeans to cluster the features. The entire dataset never has to be

entirely loaded off disk at once and the memory footprint is extremely small, even for

thousands of feature vectors.