Facial Detection with openCV and Python
We can use openCV for facial detection. OpenCV is written in C, but there are bindings for Python and actually PHP.
Use Cases:
- Cropping user-uploaded images, without cutting out faces. Honey.is does this effectively for user profile images.
How To:
I'm going to do this using Python. We can do this by installing openCV and the Python bindings and then writing a quick script to detect faces in an image.
Note: I didn't use virtualenv
because of the ease of using apt-get
packages and because I have Vagrant to spin up and destroy VM's easily enough to not worry about using virtualenv
.
We'll use cv2
, instead of the older cv
.
Server Install
On Ubuntu 12.04 LTS, I ran this:
$ sudo apt-get update
$ sudo apt-get install -y vim build-essential python-software-properties # The Basics
$ sudo apt-get install -y python-opencv python-numpy python-scipy # OpenCV items
Download Face Detection Trainer
You'll need an XML file with training data for the program. Download the Haar Cascade Frontal Face xml file.
$ wget http://eclecti.cc/files/2008/03/haarcascade_frontalface_alt.xml
The Script
Here is the Python used to check images for faces.
import cv2
def detect(path):
img = cv2.imread(path)
cascade = cv2.CascadeClassifier("/vagrant/detect/haarcascade_frontalface_alt.xml")
rects = cascade.detectMultiScale(img, 1.3, 4, cv2.cv.CV_HAAR_SCALE_IMAGE, (20,20))
if len(rects) == 0:
return [], img
rects[:, 2:] += rects[:, :2]
return rects, img
def box(rects, img):
for x1, y1, x2, y2 in rects:
cv2.rectangle(img, (x1, y1), (x2, y2), (127, 255, 0), 2)
cv2.imwrite('/vagrant/img/detected.jpg', img);
rects, img = detect("/vagrant/img/one.jpg")
box(rects, img)
The Results
The original image:
With faces detected:
Note: One of the faces was not detected! If you take a look, you'll see that the person on the far right's face is covered in shadow and has less distinguishable features (no glasses).
It's certainly not perfect. Facial detection is not an easy nor exact science.
Also noteworthy is that the Haar Cascade XML file used is meant to detect "frontal faces", rather than faces in profile. There is detection available for other features such as hands. See the Gist below for some of those XML files.
To Do
- Crop the image(s) so faces are included (user-uploaded images are often cropped in a way to cut off faces)
- Convert image to gray scale for better facial detection
Resources:
- SO Question with great answer, includes grayscale and cropping
-
Gist for more examples - Note, the
detectMultiScale
function has incorrect parameter order, fixed my code above - openCV for PHP
- openCV Questions Site
- openCV Python docs
- Thumbor: Python-based image "smart cropping" which incorporates these techniques