Facial Detection with openCV and Python

We can use openCV for facial detection. OpenCV is written in C, but there are bindings for Python and actually PHP.

Use Cases:

  1. Cropping user-uploaded images, without cutting out faces. Honey.is does this effectively for user profile images.

How To:

I'm going to do this using Python. We can do this by installing openCV and the Python bindings and then writing a quick script to detect faces in an image.

Note: I didn't use virtualenv because of the ease of using apt-get packages and because I have Vagrant to spin up and destroy VM's easily enough to not worry about using virtualenv.

We'll use cv2, instead of the older cv.

Server Install

On Ubuntu 12.04 LTS, I ran this:

$ sudo apt-get update
$ sudo apt-get install -y vim build-essential python-software-properties    # The Basics
$ sudo apt-get install -y python-opencv python-numpy python-scipy        # OpenCV items

Download Face Detection Trainer

You'll need an XML file with training data for the program. Download the Haar Cascade Frontal Face xml file.

$ wget http://eclecti.cc/files/2008/03/haarcascade_frontalface_alt.xml

The Script

Here is the Python used to check images for faces.

import cv2

def detect(path):
    img = cv2.imread(path)
    cascade = cv2.CascadeClassifier("/vagrant/detect/haarcascade_frontalface_alt.xml")
    rects = cascade.detectMultiScale(img, 1.3, 4, cv2.cv.CV_HAAR_SCALE_IMAGE, (20,20))

    if len(rects) == 0:
        return [], img
    rects[:, 2:] += rects[:, :2]
    return rects, img

def box(rects, img):
    for x1, y1, x2, y2 in rects:
        cv2.rectangle(img, (x1, y1), (x2, y2), (127, 255, 0), 2)
    cv2.imwrite('/vagrant/img/detected.jpg', img);

rects, img = detect("/vagrant/img/one.jpg")
box(rects, img)

The Results

The original image: faces original

With faces detected: faces detected

Note: One of the faces was not detected! If you take a look, you'll see that the person on the far right's face is covered in shadow and has less distinguishable features (no glasses).

It's certainly not perfect. Facial detection is not an easy nor exact science.

Also noteworthy is that the Haar Cascade XML file used is meant to detect "frontal faces", rather than faces in profile. There is detection available for other features such as hands. See the Gist below for some of those XML files.

To Do

  1. Crop the image(s) so faces are included (user-uploaded images are often cropped in a way to cut off faces)
  2. Convert image to gray scale for better facial detection

Resources:

  1. SO Question with great answer, includes grayscale and cropping
  2. Gist for more examples - Note, the detectMultiScale function has incorrect parameter order, fixed my code above
  3. openCV for PHP
  4. openCV Questions Site
  5. openCV Python docs
  6. Thumbor: Python-based image "smart cropping" which incorporates these techniques