Wavelabs Face Detection

How to do Face Detection Without Using any Deep Learning

Facial Detection is a way of identifying human faces in digital images or video streams. It has got in-numerous applications in various industries like security, surveillance, health sector, social media, biometrics, law enforcement, entertainment etc. It is widely used in cameras to identify multiple appearances in the frame of Mobile cameras and DSLR’s. Facebook is also using a face detection algorithm to detect faces in the images and recognize them. There are many ways of doing it, starting from age-old image processing techniques like Haar cascade to new Deep learning techniques like R-CNNs and YOLOs. Here in this post, I am going to show some quick and computationally light ways of doing it.

Evidently, knowing how to do this can add value if you are planning a career in Computer Vision.

What can you expect from this post?
This post is primarily focused on beginners to AI space with a fair amount of knowledge in python. This can be a cool way to get your hands on computer vision, especially face detection with minimal lines of code.

What is not covered in this post?
I am not going to explain much of the theoretical concepts that go into face detection algorithms.

Prerequisites

Dlib –  Dlib is a modern C++ toolkit containing machine learning algorithms and tools for creating complex software in C++ to solve real-world problems. It’s quite a handy tool to have for a computer vision developer to quickly spin up computer vision prototypes. We will use Dlib’s facial detector functions in this post.

For windows:

pip install dlib

Installing Dlib on Mac or Linux:

Click here to learn how to install dlib from source on macOS or Ubuntu

OpenCV — OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in commercial products. OpenCV is kind of a ‘must know’ for a person stepping into computer vision. We will use OpenCV’s video capture function to use the webcam from the local machine to get the video feed for the example.

pip install opencv-python

Face_Recognition — Recognize and manipulate faces from Python or from the command line with the world’s simplest face recognition library. It is Built using dlib’s state-of-the-art face recognition with deep learning. We will use this library for one of the ways of detecting faces in our example.

pip3 install face_recognition

Let’s write up some code that is common to all the examples of detection discussed in the post.

Import the required libraries.

import dlib
import cv2
import face_recognition
import sys

Use cv’s video capture to start up the webcam.

# Start video capture for webcam -  Specifying 0 as an argument 
# fires up the webcam feed
video_capture = cv2.VideoCapture(0)

Read each frame from the video_capture object.

# Grab a single frame of video
ret, frame = video_capture.read()

Usually, we get a frame of resolution 1280*720 from any typical HD webcam which is quite heavy for face detection operations. So we resize it by 1/4th for faster processing.

# Resize frame of video to 1/4 size for faster face detection   
small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)

As we read the image using CV, it is read in a BGR channel rather than traditional RGB channel (Red, Green, Blue ). We need to do the conversion.

# Convert the image from BGR color (which OpenCV uses) to RGB color 
rgb_small_frame = small_frame[:, :, ::-1]

The next step is calling library functions to get the detection coordinates by passing the above-converted image. I am going to show 3 ways of achieving the same. You can use any one, which works better for your use-case.

Face Detection using Face_Recognition API

Using face_recognition library, call up face_locations() on the above-converted frame to get the detections.

# Calling face_locations function on the converted frame.
dets = face_recognition.face_locations(rgb_small_frame)

The dets here is a list of tuples with top, right, bottom, left coordinates of all the face locations in the frame.

Loop over the dets to get the coordinates and draw the rectangles around faces.

# Loop over the identified locations to draw a rectangle on the face
for (top, right, bottom, left) in dets:

# Loop over the identified locations to draw a rectangle on the face
    for (top, right, bottom, left) in dets:
        
# Scale back up face locations since the frame we detected in was 
# scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4
# Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

Once all the rectangles are drawn on the frame, display the frame using cv’s inbuilt imshow() function.

# Display the resulting image
cv2.imshow('Face_recognition', frame)
Put all the above code in a while True loop and add the below code for quitting condition.

# Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

Followed by

video_capture.release()
cv2.destroyAllWindows()

#!/usr/bin/env python
# coding: utf-8

import dlib
import cv2
import face_recognition
import sys

# Start video capture for webcam -  Specifying 0 as an argument fires up the webcam feed
video_capture = cv2.VideoCapture(0)

while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()
    # Resize frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]

    # Calling face_locations function on the converted frame.  
    dets = face_recognition.face_locations(rgb_small_frame)
    
    # Loop over the identified locations to draw a rectangle on the face
    for (top, right, bottom, left) in dets:
        
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top *= 4
        right *= 4
        bottom *= 4
        left *= 4

        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

    # Display the resulting image
    cv2.imshow('Face_recognition', frame)

    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
video_capture.release()
cv2.destroyAllWindows()

Face Detection using Dlib face detector (HOG based)

Using Dlib library, instantiate get_frontal_face_detector class and pass the above-converted image to get the detections. This detection method is based on the H.O.G concept. You can get a fair idea about it in my post on H.O.G. feature descriptor.

# Creating dlib frontal face detector object
detector = dlib.get_frontal_face_detector()
# Using the detecor object to get detections
dets = detector(rgb_small_frame)

The dets here is a list of objects with top, right, bottom, left functions that give the corresponding coordinates of all the face locations in the frame.

Loop over the dets to get the coordinates and draw the rectangles around faces.

# Loop over the identified locations to draw a rectangle on the face
    for det in dets:
# Scale back up face locations since the frame we detected in was 
# scaled to 1/4 size
        top = 4 * det.top()
        right = 4 * det.right()
        bottom = 4 * det.bottom()
        left = 4 * det.left()
        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

Once all the rectangles are drawn on the frame, display the frame using cv’s inbuilt imshow() function.

# Display the resulting image
cv2.imshow('Dlib_HOG', frame)
Keep all the code in a while true loop with below breaking condition

# Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
video_capture.release()
cv2.destroyAllWindows()
#!/usr/bin/env python
# coding: utf-8

import dlib
import cv2
import face_recognition
import sys

# Start video capture for webcam -  Specifying 0 as an argument fires up the webcam feed
video_capture = cv2.VideoCapture(0)

detector = dlib.get_frontal_face_detector()

while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()
    # Resize frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]
    
    dets = detector(rgb_small_frame)
    for det in dets:
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top = 4 * det.top()
        right = 4 * det.right()
        bottom = 4 * det.bottom()
        left = 4 * det.left()
        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
        
    cv2.imshow('Dlib_HOG', frame)
    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
video_capture.release()
cv2.destroyAllWindows()

Face Detection using Dlib face detector(CNN based)

Using Dlib library, instantiate the cnn_face_detection_v1 class by passing the pre-trained weights as a .dat file. You can download the file from here. This is the deep learning model version of Dlib’s detection function. It’s proven to be quite accurate but a little slow compared to the above approaches.

# Creating dlib cnn face detector object
cnn_face_detector = dlib.cnn_face_detection_model_v1('mmod_human_face_detector.dat')
# Using the detecor object to get detections
dets = cnn_face_detector(rgb_small_frame, 1)

The dets here is a list of objects that encompass rect object which contains top, right, bottom, left functions that give the corresponding coordinates of all the face locations in the frame.

Loop over the dets to get the coordinates and draw the rectangles around faces.

# Loop over the identified locations to draw a rectangle on the face
    for det in dets:
# Scale back up face locations since the frame we detected in was 
# scaled to 1/4 size
        top = 4 * det.rect.top()
        right = 4 * det.rect.right()
        bottom = 4 * det.rect.bottom()
        left = 4 * det.rect.left()
        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)

Once all the rectangles are drawn on the frame, display the frame using cv’s inbuilt imshow() function.

# Display the resulting image
cv2.imshow('Dlib_CNN', frame)
Keep all the code in a while loop with below breaking condition

# Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
video_capture.release()
cv2.destroyAllWindows()
#!/usr/bin/env python
# coding: utf-8

import dlib
import cv2
import face_recognition
import sys


# Start video capture for webcam -  Specifying 0 as an argument fires up the webcam feed
video_capture = cv2.VideoCapture(0)

# You can get the mmod_human_face_detector.dat file from http://dlib.net/files/mmod_human_face_detector.dat.bz2"
cnn_face_detector = dlib.cnn_face_detection_model_v1('mmod_human_face_detector.dat')


while True:
    # Grab a single frame of video
    ret, frame = video_capture.read()
    # Resize frame of video to 1/4 size for faster face recognition processing
    small_frame = cv2.resize(frame, (0, 0), fx=0.25, fy=0.25)
    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
    rgb_small_frame = small_frame[:, :, ::-1]
    
    dets = cnn_face_detector(rgb_small_frame, 1)
    for det in dets:
        # Scale back up face locations since the frame we detected in was scaled to 1/4 size
        top = 4 * det.rect.top()
        right = 4 * det.rect.right()
        bottom = 4 * det.rect.bottom()
        left = 4 * det.rect.left()
        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
        
    cv2.imshow('Dlib_CNN', frame)
    # Hit 'q' on the keyboard to quit!
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
video_capture.release()
cv2.destroyAllWindows()

With the facial detection coordinates at hand, the possible applications where it can be feed into becomes endless. One such application can be face recognition and there are many more. Your creativity is the only limit.

Finally, some results!

This functionality is provided by Google Firebase at a cost. With Google’s ML Kit’s face detection API, you can detect faces in an image, identify key facial features, and get the contours of detected faces. There are other players as well in the market who offer the same features but with the above method and a bit of custom coding you can get these premium features for free.