Detect and Blur Faces Programmatically

In this post, you will learn how modern face detection algorithms work under the hood, the privacy concerns related to the use of such technology, and finally, how to make use of the PixLab API to detect faces at first, extract their coordinates and finally apply a blur filter for each extracted face (i.e. bounding boxes). Let's dive in!

Face Detection Algorithms

Face detection has been a solved problem since the early 2000s, but it still faces a few challenges, including detecting tiny, non-frontal faces at real-time on cheap CPUs on low-end mobile/IoT devices.

The most widely used technique is a combination of Histogram of Oriented Gradients (HOG for short) and Support Vector Machine (SVM) that achieve mediocre to relatively good detection ratios given a good quality image.

PixLab, on the other side, developed a brand new architecture targeting single class object detection and suitable for face detection. This detector is based on an architecture named RealNets and uses a set of decision tress organized as a classification cascade that works at Real-time on the CPU of cheap Android devices and easily outperform the HOG/SVM combination. 

The detector is already shipped within the open source SOD Computer Vision Library (An OpenCV alternative) and has even been ported to Javascript/WebAssembly so you can perform real-time face detection on your browser. You can play with the detector on your browser as well find more information about the WebAssembly port via this blog post. Let's talk about the RealNets architecture in the next section.

The basic idea behind the RealNet face detector algorithm is as follows:

1. Scan the input image/frame with a cascade of binary classifiers at all reasonable positions and scales.
2. A region of the target image/frame is classified as an object of interest if it successfully passes all the members of the cascade. Each binary classifier consists of an ensemble of decision trees with pixel intensity comparisons as binary tests in their internal nodes. This enables the detector to process image regions at very high speed.
3. This implementation is based on the excellent paper: Object Detection with Pixel Intensity Comparisons Organized in Decision Trees and the resulting code base is integrated and freely available within the SOD SDK.

Privacy Concerns

Photo sharing on social networks can cause privacy issues. Face blurring is one strategy to increase privacy while still allowing users to share photos. Users experience ratings for face blurring were  positive, indicating  blurring  may  be  an  acceptable way to modify photos from the user perspective.  

With privacy regulations changing all the time around the globe and the recent GDPR that came into effect in the EU, it is recommended to comply with all regulations per default by adding/ integrating face blurring into your services.

The only issue with this approach is that if the viewer has some knowledge about the user, they may still identify them, even with the blur. For instance, they may use other factors such as clothes, body shape, and skin color. However, since we've just given a basic application of the API, as a developer, you can make it more complex by blurring other features that can reveal the identity of the person in the photo. In the next section, we'll make use of the Image Blur API by PixLab to apply such filter for each detected face.

The PixLab API

PixLab is an ML focused, SaaS platform offering Machine Vision & Media Processing APIs for developers via a straightforward Web or Offline SDKs. PixLab API feature set includes but not not limited to:

• Passports & ID Cards document scanning using state-of-the-art Machine Learning models via straightforward HTTP Rest API as shown in this Python & PHP gists and publicly available via the /docscan API endpoint.
• Board of face analysis API endpoints including face detection, landmarks extraction, facial recognition, content moderation and many more.
• On the fly image encryption, conversion, compression and full support for HTTP/2 and HTTP/3(QUIC) within the release of the PixLab API 1.9.72.
• Proxy for AWS S3 and other cloud storage providers.
• Over 130 Machine Vision & Media Processing API endpoints.

Python/PHP Code Samples

Given an input image with some nice faces:

Detect faces at first and apply a blur filter for each extracted region (i.e. face coordinates). The final result should look like the following:

The result above was obtained via the following Pyhton gist:

xxxxxxxxxx

import requests

import json

imgUrl = 'https://pixlab.io/images/m3.jpg' # Target picture we want to blur any face on. Note that /facedetect supports POST HTTP method so you can upload your images directly from your website or app.

# Detect all human faces in a given image via /facedetect first and blur all of them later via /mogrify.

# Call /facedetect first to extract the coordinates for each present face.

req = requests.get('https://api.pixlab.io/facedetect',params={

    'img': imgUrl,

    'key':'PIXLAB_API_KEY',

})

reply = req.json()

if reply['status'] != 200:

    print (reply['error'])

    exit();

total = len(reply['faces']) # Total detected faces

print(str(total)+" faces were detected")

if total < 1:

    # No faces were detected, exit immediately

    exit()

# Pass the detected faces coordinates untouched to mogrify 

coordinates = reply['faces']

# Call mogrify & blur the face(s)

req = requests.post('https://api.pixlab.io/mogrify',headers={'Content-Type':'application/json'},data=json.dumps({

    'img': imgUrl,

    'key':'PIXLAB_API_KEY',

    'cord': coordinates # The field of interest

}))

reply = req.json()

if reply['status'] != 200:

    print (reply['error'])

else:

    print ("Blurred Picture URL: "+ reply['ssl_link'])

Python Gist Source Code: https://github.com/symisc/pixlab/blob/master/python/blur_human_faces.py

The same logic using PHP now:

/*

 * PixLab PHP Client which is just a single class PHP file without any dependency that you can get from Github

 * https://github.com/symisc/pixlab-php 

 */

require_once "pixlab.php";

# Detect all human faces in a given image via `facedetect` and blur all of them via `mogrify`.

# Target Image we want to blur face(s) on

$img = 'https://pixlab.io/images/m3.jpg';

# Your PixLab API key

$key = 'PIXLAB_API_KEY';

$pix = new Pixlab($key);

echo "Detecting faces first...\n";

/* Invoke facedetect first  */

if( !$pix->get('facedetect',array('img' => $img)) ){

    echo $pix->get_error_message();

    die;

}

/* Grab the total number of detected faces */

$faces = $pix->json->faces;

echo "Total number of detected faces: ".count($faces)."\n";

if( count($faces) < 1 ){

    echo "No human faces were were detected on this picture\n";

}else{

    echo "Blurring faces...\n";

    /* Call mogrify (Only POST) */

    if( !$pix->post('mogrify', ['img' => $img,'cord' => $faces]) ){

        echo $pix->get_error_message();

    }else{

        echo "Blurred faces URL: ".$pix->json->link."\n";

    }

}

PHP Gist Source Code: https://github.com/symisc/pixlab/blob/master/PHP/blur_human_faces.php

Regardless of the underlying programming language, the logic is always same. We made a simple HTTP GET request with the input image URL as a sole parameter. Most PixLab endpoints support multiple HTTP methods so you can easily switch to POST based requests if you want to upload your images & videos directly from your mobile or web app for analysis. Back to our sample, only two API endpoints are needed for our face blurring task:

1. facedetect is the analysis endpoint that is called first. It outputs the rectangle coordinates for each detected human face in a given image or video frame. Each returned coordinates includes the face width, height and the X & Y position on the input image or video frame. We will use these information to apply a blur filter on each target region later. You can find out more information about the facedetect endpoint here.
2. mogrify is called later after we obtain the rectangle coordinates for all the detected faces. In which case, we simply pass these coordinates to mogrify untouched and the target regions of the image are blurred. When done, mogrify returns a direct link to the blurred picture stored in PixLab CDN or your own S3 bucket if you already connected your AWS S3 bucket from the PixLab Dashboard. Note that mogrify can return the raw blurred image directly instead of cloud storage by setting the blob parameter this endpoint takes to true. Finally, The mogrify endpoint is documented at pixlab.io/cmd?id=mogrify.

Conclusion

Surprisingly, blurring faces automatically is straightforward for the average web developer or site administrator who may lack technical machine learning skills thanks to open computer vision technologies such the one provided by PixLab.

With privacy regulations changing all the time around the globe and the recent GDPR that came into effect in the EU, it is recommended to comply with all regulations per default by adding/integrating face blurring into your services. Find out more code samples on https://github.com/symisc/pixlab and https://github.com/symisc/sod if your are C/C++ developer for an offline SDK.