Developing TPU Based AI Solutions Using TensorFlow Lite

AI has become ubiquitous today, from personal devices to enterprise applications; you see them everywhere. The advent of IoT, clubbed with rising demand for data privacy, low power, low latency, and bandwidth constraints has increasingly pushed for AI models to be running at the edge instead of the cloud. According to Grand View Research, the global edge artificial intelligence chips market was valued at USD 1.8 billion in 2019 and is expected to grow at a CAGR of 21.3 percent from 2020 to 2027. At this onset, Google introduced Edge TPU, also known as Coral TPU, its purpose-built ASIC for running AI at the edge. 

It's designed to give excellent performance while taking up minimal space and power. When we train an AI model, we have AI models with high storage requirements and GPU processing power. We cannot execute them on devices with low memory and processing footprints. TensorFlow Lite is useful in this situation. TensorFlow Lite is an open-source deep learning framework that runs on the Edge TPU and allows for on-device inference and AI model execution. Also, note that TensorFlow Lite is only for executing inference on edge, not training a model. For training an AI model, we must use TensorFlow.  

Combining Edge TPU and TensorFlow Lite 

When we talk about deploying an AI model on Edge TPU, we cannot deploy any AI model. 

The Edge TPU supports NN (Neural Network) operations, and designs enable high-speed neural network performance with low power consumption. Apart from specific networks, it only supports 8-bit quantized and compiled TensorFlow Lite models for Edge TPU. 

For a quick summary, TensorFlow Lite is a lightweight version of TensorFlow specially designed for mobile and embedded devices. It achieves low latency results with a small storage size. A TensorFlow Lite converter allows converting a TensorFlow-based AI model file (. pb) to a TensorFlow Lite file (.tflite).  Below is a standard workflow for deploying applications on Edge TPU

 Application deployment on Edge TPU

Let’s look at some interesting real-world applications that can be built using TensorFlow Lite on edge TPU. 

Human Detection and Counting

This solution has many practical applications, especially in malls, retail, government offices, banks, and enterprises. One may wonder what one can do with detecting and counting humans. Data now has the value of time and money. Let us see how the insights from human detection and counting can be used. 

1. Estimating Footfalls: For the retail industry, this is important as it gives an idea if their stores are doing well. Whether their displays are attracting customers to enter the shops, it also helps them to know if they need to increase or decrease support staff. For other organizations, they help in taking adequate security measures for people. 
2. Crowd Analytics and Queue Management: For govt offices and enterprises, queue management via human detection and counting helps them manage longer queues and save people’s time. Studying queues can contribute to individual and organizations' performance. Crowd detection can help analyze crowd alerts for emergencies, security incidents, etc., and take appropriate actions. Such solutions give the best results when deployed on edge, as required actions can be taken close to real-time. 

Age and Gender-based Targeted Advertisements. 

This solution mainly has practical applications in the retail and advertisement industry. Imagine you walking towards the advertisement display, which was showing a women’s shoe ad. Suddenly, the advertisement changes to a male’s shoe ad as it determined you being male. Targeted advertisements help retailers and manufacturers target their products better and create brand awareness that a normal person would never get to see in his busy life.  

This cannot be restricted to only advertisements; age and gender detection can also help businesses in taking quick decisions by managing appropriate support staff in retail stores, what age and gender people prefer visiting your store, businesses, etc. All this is more powerful and effective if you are very quick to determine and act. So, even more, a reason to have this solution on Edge TPU. 

Face Recognition

The very first face recognition system was built in 1970, and to date, this is still being developed, being made more robust and effective. The main advantage of having face recognition on edge is real-time recognition. Another advantage is having face encryption and feature extraction on edge and sending encrypted and extracted data to the cloud for matching, thereby protecting PII level privacy of face images (as you don’t save face images on edge and cloud) and complying with stringent stringent requirements
privacy laws.

Edge TPU combined with the TensorFlow Lite framework opens several edges AI applications opportunities. As the framework is open-source, the Open-Source Software (OSS) community also supports it, making it even more popular for machine learning use cases. The overall platform of TensorFlow Lite enhances the environment for the growth of edge applications for embedded and IoT devices.