TensorFlow on the Edge, Part 1 of 5

Deep Learning is growing in power, scale, availability and frameworks. Open Source tools for Neural networks are everywhere, you have so much choice. Interesting new developments like PaddlePaddle, Keras and Deep Water are showing up and updating frequently. Unfortunately a lot of them take some serious power in GPUs, number of nodes, RAM, disk space, network bandwidth and CPU cores like IM2TXT for TensorFlow. These are very good uses of your 100+ node Hadoop clusters especially ones running on AWS with GPUs. Or if you can get a few of the new NVIDIA Tesla K20s. Sometimes you want to, only have access to or are just learning like I am. So you can start with a $35, swiss army knife of computers, the RPI!

With the power of Raspberry PI 3 Bs you can now run processing at the edge including image processing, text processing, JSON munging, and even deep learning. The Raspberry Pi has a powerful 64bit QuadCore processor with 1 Gigabyte of RAM, that's pretty powerful for running some Python scripts and sending a stream of data to an regional server for additional processing.

Obviously your main Hadoop clusters will have beefy servers with 256G of RAM or more, 32 or more CPU Cores and other impressive specifications for processing petabytes of data. For real-time streaming it's great to push some of the initial processing to the IoT edge nodes that are doing the initial ingest.

A good example of processing at the edge is in my article here.

What are some good uses for these devices?

IoT, Streaming, ML, DL, Big Data Usage Patterns

1.  Ingest data from sensors attached to device (Sense-Hat on RPi), read in Python, send via MQTT to Apache NiFi.

2.  Ingest data from attached Web Cam (PiCamera on RPi), process in Python, run image recognition with TensorFlow and send via sFTP to Apache NiFi.

3.  Ingest data from sensors, logs and devices, quick process with OpenCV, ImageMagick and other tools and send via MiNiFi to Apache NiFi.

4.  Ingest social media streams like Twitter via Python libraries and send them on.   This is easy to do in twitter, but it doesn't hurt to spread out the load.   If you have many regional work sites and they have their own Twitter or Facebook or other social accounts, you can use those to grab more data.

5.  Ingest data from SCADA and Smart Meters that you can connect to from a local Pi either through TCP/IP, wired connection, ZigBee, BLE or any number of protocols.  For security, heat or other reasons there may be no other way to interact with these sensors, devices or machines.

What's amazing is you can now do the initial processing including some Deep Learning on the device before the data is sent on to the next step.   You can run more powerful transformations, machine learning, NLP, Sentiment Analysis, more deep learning on the cloud or regional NiFi server and then send it to a centrally hosted data center or cloud based Hadoop or push to Kafka for further processing Spark, Storm or Flink (or Apache Beam to isolate your choice).

Google has announced more growing support for Raspberry Pi, and they already have example TensorFlow's that work on Raspberry Pi 3 B.

For an example of how fast this is, check out these timings:

root@raspberrypi:/opt/demo/tensorflow-master# tensorflow/contrib/pi_examples/label_image/gen/bin/label_image
2017-01-28 01:46:48: I tensorflow/contrib/pi_examples/label_image/label_image.cc:144] Loaded JPEG: 512x600x3
2017-01-28 01:46:50: W tensorflow/core/framework/op_def_util.cc:332] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization().
2017-01-28 01:46:52: I tensorflow/contrib/pi_examples/label_image/label_image.cc:378] Running model succeeded!
2017-01-28 01:46:52: I tensorflow/contrib/pi_examples/label_image/label_image.cc:272] military uniform (866): 0.624294
2017-01-28 01:46:52: I tensorflow/contrib/pi_examples/label_image/label_image.cc:272] suit (794): 0.0473981
2017-01-28 01:46:52: I tensorflow/contrib/pi_examples/label_image/label_image.cc:272] academic gown (896): 0.0280925
2017-01-28 01:46:52: I tensorflow/contrib/pi_examples/label_image/label_image.cc:272] bolo tie (940): 0.0156955
2017-01-28 01:46:52: I tensorflow/contrib/pi_examples/label_image/label_image.cc:272] bearskin (849): 0.0143348

Four seconds is pretty good, with the fast networking on RPI and only using it for capturing an image, process it and send it, you can push close to one image every 5 seconds with a decent SD card and not much else running.

For a future test I am looking at running on Onion, Pine64, Dell Inspiron Cube, Intel Edison, Arduino and some other small devices all of which have their own unique skills and features. I am also looking at running DL4J and PI4J for JVM Deep Learning.

References

• IoT Capturing Photos and Analyzing the Images with TensorFlow on RPI

• TensorFlow Image Recognition Tutorial 

• GoogleNet Inception v3 

• Tesla K20