Spring Boot 2.0 on ACID! Big Data + Spring Boot

When I first joined Hortonworks, I wrote an article on integrating Apache Hive and Spring Boot — I came from Pivotal and was a huge fan of Spring Boot. It's been almost 1.5 years and Spring Boot is now in 2.0 and Apache Hive is now in LLAP with ACID transactional tables. So it's time for a remix!

The Use Case for this microservice is to query the Hive ACID table:   inception.   This table is built by Apache NiFi and populated with the results of an MXNet Inception analysis of images.    You pass in a query to the microservice and it runs a search against the text descriptions of the image.   It searches the top1, top2, top3, top4 and top5 fields.  These fields contain a few words each describing what Inception thought the image was.   This is a good way to browse the data for topics.   This is also really simple in Spring Boot.   I could have added the Spring Data JPA project to make it easier, we will probably do that in the next article.   We return the data as JSON even though the table is a regular Hive ACID table stored as Apache ORC files.   This JSON is easy to work with and easy to integrate in Apache NiFi.

It is crazy easy to start your project — just head to https://start.spring.io/, build a bomb POM, and let's begin.

You will need to modify the built POM.XML (Maven) to include some Hadoop and Hive references. No big deal.

Clone it, then own it:

git clone https://github.com/tspannhw/hivereader.git

You can build the code via build.sh.

mvn install

Then run it via run.sh (change HDP version to match yours).

java -Xms512m -Xmx2048m -Dhdp.version=2.6.4 -Djava.net.preferIPv4Stack=true -jar target/hivereader-0.0.1-SNAPSHOT.jar

You must have the Java JDK, Git, and Maven installed on your machine. If you don't, why are you running Java programs? Obviously, this can be inside a Docker container or virtual environment.

Maven build script:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.dataflowdeveloper</groupId>
    <artifactId>hivereader</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>
    <name>hivereader</name>
    <description>Spring Boot 2.0 Apache Hive ACID Reader</description>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.0.0.RELEASE</version>
        <relativePath />
        <!-- lookup parent from repository -->
    </parent>
    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-rest</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.restdocs</groupId>
            <artifactId>spring-restdocs-mockmvc</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-jdbc</artifactId>
            <version>1.2.1</version>
            <exclusions>
                <exclusion>
                    <groupId>org.eclipse.jetty.aggregate</groupId>
                    <artifactId>*</artifactId>
                </exclusion>
                <exclusion>
                    <artifactId>slf4j-log4j12</artifactId>
                    <groupId>org.slf4j</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>log4j</artifactId>
                    <groupId>log4j</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>servlet-api</artifactId>
                    <groupId>javax.servlet</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>2.7.3</version>
            <type>jar</type>
            <exclusions>
                <exclusion>
                    <artifactId>slf4j-log4j12</artifactId>
                    <groupId>org.slf4j</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>log4j</artifactId>
                    <groupId>log4j</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>servlet-api</artifactId>
                    <groupId>javax.servlet</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.7.3</version>
            <exclusions>
                <exclusion>
                    <artifactId>servlet-api</artifactId>
                    <groupId>javax.servlet</groupId>
                </exclusion>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-exec</artifactId>
            <version>1.2.1</version>
            <type>jar</type>
            <exclusions>
                <exclusion>
                    <artifactId>servlet-api</artifactId>
                    <groupId>javax.servlet</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>slf4j-log4j12</artifactId>
                    <groupId>org.slf4j</groupId>
                </exclusion>
                <exclusion>
                    <artifactId>log4j</artifactId>
                    <groupId>log4j</groupId>
                </exclusion>
            </exclusions>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

You can update java.version to 1.9 or 1.10 if you are running those. I have it set to hive-jdbc 1.2.1, Spring Boot 2.0.0.RELEASE, and hadoop-client 2.7.3. When you have newer versions, update these and rebuild.

Running the Spring Boot microservices JAR:

Example JSON returned by REST call:

Let's call the Spring Boot 2.0 REST API:

Let's build a schema for this data:

Let's examine the data in Apache Zeppelin:

Now that we have the data, let's query it with Apache Calcite and do some more processing... Add your own next step:

The main Apache NiFi flow to ingest the Spring Boot REST API:

Let's use the QueryRecord Processor to query our records and only look at ones with a good Top 1 Percent.

Let's look at some data provenance:

Let's create a table on ACID!

application.properties:

If you have kerberos, there's some funky things to do. I leave this to the experts. Change the Hive URI to match your server, port, and database. The server.port is the port you want the Spring Boot microservice to run on.

Obviously, you can run this through Apache YARN, Dockerized containers, or Kubernetes. In the next sequel, I will show this running Dockerized on Apache Hadoop 3.0 on YARN.

Inception bean:

package com.dataflowdeveloper.hivereader;
/** inception **/
public class Inception {
    private String top1pct;
    private String top1;
    private String top2pct;
    private String top2;
    private String top3pct;
    private String top3;
    private String top4pct;
    private String top4;
    private String top5pct;
    private String top5;
    private String imagefilename;
    public String getTop1pct() {
        return top1pct;
    }
    public void setTop1pct(String top1pct) {
        this.top1pct = top1pct;
    }
    public String getTop1() {
        return top1;
    }
    public void setTop1(String top1) {
        this.top1 = top1;
    }
    public String getTop2pct() {
        return top2pct;
    }
    public void setTop2pct(String top2pct) {
        this.top2pct = top2pct;
    }
    public String getTop2() {
        return top2;
    }
    public void setTop2(String top2) {
        this.top2 = top2;
    }
    public String getTop3pct() {
        return top3pct;
    }
    public void setTop3pct(String top3pct) {
        this.top3pct = top3pct;
    }
    public String getTop3() {
        return top3;
    }
    public void setTop3(String top3) {
        this.top3 = top3;
    }
    public String getTop4pct() {
        return top4pct;
    }
    public void setTop4pct(String top4pct) {
        this.top4pct = top4pct;
    }
    public String getTop4() {
        return top4;
    }
    public void setTop4(String top4) {
        this.top4 = top4;
    }
    public String getTop5pct() {
        return top5pct;
    }
    public void setTop5pct(String top5pct) {
        this.top5pct = top5pct;
    }
    public String getTop5() {
        return top5;
    }
    public void setTop5(String top5) {
        this.top5 = top5;
    }
    public String getImagefilename() {
        return imagefilename;
    }
    public void setImagefilename(String imagefilename) {
        this.imagefilename = imagefilename;
    }
}

DataController:

package com.dataflowdeveloper.hivereader;

import java.util.List;
import javax.servlet.http.HttpServletRequest;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.util.Assert;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.context.request.RequestAttributes;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
/** *  * @author tspann * */
@RestControllerpublic class DataController {
    public static HttpServletRequest getCurrentRequest() {
        RequestAttributes requestAttributes = RequestContextHolder.getRequestAttributes();
        Assert.state(requestAttributes != null, "Could not find current request via RequestContextHolder");
        Assert.isInstanceOf(ServletRequestAttributes.class, requestAttributes);
        HttpServletRequest servletRequest = ((ServletRequestAttributes) requestAttributes).getRequest();
        Assert.state(servletRequest != null, "Could not find current HttpServletRequest");
        return servletRequest;
    }
    Logger logger = LoggerFactory.getLogger(DataController.class);
    @Autowiredprivate DataSourceService dataSourceService;
    @RequestMapping("/query/{query}") public List < Inception > query(@PathVariable(value = "query") String query) {
        List <Inception> value = dataSourceService.search(query);
        final String userIpAddress = getCurrentRequest().getRemoteAddr();
        final String userAgent = getCurrentRequest().getHeader("user-agent");
        final String userDisplay = String.format("Query:%s,IP:%s Browser:%s", query, userIpAddress, userAgent);
        logger.error(userDisplay);
        return value;
    }
}

HivereaderApplication:

We need a class to set up the Hive datasource and initialize the Spring Application. This is the only boilerplate and it's pretty minimal. Great job Spring people!

DataSourceService:

Get the datasource, the limit to the query, and then our one search method. It takes a query parameter received from the REST API and sets that in a few places to query all of our text results, then adds the limit. We add that to our handy object and put it in a list. There are tons of helpers and shortcuts to make this easier, but this works for me. Close everything up and return this list. Spring Boot returns this as a JSON array of clean data. This is very easy to work with in Apache NiFi. This makes for a nice combination.

Running the Raw REST API

http://localhost:9999/time/now

{
    "time":"1520548296121"
}

http://localhost:9999/query/ice

[
    {
        "top1pct":"16.5000006557",
        "top1":"n04270147 spatula",
        "top2pct":"12.3999997973",
        "top2":"n03494278 harmonica, mouth organ, harp, mouth harp",
        "top3pct":"3.79999987781",
        "top3":"n07615774 ice lolly, lolly, lollipop, popsicle",
        "top4pct":"2.89999991655",
        "top4":"n04370456 sweatshirt",
        "top5pct":"2.60000005364",
        "top5":"n03891332 parking meter",
        "imagefilename":"images/tx1_image_img_20180308170314.jpg"
    },
    {
        "top1pct":"8.50000008941",
        "top1":"n04270147 spatula",
        "top2pct":"6.80000036955",
        "top2":"n02823428 beer bottle",
        "top3pct":"4.50000017881",
        "top3":"n03902125 pay-phone, pay-station",
        "top4pct":"3.90000008047",
        "top4":"n07614500 ice cream, icecream",
        "top5pct":"3.59999984503",
        "top5":"n03657121 lens cap, lens cover",
        "imagefilename":"images/tx1_image_img_20180308170832.jpg"
    },
    {
        "top1pct":"11.2000003457",
        "top1":"n04270147 spatula",
        "top2pct":"5.20000010729",
        "top2":"n03814639 neck brace",
        "top3pct":"4.69999983907",
        "top3":"n07615774 ice lolly, lolly, lollipop, popsicle",
        "top4pct":"4.10000011325",
        "top4":"n03494278 harmonica, mouth organ, harp, mouth harp",
        "top5pct":"3.79999987781",
        "top5":"n02823428 beer bottle",
        "imagefilename":"images/tx1_image_img_20180308165958.jpg"
    },
    {
        "top1pct":"8.90000015497",
        "top1":"n02823428 beer bottle",
        "top2pct":"5.70000000298",
        "top2":"n03902125 pay-phone, pay-station",
        "top3pct":"5.20000010729",
        "top3":"n04525305 vending machine",
        "top4pct":"4.30000014603",
        "top4":"n04200800 shoe shop, shoe-shop, shoe store",
        "top5pct":"4.30000014603",
        "top5":"n07614500 ice cream, icecream",
        "imagefilename":"images/tx1_image_img_20180308171043.jpg"
    },
    {
        "top1pct":"14.8000001907",
        "top1":"n04270147 spatula",
        "top2pct":"7.50000029802",
        "top2":"n04579432 whistle",
        "top3pct":"4.80000004172",
        "top3":"n07614500 ice cream, icecream",
        "top4pct":"4.60000000894",
        "top4":"n02883205 bow tie, bow-tie, bowtie",
        "top5pct":"4.50000017881",
        "top5":"n03494278 harmonica, mouth organ, harp, mouth harp",
        "imagefilename":"images/tx1_image_img_20180308173619.jpg"
    },
    {
        "top1pct":"16.5999993682",
        "top1":"n04111531 rotisserie",
        "top2pct":"8.69999974966",
        "top2":"n04270147 spatula",
        "top3pct":"3.59999984503",
        "top3":"n07614500 ice cream, icecream",
        "top4pct":"3.5000000149",
        "top4":"n03666591 lighter, light, igniter, ignitor",
        "top5pct":"2.70000007004",
        "top5":"n03902125 pay-phone, pay-station",
        "imagefilename":"images/tx1_image_img_20180308152145.jpg"
    },
    {
        "top1pct":"24.0999996662",
        "top1":"n02667093 abaya",
        "top2pct":"4.50000017881",
        "top2":"n07614500 ice cream, icecream",
        "top3pct":"3.29999998212",
        "top3":"n02823750 beer glass",
        "top4pct":"2.89999991655",
        "top4":"n02883205 bow tie, bow-tie, bowtie",
        "top5pct":"2.40000002086",
        "top5":"n04584207 wig",
        "imagefilename":"images/tx1_image_img_20180307214015.jpg"
    },
    {
        "top1pct":"6.40000030398",
        "top1":"n04370456 sweatshirt",
        "top2pct":"5.4999999702",
        "top2":"n07614500 ice cream, icecream",
        "top3pct":"5.40000014007",
        "top3":"n04229816 ski mask",
        "top4pct":"4.69999983907",
        "top4":"n03724870 mask",
        "top5pct":"4.39999997616",
        "top5":"n04270147 spatula",
        "imagefilename":"images/tx1_image_img_20180308151828.jpg"
    },
    {
        "top1pct":"8.60000029206",
        "top1":"n04579432 whistle",
        "top2pct":"6.19999989867",
        "top2":"n04270147 spatula",
        "top3pct":"5.99999986589",
        "top3":"n07614500 ice cream, icecream",
        "top4pct":"5.99999986589",
        "top4":"n03494278 harmonica, mouth organ, harp, mouth harp",
        "top5pct":"5.90000003576",
        "top5":"n07880968 burrito",
        "imagefilename":"images/tx1_image_img_20180308151620.jpg"
    },
    {
        "top1pct":"22.6999998093",
        "top1":"n02667093 abaya",
        "top2pct":"5.29999993742",
        "top2":"n03347037 fire screen, fireguard",
        "top3pct":"3.20000015199",
        "top3":"n04070727 refrigerator, icebox",
        "top4pct":"3.20000015199",
        "top4":"n04179913 sewing machine",
        "top5pct":"3.09999994934",
        "top5":"n04532106 vestment",
        "imagefilename":"images/tx1_image_img_20180307214952.jpg"
    },
    {
        "top1pct":"7.99999982119",
        "top1":"n02667093 abaya",
        "top2pct":"5.60000017285",
        "top2":"n02823428 beer bottle",
        "top3pct":"3.59999984503",
        "top3":"n07614500 ice cream, icecream",
        "top4pct":"3.09999994934",
        "top4":"n02883205 bow tie, bow-tie, bowtie",
        "top5pct":"3.09999994934",
        "top5":"n04584207 wig",
        "imagefilename":"images/tx1_image_img_20180307213659.jpg"
    },
    {
        "top1pct":"83.0999970436",
        "top1":"n02667093 abaya",
        "top2pct":"2.80000008643",
        "top2":"n03045698 cloak",
        "top3pct":"1.30000002682",
        "top3":"n02977058 cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
        "top4pct":"1.20000001043",
        "top4":"n04243546 slot, one-armed bandit",
        "top5pct":"0.600000005215",
        "top5":"n04070727 refrigerator, icebox",
        "imagefilename":"images/tx1_image_img_20180308141123.jpg"
    },
    {
        "top1pct":"9.00000035763",
        "top1":"n07614500 ice cream, icecream",
        "top2pct":"7.69999995828",
        "top2":"n04376876 syringe",
        "top3pct":"5.79999983311",
        "top3":"n04270147 spatula",
        "top4pct":"3.20000015199",
        "top4":"n07880968 burrito",
        "top5pct":"2.89999991655",
        "top5":"n07695742 pretzel",
        "imagefilename":"images/tx1_image_img_20180307213805.jpg"
    },
    {
        "top1pct":"25.9000003338",
        "top1":"n02667093 abaya",
        "top2pct":"4.80000004172",
        "top2":"n02883205 bow tie, bow-tie, bowtie",
        "top3pct":"3.90000008047",
        "top3":"n02786058 Band Aid",
        "top4pct":"3.20000015199",
        "top4":"n07614500 ice cream, icecream",
        "top5pct":"2.60000005364",
        "top5":"n04229816 ski mask",
        "imagefilename":"images/tx1_image_img_20180307213911.jpg"
    },
    {
        "top1pct":"9.20000001788",
        "top1":"n02786058 Band Aid",
        "top2pct":"7.59999975562",
        "top2":"n04229816 ski mask",
        "top3pct":"5.79999983311",
        "top3":"n02883205 bow tie, bow-tie, bowtie",
        "top4pct":"5.70000000298",
        "top4":"n04370456 sweatshirt",
        "top5pct":"4.69999983907",
        "top5":"n07614500 ice cream, icecream",
        "imagefilename":"images/tx1_image_img_20180308163321.jpg"
    },
    {
        "top1pct":"9.7000002861",
        "top1":"n04270147 spatula",
        "top2pct":"5.40000014007",
        "top2":"n02786058 Band Aid",
        "top3pct":"4.80000004172",
        "top3":"n04356056 sunglasses, dark glasses, shades",
        "top4pct":"4.39999997616",
        "top4":"n03250847 drumstick",
        "top5pct":"3.79999987781",
        "top5":"n07615774 ice lolly, lolly, lollipop, popsicle",
        "imagefilename":"images/tx1_image_img_20180308161932.jpg"
    },
    {
        "top1pct":"5.60000017285",
        "top1":"n07614500 ice cream, icecream",
        "top2pct":"5.00000007451",
        "top2":"n02786058 Band Aid",
        "top3pct":"3.5000000149",
        "top3":"n04270147 spatula",
        "top4pct":"3.40000018477",
        "top4":"n01984695 spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish",
        "top5pct":"3.20000015199",
        "top5":"n02992529 cellular telephone, cellular phone, cellphone, cell, mobile phone",
        "imagefilename":"images/tx1_image_img_20180308142106.jpg"
    },
    {
        "top1pct":"6.10000006855",
        "top1":"n04270147 spatula",
        "top2pct":"5.90000003576",
        "top2":"n03250847 drumstick",
        "top3pct":"3.40000018477",
        "top3":"n02883205 bow tie, bow-tie, bowtie",
        "top4pct":"3.29999998212",
        "top4":"n03249569 drum, membranophone, tympan",
        "top5pct":"3.09999994934",
        "top5":"n07614500 ice cream, icecream",
        "imagefilename":"images/tx1_image_img_20180308162554.jpg"
    },
    {
        "top1pct":"13.0999997258",
        "top1":"n02787622 banjo",
        "top2pct":"8.60000029206",
        "top2":"n03447721 gong, tam-tam",
        "top3pct":"5.79999983311",
        "top3":"n03250847 drumstick",
        "top4pct":"3.70000004768",
        "top4":"n07614500 ice cream, icecream",
        "top5pct":"3.59999984503",
        "top5":"n07695742 pretzel",
        "imagefilename":"images/tx1_image_img_20180308141540.jpg"
    },
    {
        "top1pct":"24.699999392",
        "top1":"n04270147 spatula",
        "top2pct":"9.39999967813",
        "top2":"n07615774 ice lolly, lolly, lollipop, popsicle",
        "top3pct":"5.60000017285",
        "top3":"n03724870 mask",
        "top4pct":"2.70000007004",
        "top4":"n03494278 harmonica, mouth organ, harp, mouth harp",
        "top5pct":"2.40000002086",
        "top5":"n04229816 ski mask",
        "imagefilename":"images/tx1_image_img_20180308145223.jpg"
    },
    {
        "top1pct":"15.5000001192",
        "top1":"n02667093 abaya",
        "top2pct":"5.40000014007",
        "top2":"n01985128 crayfish, crawfish, crawdad, crawdaddy",
        "top3pct":"4.60000000894",
        "top3":"n07614500 ice cream, icecream",
        "top4pct":"4.50000017881",
        "top4":"n03980874 poncho",
        "top5pct":"3.79999987781",
        "top5":"n03720891 maraca",
        "imagefilename":"images/tx1_image_img_20180307213449.jpg"
    },
    {
        "top1pct":"31.7999988794",
        "top1":"n02667093 abaya",
        "top2pct":"3.29999998212",
        "top2":"n03045698 cloak",
        "top3pct":"2.89999991655",
        "top3":"n04070727 refrigerator, icebox",
        "top4pct":"2.80000008643",
        "top4":"n02977058 cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
        "top5pct":"2.80000008643",
        "top5":"n04591713 wine bottle",
        "imagefilename":"images/tx1_image_img_20180307214328.jpg"
    },
    {
        "top1pct":"24.1999998689",
        "top1":"n02667093 abaya",
        "top2pct":"3.40000018477",
        "top2":"n03045698 cloak",
        "top3pct":"3.20000015199",
        "top3":"n02977058 cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
        "top4pct":"2.89999991655",
        "top4":"n04070727 refrigerator, icebox",
        "top5pct":"2.50000003725",
        "top5":"n04179913 sewing machine",
        "imagefilename":"images/tx1_image_img_20180307214848.jpg"
    },
    {
        "top1pct":"23.1000006199",
        "top1":"n03721384 marimba, xylophone",
        "top2pct":"5.40000014007",
        "top2":"n04370456 sweatshirt",
        "top3pct":"4.89999987185",
        "top3":"n01704323 triceratops",
        "top4pct":"3.29999998212",
        "top4":"n03980874 poncho",
        "top5pct":"2.80000008643",
        "top5":"n03944341 pinwheel",
        "imagefilename":"images/tx1_image_img_20180308152459.jpg"
    }
]

Pro tip: When doing PutHiveStreaming caused by:

org.apache.hadoop.ipc.RemoteException: Permission denied: user=nifi, access=WRITE, inode="/apps/hive/warehouse/inception":admin:hdfs:drwxr-xr-x at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:399) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:255) at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:193)

hdfs dfs -chmod -R 777 /apps/hive/warehouse/inception

Make sure the nifi user has write permissions to the file.

Github source:

https://github.com/tspannhw/hivereader

Release:

https://github.com/tspannhw/hivereader/releases/tag/1.0

If you don't want to install Maven and the JDK, you can download the run the JAR file with just the Java JVM runtime (and a copy of the application.properties pointing to your stuff).