Big Data Resources

Evaluating Message Brokers: Kafka vs. Kinesis vs. SQS

A comparison of the best message brokers for big data applications between SQS, Kinesis, and Kafka.

May 14, 2016

by Swaroop Ramachandra

· 68,404 Views · 20 Likes

IoT Arduino Programming: Monitor Enviroment

In this IoT Arduino programming tutorial, the Arduino board is connected to a set of sensors and an ethernet shield. Sensors used in this IoT Arduino programming project are:

May 9, 2016

by Francesco Azzola

· 13,544 Views · 5 Likes

IOT Standards and Remote Services

A thorough and well though out discussion of a simple yet important question; how can we embrace open standards in IoT?

May 4, 2016

by Scott Lewis

· 2,913 Views · 1 Like

Working With AVRO and Parquet Files

Installing and working with tools for AVRO and Parquet files with Scala and Spark

April 27, 2016

by Tim Spann

CORE

· 18,714 Views · 5 Likes

Hadoop in Healthcare, Part 1

How Hadoop and open source big data analytics technology are positively helping the healthcare industry become more effective.

April 26, 2016

by Richard Proctor

· 10,776 Views · 4 Likes

Looking Forward to PGConf US 2016

In this piece, we interview Joshua Drake and Amanda Gilmore, speakers at PGConf US 2016, for a bit of background info about each of their personal experiences with Postgres and catch a quick glimpse of what they'll be covering in their upcoming talks.

April 14, 2016

by Michael Tharrington

· 3,311 Views · 2 Likes

Analyzing the Panama Papers With Neo4j: Data Models, Queries, and More

In this post, we look at the graph data model used by the International Consortium of Investigative Journalists (ICIJ) and show how to construct it using Cypher in Neo4j. We dissect an example from the leaked data, recreating it using Cypher, and show how the model could be extended.

April 13, 2016

by William Lyon

· 10,493 Views · 3 Likes

Getting Started With Spark Streaming

An introduction to Spark Streaming and how to use it with an example data set.

April 10, 2016

by Carol McDonald

· 83,941 Views · 13 Likes

A Data Lake Architecture With Hadoop and Open Source Search Engines

"Big data" and "data lake" only have meaning to an organization’s vision when they solve business problems by enabling data democratization, re-use, exploration, and analytics. Read on to learn what a data lake is, its various benefits, and what's to come.

April 4, 2016

by Carlos Maroto

· 35,376 Views · 6 Likes

Spark Salesforce Connector Tutorial Using JDBC

We have a quick look at leveraging Apache Spark to access data in Salesforce via its JDBC connector. Come see how it's done!

April 1, 2016

by Nishanth Kadiyala

· 10,290 Views · 4 Likes

Word Count Programs: Spark vs. MapReduce

Why use Spark for big data processing? Compare these two word count program examples.

March 23, 2016

by Shiv Shet

· 25,813 Views · 6 Likes

A Beginner's Guide to Apache Kafka

A bare bones, bare necessities guide to what Apache Kafka can do and why it is popular.

March 23, 2016

by Shiv Shet

· 10,443 Views · 7 Likes

Comparison of Asynchronous Messaging Technologies: JMS, AMQP, and MQTT

A comparison of three different IoT messaging protocols by DZone user Chanaka Fernando.

March 17, 2016

by Chanaka Fernando

CORE

· 49,721 Views · 17 Likes

Apache Kafka Performance Tuning

Apache Kafka is a great solution for handling real-time data feeds. Learn more about tuning Kafka to meet your high performance needs in this great video.

March 17, 2016

by Rohit Dhall

· 14,267 Views · 16 Likes

The Future of Data and Databases

Consolidation around big data with rationalization down to 10 core technologies.

March 14, 2016

by Tom Smith

CORE

· 13,812 Views · 3 Likes

Build an Online Store With MongoDB, Node.js, and Elasticsearch

Check out the following infographics and brief break-down to help you visualize and understand the decisions involved in choosing a microservice architecture.

March 4, 2016

by Siva Prasad Rao Janapati

· 22,396 Views · 15 Likes

Word Count Program With MapReduce and Java

In this post, we provide an introduction to the basics of MapReduce, along with a tutorial to create a word count app using Hadoop and Java.

March 3, 2016

by Shital Kat

· 443,591 Views · 31 Likes

Linking Apache Ignite and Apache Kafka for Highly Scalable and Reliable Data Processing

Here's how to link Apache Kafka and Ignite, for maintaining scalability and reliability for data processing. We'll explore injecting data with KafkaStreamer, as well as IgniteSinkConnector.

March 3, 2016

by Roman Shtykh

· 23,250 Views · 13 Likes

Real-time Data Pipelines with Kafka Connect

Information about Kafka Connect sourced from Spark Summit East 2016.

February 19, 2016

by Tim Spann

CORE

· 16,002 Views · 5 Likes

Loading Data Into Azure SQL Data Warehouse

I’m not an ETL expert. In fact, I haven’t done any professional ETL work for several years. My skills are, at best, rusty. With this in mind, I knew I’d have a hard time extracting data from a local database in order to move it up to Azure SQL Data Warehouse. Read on to hear about my journey.

February 18, 2016

by Grant Fritchey

· 13,954 Views · 6 Likes

The Latest Big Data Topics