Big Data Resources

Kafka Architecture

Learn about the architecture and functionality of Kafka, the software for building real-time streaming data pipelines, in this comprehensive primer.

August 9, 2017

by Jean-Paul Azar

· 134,890 Views · 89 Likes

The Role of Predictive Analytics in DevOps

Learn how data and predictive analysis can be used by DevOps engineers to further develop and optimize the DevOps workflow.

August 5, 2017

by Badri Srinivasan

CORE

· 11,197 Views · 4 Likes

Coffee With a Data Scientist: Tuhin Chattopadhyay, Ph.D.

In the fourth issue of DZone's Coffee With a Data Scientist, we had a chat with business analytics evangelist, Tuhin Chattopadhyay, to glean some of his expert insights and opinions on the Big Data space.

August 4, 2017

by Michael Tharrington

· 8,443 Views · 4 Likes

Event Driven Microservices Patterns

Read about the motivation behind the switch to microservices, and some of the patterns that make these applications more scalable.

August 4, 2017

by Carol McDonald

· 66,763 Views · 75 Likes

Using Airflow to Manage Talend ETL Jobs

Learn how to schedule and execute Talend jobs with Airflow, an open-source platform that programmatically orchestrates workflows as directed acyclic graphs of tasks.

August 3, 2017

by Rathnadevi Manivannan

· 21,586 Views · 4 Likes

Where Are the Biggest Opportunities for AI?

Here's what 22 executives who are familiar with AI said when we asked them, "What are the most common issues you see preventing companies from realizing the benefits of AI?"

July 31, 2017

by Tom Smith

CORE

· 9,990 Views · 3 Likes

Don't Use Apache Kafka Consumer Groups the Wrong Way!

Apache Kafka is great — but if you're going to use it, you have to be very careful not to break things. Here's how you can avoid the pain!

July 30, 2017

by Paolo Patierno

· 273,975 Views · 20 Likes

How I Used Deep Learning to Train a Chatbot to Talk Like Me (Sorta)

See how to use a deep learning model to train a chatbot based on past social media conversations in hopes of getting the chatbot to respond to messages the way you would.

July 27, 2017

by Adit Deshpande

· 39,775 Views · 18 Likes

Working With Centralized Logging With the Elastic Stack

See how Filebeat works with the Elastic, or ELK, stack to locate problems in distributed logs for an application with microservices architecture.

July 24, 2017

by Rafael Salerno

· 6,485 Views · 2 Likes

How to Use the Kafka Streams API

The Kafka Streams API allows you to create real-time applications that power your core business. Here's everything you need to know about it!

July 23, 2017

by Anuj Saxena

· 110,877 Views · 15 Likes

Data Flow Pipeline Using StreamSets

Learn about configuring JDBC Query Consumer, performing JDBC lookup with multiple tables, creating a data flow pipeline, and monitoring the stage and pipeline stats.

July 18, 2017

by Rathnadevi Manivannan

· 16,173 Views · 7 Likes

Is ElasticSearch SET/GET Eventually Consistent?

The author takes a deep dive into the intricacies of SET/GET in ElasticSearch to determine if there is eventual consistency.

July 18, 2017

by Tomer Ben David

· 16,260 Views · 9 Likes

Hadoop Cluster Capacity Planning of Data Nodes for Batch and In-Memory Processes

Let's talk about capacity planning for data nodes. We'll start with gathering the cluster requirements and end by learning about RAM requirements.

Updated July 14, 2017

by Mamta Chawla

· 47,756 Views · 8 Likes

An Introduction to Kafka

Learn the basics of Apache Kafka, an open-source stream processing platform, and learn how to create a general single broker cluster.

July 14, 2017

by Prashant Sharma

· 74,171 Views · 51 Likes

Filebeat vs. Logstash — The Evolution of a Log Shipper

This comparison of log shippers Filebeat and Logstash reviews their history, and when to use each one- or both together.

July 8, 2017

by Daniel Berman

· 29,649 Views · 12 Likes

The Most Important Lessons Learned From Data Science Projects

The biggest advantage of data science over traditional statistics is that it can draw conclusions from a junk pile of the supposedly unrelated information.

July 7, 2017

by Chris Richardson

· 9,401 Views · 2 Likes

Big Data Ingestion: Flume, Kafka, and NiFi

Flume, Kafka, and NiFi offer great performance, can be scaled horizontally, and have a plug-in architecture where functionality can be extended through custom components.

July 7, 2017

by Tony Siciliani

· 88,871 Views · 24 Likes