Big Data Resources

Getting Started With Redpanda in Kubernetes

Learn how Redpanda is deployed in K8s with components like the reimplementation of Kafka broker, StatefulSets, nodeport, persistent storage, and observability.

May 1, 2023

by Christina Lin

CORE

· 5,355 Views · 4 Likes

Java Consumer and Producer Messages Between Kafka Server [Video Tutorials]

This video tutorial series explores the procedure of sending and receiving messages between a Java consumer and producer and the Apache Kafka server.

April 28, 2023

by Ram N

· 7,870 Views · 2 Likes

ChatGPT as a Revolution

In this blog, the reader will learn how ChatGPT came into existence, its history, the working of GPT, and be presented with use cases.

April 28, 2023

by Shashank Gupta

· 6,583 Views · 3 Likes

How ChatGPT Writes Code for Automation Tool Cypress

ChatGPT may be used to write code in a variety of programming languages and technologies. After more investigation, I made the decision to create some scenarios using it.

April 28, 2023

by Kailash Pathak

CORE

· 8,965 Views · 2 Likes

User Data Governance and Processing Using Serverless Streaming

This article delves into the concept of User Data Governance and its implementation using serverless streaming in the digital age.

April 27, 2023

by Maharshi Jha

· 6,016 Views · 4 Likes

7 Essential Data Quality Tests

Getting started with data quality testing? Here are the 7 must-have checks to improve data quality and ensure reliability for your most critical assets.

April 27, 2023

by Lior Gavish

· 3,883 Views · 1 Like

Delta, Hudi, and Iceberg: The Data Lakehouse Trifecta

Get a detailed overview of Delta Lake, Apache Hudi, and Apache Iceberg as we discuss their data storage, processing capabilities, and deployment options.

April 26, 2023

by Andrey Gusarov

· 9,856 Views · 4 Likes

Frequently Faced Challenges in Implementing Spark Code in Data Engineering Pipelines

This article discusses some of the common challenges faced by data engineers in Pyspark applications and the possible solutions to overcome these challenges.

April 25, 2023

by Amlan Patnaik

· 5,129 Views · 2 Likes

A Critical Detail About Kafka Partitioners

Understanding the role of a partitioner in the Kafka producer is important to understand to build effective Kafka applications.

April 25, 2023

by Bill Bejeck

· 4,047 Views · 2 Likes

How To Install CMAK, Apache Kafka, Java 18, and Java 19 [Video Tutorials]

This article has video tutorials on installing CMAK, Apache, Kafka, Java 18, Java 19, and JDK on Windows, Amazon EC2, and Linux.

April 25, 2023

by Ram N

· 8,494 Views · 2 Likes

API Gateway For ChatGPT Plugins

Learn how API Gateway can be beneficial for ChatGPT plugin developers to expose, secure, manage, and monitor their API endpoints.

April 23, 2023

by Bobur Umurzokov

· 4,521 Views · 2 Likes

Producing and Consuming Avro Messages With Redpanda Schema Registry

In this article, learn how to use Redpanda’s built-in schema registry to produce and consume Apache Avro messages in five simple steps.

April 21, 2023

by Dunith Dhanushka

CORE

· 4,090 Views · 1 Like

Lambda Architecture: A Powerful Approach to Data Engineering

In this article, we will explore the Lambda Architecture in detail, understanding its key concepts, benefits, and challenges.

April 21, 2023

by Amlan Patnaik

· 4,764 Views · 6 Likes

How to Stream Sensor Data to Apache Pinot for Real Time Analysis

Learn how to use Arduino, MQTT, Kafka, and Apache Pinot to stream environmental data for real-time access to sensor data.

April 21, 2023

by David G. Simmons

CORE

· 5,988 Views · 7 Likes

IoT Testing: How It Works and Why You Can’t Do Without It

This article explains IoT testing, how to do it correctly, which challenges you can face along the way, and why.

April 21, 2023

by Inna Martyniuk

· 3,711 Views · 2 Likes

Consumer Group in Kafka [Video Tutorials]: Partitions and Consumers

In this video tutorial series, take a deeper look at the consumer group in Kafka with varying numbers of partitions and consumers.

April 21, 2023

by Ram N

· 5,519 Views · 2 Likes

Static Proxies or Rotating Proxies: Which One Should You Choose?

This article explains the differences between Static proxies and Rotating proxies. Let's find out what they are.

April 20, 2023

by Hanzala Ali

· 4,761 Views · 1 Like

Kafka: The Basics

Kafka is a powerful tool for building streaming architectures. The article serves as an introduction to both the technology and associated data producers and consumers.

April 20, 2023

by Pavel Micka

· 7,611 Views · 6 Likes

Recording Badge Scans in Apache Pinot

Use a real-time analytics database to record NFC badge scans in part 1 of this 3-part project involving environmental data, real-time notifications, and more.

April 19, 2023

by David G. Simmons

CORE

· 4,348 Views · 4 Likes

Integrate Apache Spark and QuestDB for Time-Series Analytics

In this tutorial, we will explore how a Spark Application can load and save data from/to QuestDB and provide practical advice to achieve the best performance.

April 19, 2023

by Imre Aranyosi

· 3,956 Views · 8 Likes

The Latest Big Data Topics