Big Data Resources

Partitioning Hot and Cold Data Tier in Apache Kafka Cluster for Optimal Performance

Discover how by partitioning the hot and cold data tiers in the Apache Kafka Cluster, we can optimize storage resources based on data characteristics.

June 28, 2024

by Gautam Goswami

CORE

· 9,286 Views · 5 Likes

Node.js Walkthrough: Build a Simple Event-Driven Application With Kafka

Build a real-time event-driven app with Node.js and Kafka on Heroku. Follow this step-by-step guide to set up, deploy, and manage your application efficiently.

June 27, 2024

by Alvin Lee

CORE

· 15,335 Views · 3 Likes

Data Governance: Data Integration (Part 4)

It's important to have a strong data governance framework in place when integrating data from different source systems. Learn more!

June 27, 2024

by Sukanya Konatam

· 12,322 Views · 1 Like

Building Kafka Producer With Spring Boot

In this article, I am going to show you how to build Message Publisher using Apache Kafka and Spring Boot. First, we will talk about what Apache Kafka is.

June 26, 2024

by Sai Krishna Reddy Chityala

· 7,165 Views · 3 Likes

Cluster Logging of Telecom 5G IOT Microservice Pods

Learn how Kibana visualizes data indexed in Elasticsearch log storage for vast 5G Telecom data volumes in the Kubernetes Cloud.

June 26, 2024

by BINU SUDHAKARAN PILLAI

· 14,995 Views · 2 Likes

Pure Storage Unveils Groundbreaking Innovations in AI, Automation, and Cyber Resilience at Accelerate 2024

Pure Storage announces next-gen storage automation, AI-powered management, and advanced cyber resilience solutions at Accelerate 2024 to make storage simpler.

June 26, 2024

by Tom Smith

CORE

· 9,417 Views · 1 Like

What Is ELT?

Learn about ELT (Extract, Load, Transform), a modern data management approach that loads raw data into cloud systems for scalable, real-time transformation and analytics.

June 25, 2024

by Suhas Jangoan

· 8,181 Views · 2 Likes

When Not To Use Apache Kafka (Lightboard Video)

When not to use Apache Kafka, do's and don't's; no matter if you use open source, Confluent, Amazon MSK, Event Hubs, Redpanda, Warpstream, et al.

June 25, 2024

by Kai Wähner

CORE

· 2,375 Views · 1 Like

Advanced SQL for Data Engineering

Advanced SQL is a powerful tool that allows you to retrieve, analyze, and manipulate large amounts of data in a structured and efficient way.

June 25, 2024

by Harsh Daiya

CORE

· 6,471 Views · 7 Likes

Optimizing InfiniBand Bandwidth Utilization for NVIDIA DGX Systems Using Software RAID Solutions

Leveraging SPDK vhost-blk with software RAID on NVIDIA DGX systems significantly improves InfiniBand bandwidth utilization and performance.

June 24, 2024

by Sergey Platonov

· 2,299 Views · 1 Like

MaxLinear Empowers High-Speed Connectivity and Data Acceleration Solutions for Next-Gen Computing

Advanced connectivity solutions and Panther III storage accelerator empower software engineers to build high-performance, data-driven computing systems.

June 22, 2024

by Tom Smith

CORE

· 3,257 Views · 1 Like

Embracing NoSQL: The Future of Data Storage and Retrieval

NoSQL databases handle unstructured data, provide scalability, and offer high availability, ideal for modern, big data, and real-time applications.

June 21, 2024

by Lalithkumar Prakashchand

· 3,929 Views · 4 Likes

IoT Needs To Get Serious About Security

Security issues in IoT have gotten worse, not better, and it's time we acknowledge that and fix it. It's long past time.

June 20, 2024

by David G. Simmons

CORE

· 7,415 Views · 3 Likes

Unleashing the Power of Cloud Storage With JuiceFS

JuiceFS, a cloud-native file system, empowers enterprises with scalable, high-performance storage for data-intensive AI and analytics workloads.

June 20, 2024

by Tom Smith

CORE

· 3,646 Views · 2 Likes

Harnessing Kafka Streams for Seamless Data Tasks

Learn about the use of Kafka for convenient data exploration and dynamic data integration with its "publish-subscribe" model, organizing data into topics.

June 18, 2024

by Ilia Ivankin

· 4,353 Views · 1 Like

Another Lifesaver for Data Engineers: Apache Doris Job Scheduler for Task Automation

The built-in Doris Job Scheduler triggers pre-defined operations efficiently and reliably. It is useful in many cases including ETL and data lake analytics.

June 17, 2024

by Shirley H.

· 1,585 Views · 1 Like

Build an ELT Pipeline From MySQL Database

Easily set up MySQL Change Data Capture using open-source Airbyte and Debezium to construct a near real-time ELT pipeline.

June 17, 2024

by Thalia Barrera

· 2,837 Views · 2 Likes

The Past, Present, and Future of Stream Processing

Stream Processing Journey with IBM, Apama, TIBCO StreamBase, Kafka Streams, Apache Flink, Streaming Databases, GenAI, and Apache Iceberg.

June 17, 2024

by Kai Wähner

CORE

· 8,359 Views · 7 Likes

Explore the Complete Guide to Various Internet of Things (IoT) Protocols

As the IoT industry continues to evolve, it brings about revolutionary changes in device communication, further underscoring the importance of IoT protocols.

June 17, 2024

by BINU SUDHAKARAN PILLAI

· 2,781 Views · 2 Likes

Custom Health Checks in Spring Boot

Spring Boot provides health indicators to monitor application health and database health with the help of Spring Boot Actuator.

June 16, 2024

by Sai Krishna Reddy Chityala

· 8,037 Views · 4 Likes

The Latest Big Data Topics