Big Data Resources

JavaScript, Node.js, and Apache Kafka for Full-Stack Data Streaming

Explore JavaScript, Node.js, and Apache Kafka for full-stack development for data streaming with open-source clients.

June 10, 2024

by Kai Wähner

CORE

· 6,619 Views · 2 Likes

Twilio Empowers Developers With Enhanced Data Warehouse Interoperability

Linked Audiences and Segment Data Graph revolutionize customer engagement by unlocking the full potential of data warehouses for developers and marketers.

June 10, 2024

by Tom Smith

CORE

· 2,020 Views · 1 Like

Using AWS Data Lake and S3 With SQL Server: A Detailed Guide With Research Paper Dataset Example

The integration of AWS Data Lake and Amazon S3 with SQL Server provides the ability to store data at any scale and leverage advanced analytics capabilities.

June 7, 2024

by Vijay Panwar

CORE

· 5,574 Views · 1 Like

Comprehensive Guide to Data Analysis and Visualization With Pandas and Matplotlib

Covering data preparation, descriptive statistics, profit margin calculations, and various types of plots such as bar charts, pie charts, and scatter plots.

June 7, 2024

by Krishnamurty Raju Mudunuru

· 6,900 Views · 1 Like

The Role of Data Engineering in Building Scalable AI Systems

Explore the role of data engineering in the field of artificial intelligence, specifically how data engineering influences the success and scalability of AI projects.

June 7, 2024

by Karthik Rajashekaran

CORE

· 2,939 Views · 2 Likes

ChatGPT: Your Guide to SQL Query Translation Between Databases

ChatGPT offers a solution for SQL translation by accurately translating queries, handling data types, and suggesting alternatives for proprietary features.

June 7, 2024

by Dmitry Narizhnykh

CORE

· 2,581 Views · 2 Likes

Building a Tool To Generate Text With OpenAI’s GPT-4 Model

Build a text generation tool using OpenAI's GPT-4. Set up your environment, authenticate the API, make API calls, handle errors, and integrate with Flask for a web app.

June 6, 2024

by Neha Dhaliwal

· 2,982 Views · 4 Likes

Streamlining Data Integration

This article demonstrates integrating Salesforce and Oracle into Amazon Redshift using Python and stored procedures, leveraging the ELT approach.

June 6, 2024

by Rajesh Remala

· 4,251 Views · 2 Likes

Take Your First Steps for Building on LLMs With Google Gemini

Learn to build an LLM application using the Google Gemini API and deploy it to Heroku. This guide walks you through setup, code, and deployment step-by-step.

June 5, 2024

by Alvin Lee

CORE

· 4,697 Views · 2 Likes

What Is Reverse ETL? Overview, Use Cases, and Key Benefits

Looking to go beyond traditional analytics? Reverse ETL is a nuanced process for businesses aiming to leverage data warehouses and other data platforms.

June 4, 2024

by Suhas Jangoan

· 5,279 Views · 2 Likes

High-Performance Storage Solutions

Pairing PostgreSQL with fast storage solutions is crucial for optimizing performance, minimizing downtime, and ensuring seamless data access.

June 3, 2024

by Sergey Platonov

· 2,062 Views · 3 Likes

An Overview of Data Pipeline Architecture

Dive into how a data pipeline helps process enormous amounts of data, key components, various architecture options, and best practices for maximum benefits.

Updated June 3, 2024

by Sreenath Devineni

CORE

· 13,939 Views · 5 Likes

Profiling Big Datasets With Apache Spark and Deequ

Struggling to maintain large dataset quality or ensure reliable data attributes in your analyses? Integrating Deequ with Spark might be the solution you need.

May 29, 2024

by Akshay Jain

· 77,702 Views · 62 Likes

Streamlining Data Warehouse Migrations From RedShift, Snowflake, and BigQuery to Apache Doris

Migrate Redshift, Snowflake, and BigQuery to Apache Doris for high-performance OLAP using tools like Apache Nifi, Talend, and Python.

May 28, 2024

by Sameer Danave

· 889 Views · 2 Likes

Preventing and Fixing Bad Data in Event Streams (Part 1)

In this blog, we’ll take a look at how bad data may come to be, and how we can deal with it when it comes to event streams.

May 28, 2024

by Adam Bellemare

· 3,773 Views · 3 Likes

ARM CPU for Cost-Effective Apache Kafka at the Edge and Cloud

Learn about ARM CPU for cost-effective and energy-efficient data streaming with Apache Kafka at the Edge and in the Cloud, including use cases.

May 27, 2024

by Kai Wähner

CORE

· 1,986 Views · 1 Like

Bridging the Gap: Exploring Innovative Paths in Data Integration

This article solely focuses on delving into understanding data integrations, associated challenges, emerging technologies, and advanced methodologies.

May 27, 2024

by Yash Mehta

· 794 Views · 1 Like

Apache Doris for Log and Time Series Data Analysis

NetEase has replaced Elasticsearch and InfluxDB with Apache Doris, achieving 11X query performance and saving 70% of resources.

May 25, 2024

by Shirley H.

· 1,419 Views · 1 Like

Capturing Acknowledgement in Kafka Streaming With RecordMetadata

In this article, explore RecordMetadata, and its attributes, and discover how it can be leveraged using various use cases.

May 24, 2024

by Raja Chattopadhyay

· 2,911 Views · 2 Likes

Useful Tips and Tricks for Data Scientists

In this article, I will share some of the tricks and tools that I am using to interpret the data in a fast and precise way and get useful insights from it.

May 22, 2024

by Pavel Perfilov

· 5,138 Views · 35 Likes

The Latest Big Data Topics