Sr Engineering Director at Factorial
Galicia, ES
Joined Mar 2021
Miguel has a great background in leading teams and building high-performance solutions for the retail sector. An advocate of platform design as a service and data as a product.
Stats
Reputation: | 1454 |
Pageviews: | 210.7K |
Articles: | 19 |
Comments: | 7 |
Getting Started With Vector Databases
Real-Time Data Architecture Patterns
Data Engineering
Over a decade ago, DZone welcomed the arrival of its first ever data-centric publication. Since then, the trends surrounding the data movement have held many titles — big data, data science, advanced analytics, business intelligence, data analytics, and quite a few more. Despite its varying vernacular, the purpose has remained the same: to build intelligent, data-driven systems. The industry has come a long way from organizing unstructured data and driving cultural acceptance to adopting today's modern data pipelines and embracing business intelligence capabilities.This year's Data Engineering Trend Report draws all former terminology, advancements, and discoveries into the larger picture, illustrating where we stand today along our unique, evolving data journeys. Within these pages, readers will find the keys to successfully build a foundation for fast and vast data intelligence across their organization. Our goal is for the contents of this report to help guide individual contributors and businesses alike as they strive for mastery of their data environments.
Data Pipelines
Enter the modern data stack: a technology stack designed and equipped with cutting-edge tools and services to ingest, store, and process data. No longer are we using data only to drive business decisions; we are entering a new era where cloud-based systems and tools are at the heart of data processing and analytics. Data-centric tools and techniques — like warehouses and lakes, ETL/ELT, observability, and real-time analytics — are democratizing the data we collect. The proliferation of and growing emphasis on data democratization results in increased and nuanced ways in which data platforms can be used. And of course, by extension, they also empower users to make data-driven decisions with confidence.In our 2023 Data Pipelines Trend Report, we further explore these shifts and improved capabilities, featuring findings from DZone-original research and expert articles written by practitioners from the DZone Community. Our contributors cover hand-picked topics like data-driven design and architecture, data observability, and data integration models and techniques.
Data Pipelines
Data is at the center of everything we do. As each day passes, more and more of it is collected. With that, there’s a need to improve how we accept, store, and interpret it. What role do data pipelines play in the software profession? How are data pipelines designed? What are some common data pipeline challenges? These are just a few of the questions we address in our research.In DZone’s 2022 Trend Report, "Data Pipelines: Ingestion, Warehousing, and Processing," we review the key components of a data pipeline, explore the differences between ETL, ELT, and reverse ETL, propose solutions to common data pipeline design challenges, dive into engineered decision intelligence, and provide an assessment on the best way to modernize testing with data synthesis. The goal of this Trend Report is to provide insights into and recommendations for the best ways to accept, store, and interpret data.
Comments
Nov 30, 2023 · Miguel Garcia
Thanks for your opinion!
Jul 02, 2023 · Miguel Garcia
We don't provide any data, we provide how the API works and what are the datasets available. Chatgpt returns to the frontend the API URL to get the information requested by the user. Eventually, the frontend launch the request provides by openai.
Jan 17, 2023 · Miguel Garcia
Thanks for the comment! That recommendation is a very useful recommendation and helps to solve one of the reasons. Some people have difficulties with time management, they need to learn techniques and tools to help them.
Dec 15, 2022 · Miguel Garcia
Thanks
Dec 15, 2022 · Miguel Garcia
Thanks!
Feb 05, 2022 · Dario Cazas Pernas
Great article!
Nov 17, 2021 · Miguel Garcia
Thank you for your feedback!