Building highly reliable data pipelines at Datadog | DataEngConf BCN ’18

Building highly reliable data pipelines at Datadog | DataEngConf BCN ’18

Download slides of this talk: https://www.dataengconf.com/speaker/building-highly-reliable-data-pipelines-at-datadog?hsLang=en-us?utm_source=youtube&utm_medium=social&utm_campaign=%20-%20DEC-BCN-18%20Slides%20Download

ABOUT THE TALK:

Every day at Datadog we run massive data pipelines to power core functionality. In this talk, we’ll cover best practices we use at Datadog to ensure that we reliably deliver this functionality while processing trillions of points/day in the face of exponential data growth, hardware failures, corrupt data, and even human error.

ABOUT THE SPEAKER:

Quentin is a Data Engineering Team Lead at Datadog. Over the past 3 years, he has worked with Luigi, Spark, and Go to build and scale many highly reliable data pipelines. These pipelines power numerous internal and customer facing features – including enabling Datadog’s users to perform fast real time queries against PBs of historical time series data.

Related Post: