3/13/2017 qcon-london2017-datapipelines slides file:///Users/KimberlyAmaral/Downloads/qcon-london2017-datapipelines.slides.html 1/8 Effective Data Pipelines: Data Management from Chaos Katharine Jarmul (@kjam) QCon - London - March 6, 2017 About Katharine Data Scientist, Engineer, Author, Pythonista Founder @ kjamistan UG: data science consulting & engineering Find me at: kjamistan.com - [email protected] - @kjam
8
Embed
Effective Data Pipelines: Data Management from Chaos€¦ · - There are several teams involved in my pipeline (for security, maintainability and development); however, there is
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
- There are clear solutions for replaying,rerunning and interrupting tasks ordataflow in my pipeline.
- There are several teams involved in mypipeline (for security, maintainability anddevelopment); however, there is a clearchain of responsiblity and protocol forwhen things go wrong.
- We have reviewed business andstakeholder use cases. We chose apipeline structure fitting our currentconstraints with a straightforward path forgrowth and change.