YOU ARE DOWNLOADING DOCUMENT

Please tick the box to continue:

Transcript
Page 1: EatSleepTweetRepeat_BarshaShrestha

Barsha ShresthaInsight Data Engineering

Page 2: EatSleepTweetRepeat_BarshaShrestha

Motivation:

APIs:

>300GB

Page 3: EatSleepTweetRepeat_BarshaShrestha

DATA INGESTION FILE SYSTEM/

FRONT-END

BATCH PROCESSING

DATABASE

Page 4: EatSleepTweetRepeat_BarshaShrestha

DATA INGESTION FILE SYSTEM/

BATCH PROCESSING

DATABASE

FRONT-END

Page 5: EatSleepTweetRepeat_BarshaShrestha

PRODUCER

PRODUCER

CONSUMER

Page 6: EatSleepTweetRepeat_BarshaShrestha

DATA INGESTION FILE SYSTEM/

BATCH PROCESSING

DATABASE

FRONT-END

Page 7: EatSleepTweetRepeat_BarshaShrestha

TMDB

Page 8: EatSleepTweetRepeat_BarshaShrestha

DATA INGESTION FILE SYSTEM/

FRONT-END

BATCH PROCESSING

DATABASE

Page 9: EatSleepTweetRepeat_BarshaShrestha

Challenges1. Learning new technologies!2. Next steps

Page 10: EatSleepTweetRepeat_BarshaShrestha

About me:

Barsha Shrestha

Chemistry and Computer Science Movies, Soccer, Science Jokes, Bad puns

Page 12: EatSleepTweetRepeat_BarshaShrestha