How to overcome data silos to achieve Big Data business transformation

Agenda

• 4 Vs of Big Data and their impact

• Typical ‘Big Data’ landscape

• Considerations for tools

• How to start talking Big?

• How can Ashnik help?

4 Vs of Big Data equation

• Velocity

• Volume

• Variety

• Veracity

Impact of 4 Vs on technology

• Database

• Scale out architecture

• Data integration and enrichment

• Different shades of grey

Typical Big Data Landscape

Which tools can help you?

• Data Store – Postgres, MongoDB, Elastic, Couchbase, Hadoop

• Data Integration – Pentaho, Foreign Data Wrapper, Logstash

• Data Enrichment – Pentaho, Beats, Logstash

• Data Accuracy – Natural Language Processing (Python, R etc)

Considerations for Data Store

• What data model?• Nature of queries - Correlation, Aggregation or more?• How Big is your Big Data?• Linking or embedding?• Real Time or Batch?• Cost per GB Vs Business Value?

Considerations for Data Integration tools

• What are your data sources? - REST, RDBMS, noSQL, RSS...

• What connectors are available? - API call, ODBC, JDBC...

• Do you need much custom routines? - cleansing, filtering, modelingetc...

• What Data formats? - XML, JSON, Relational etc

Considerations for Data Enrichment tools

• Final and intermediate data formats?

• Do you need to normalize/de-normalize?

• Do you need to add additional info? e.g. timestamp, user, sensor-id etc

• Do you need to change the format? e.g. XML to JSON

Learnings from Early Failures

• Choose the right data-store

• Don’t build another Data warehouse - only larger

• Don’t end up with more Data Silos - only put together in one store

• Data must correlate and make business sense

How to get you talking Big?

• Identifying business problem

• Overcoming data silos

• Integrating and co-relating the data from different sources

• Identify the queries - Analytics

How can Ashnik Help you?

• We partner with different technologies• Vendor-independent consulting

• Bring the experience to your team• Database• Data Modelling• Integration

• End-to-end engagements• Consulting• Services• Training