Enterprise Strategy Group | Getting to the bigger truth.™ Risky Business: How to Balance Innovation & Risk in Big Data Scott Gidley, VP of Products, Zaloni Nik Rouda, Senior Analyst, ESG © 2016 by The Enterprise Strategy Group, Inc.
Jan 21, 2017
©2015byTheEnterpriseStrategyGroup,Inc.
EnterpriseStrategyGroup|Gettingtothebiggertruth.™
RiskyBusiness:HowtoBalanceInnovation&RiskinBigData
ScottGidley,VPofProducts,ZaloniNikRouda,SeniorAnalyst,ESG
©2016byTheEnterpriseStrategyGroup,Inc.
©2015byTheEnterpriseStrategyGroup,Inc. Zaloni Confidential and Proprietary - Provided under NDA
• Award-winning provider of enterprise data lake management solutions:
Integrateddatalakemanagementplatform
Self-servicedatapreparation
• Data Lake Design and Implementation Services: POC,Pilot,Production,Operations,Training
• Data Science Professional Services
2
Deliveringonthebusinessofbigdata
Financed by top-tier technology investors:
AboutESG• ESGisanITanalyst,research,andstrategycompany.
• Ourfirmwasfoundedin1999withheadquartersinMilford,MA/ananalystandclientrelationspresenceinSiliconValley,CA.
• ESGconductsresearchwithandforITvendors,ITprofessionals,businessprofessionals,andchannelpartners.
• Wemaintainongoinganalystcoverageincloudcomputing,networking,storage,dataprotection,cybersecurity,datamanagementandanalytics,applicationdevelopmentanddeployment,enterprisemobility,andchannels.
• Capabilitiesinclude:Analystservices,marketresearch,technicalperformancetesting,economicvalidation,consulting,andcustomcontent.
©2016byTheEnterpriseStrategyGroup,Inc.
ImportanceofBigData&AnalyticsProjects
48%
32%
14%
3% 2% 1%
20%
40%
21%
13%
4%1%
Ourmostimportantpriority
Oneofourtop5priorities
Oneofourtop10priorities
Oneofourtop20priorities
Notamongourtop20priorities
Don’tknow/noopinion
Relativetoallofyourorganization’sbusinessandITprioritiesoverthenext12-18months,howwouldyouratetheimportanceofitsbigdataanalyticsprojectsandinitiatives?(Percentofrespondents,N=475)
Importanceofbigdataanalyticsprojectsandinitiativesrelativetoallbusinesspriorities
ImportanceofbigdataanalyticsprojectsandinitiativesrelativetoallITpriorities
ComparedtobusinessprioritiesComparedtoITpriorities
Apartialmismatchasbigdataisessentialforthebusiness,yetisjustoneofmanygoalsforIT.
©2016byTheEnterpriseStrategyGroup,Inc.
Top102016ITPriorities
18%
19%
20%
20%
20%
20%
21%
22%
23%
37%
Businesscontinuity/disasterrecoveryprograms
Improvingcollaborationcapabilities
Desktopvirtualization
Increasinguseofservervirtualization
Majorapplicationdeploymentsorupgrades
Improvingdatabackupandrecovery
Dataintegration
Managingdatagrowth
Businessintelligence/dataanalyticsinitiatives
Cybersecurityinitiatives
Takentogether,thisiseffectively
adatalake.
Top10mostimportantITprioritiesoverthenext12months.(Percentofrespondents,N=633,tenresponsesaccepted)
©2016byTheEnterpriseStrategyGroup,Inc.
ESGSurveyRespondentsRepresentStakeholders
Databaseadministrator,26%
Managerofdevelopmentordeveloperofbusinessintelligence/analytics
solutions,23%Dataengineer,13%
Dataanalyst,12%
Datawarehouse/businessintelligence/analytics
manager,9%
Enterpriseordataarchitect,7%
Businessanalyst,6%
Datascientist,5%
DBAsanddevelopersstilldominate,evenasmanyrolescomeintoplay.
Whichofthefollowingbestdescribesyourprimaryareaofresponsibility?(Percentofrespondents,N=475)
©2016byTheEnterpriseStrategyGroup,Inc.
WhoInitiatesNewBigDataandAnalyticsProjects?
18%
19%
20%
26%
28%
29%
30%
40%
41%
45%
Legal/risk/compliance
Marketingmanagement
Salesmanagement
Informationsecurityteams
Operationsmanagement
Seniorbusinessexecutives(e.g.,CEO,CFO,etc.)
Businessanalyst/datascientistteam
SeniorITexecutives(e.g.,CIO,CTO,etc.)
ITinfrastructureandoperationsteam
ITapplicationsteam
Whichofthefollowinggroupsareinitiatingnewprojectsintheareaofbigdataandanalytics?(Percentofrespondents,N=475,multipleresponsesaccepted)
IT
Bridge
Business
TraditionalIThasreassertedcontrolandtakenresponsibility.
©2016byTheEnterpriseStrategyGroup,Inc.
23%
23%
25%
27%
29%
32%
52%
53%
Serviceprovider
Value-addedreseller(VAR)
Businessapplicationvendor
Managementconsultancy
Systemsintegrator(SI)
Businessanalyst/datascientistteam
ITapplicationsteam
ITinfrastructureandoperationsteamInternalstaff
Third-partyexperts
Whichofthefollowinggroupsprovidestheskillsandmanpowertoimplementandmanagethetechnologiessupportinginitiativesinthe areaofbigdataandanalytics?(Percentofrespondents,N=475,multipleresponsesaccepted)
36%citeaproblematicskillsshortageforbusinessintelligence&analytics
DataLakesSuccessWill“TakeaVillage”
©2016byTheEnterpriseStrategyGroup,Inc.
MostExpectTimetoValueof>6Months
7%
15%
35%
31%
11%
0%
Wewillseevalueimmediately
1monthto6months 7monthsto12months
13monthsto24months
25monthsto36months
Morethan36months
Fornewinitiativesintheareaofbigdataandanalytics,howlongdoyouthinkitwilltakeforyourorganizationtostartseeingsignificantbusinessvalue?(Percentofrespondents,N=475)
©2016byTheEnterpriseStrategyGroup,Inc.
FundamentalDataLakeProcessesCaptureandaggregatedata
Prepareandtransform
Discoverandexplore
Buildmodelsandanalyze
Visualizeandreport
Maintainandsupport
©2016byTheEnterpriseStrategyGroup,Inc.
HadoopDataLakeImplementationPlans
AlreadyusingHadoop,20%
VeryinterestedinHadoop,37%
SomewhatinterestedinHadoop,27%
NotatallinterestedinHadoop,5%
NotfamiliarwithHadooptechnology,8%
Don’tknow,3%
Howwouldyourateyourorganization’sinterestinimplementingHadoop?(Percentofrespondents,N=475)
©2016byTheEnterpriseStrategyGroup,Inc.
HowHadoopDataLakesWillImpactDataWarehouses
Hadoopwilllargelyreplaceourexistingdatawarehouse,26%
Hadoopwilloffload/optimizeour
existingdatawarehouse,36%
Hadoopwillbeusedonlyforlimiteddatawarehouse-like
functions,28%
NoplanstouseHadoopforanydatawarehouse-like
functions,11%
HowdoyouanticipateHadoopwillfitagainstyourorganization’straditionaldatawarehouseapproach?(Percentofrespondents,N=94)
©2016byTheEnterpriseStrategyGroup,Inc.
TypesofHadoopDistributionsPlanned
Pureopensourcedistribution(i.e.,ApacheHadoop),24%
Commercialdistribution(e.g.,Cloudera,MapR,Hortonworks,etc.),35%
Hybridapproach(i.e.,someopensourcecombinedwith
somecommercialdistributions),40%
Don’tknow,1%
WhichofthefollowingdescribesthetypeofHadoopdistribution(s)yourorganizationiscurrentlyevaluating?(Percentofrespondents,N=300)
Zaloni Confidential and Proprietary - Provided under NDA
Data lake challenges and complications
Building: Managing: Delivering:
• Ingestion
• Lack of Visibility
• Privacy and Compliance
• Quality Issues
• Reliance on IT
• Reusability
• Rate of Change
• Skills Gap
• Complexity
Zaloni Confidential and Proprietary - Provided under NDA
Zaloni’s data lake solution
• Ingestion
• Lack of Visibility
• Privacy and Compliance
• Quality Issues
• Reliance on IT
• Reusability
• Rate of Change
• Skills Gap
• Complexity
Managing: Delivering:
Govern the data in the lake
• Cleanse• Secure
• Operationalize
Enable the data lake
• Ingest• Organize• Catalog
Building:
Engage the business
• Discover• Enrich
• Provision
Zaloni Confidential and Proprietary - Provided under NDA
Data Lake 360: Zaloni’s holistic approach to actionable big data
1. Enable the lake2. Govern the data
3. Engage the business
• Foster a data-driven business through self-service data discovery and preparation
• Safeguard sensitive data and enable regulatory compliance
• Improve data visibility, reliability and quality to reduce time-to-insight
• Leverage the full power of a scale-out architecture with an actionable, scalable data lake
Zaloni Confidential and Proprietary - Provided under NDA
Data Lake 360: Zaloni’s holistic approach to actionable big data
Zaloni Confidential and Proprietary - Provided under NDA
Data lake reference architectureConsumption
ZoneSource System
File Data
DB Data
ETL Extracts
Streaming
TransientLoading Zone
Raw Data Refined Data
Trusted Data
DiscoverySandbox
Original unaltereddata attributes
Tokenized Data
APIs
Reference Data Master Data
Data WranglingData DiscoveryExploratory Analytics
Metadata Data Quality Data Catalog Security
Data Lake
Integrate to common formatData ValidationData CleansingAggregations
OLTP or ODS
Enterprise Data Warehouse
Logs(or other unstructured
data)
Cloud Services
Business AnalystsResearchersData Scientists
Challenge: The company needed a more cost-effective, flexible and expandable data processing and storage environment that would reduce mainframe load and mainframe support risk
Solution: Zaloni built a data lake architecture to offload mainframe data security into Hadoop leveraging the Bedrock data lake management platform
Zaloni Enables EDW Augmentation That Saves Millions
GLOBAL CONSUMER INSIGHTS FIRMIndustry: Market ResearchCompany Description: Top 10 Market Research FirmTechnical Use Case: EDW AugmentationBig Data Technologies: Zaloni Bedrock, MapR, ZookeeperDeployment: On premises
At a Glance
Zaloni Confidential and Proprietary - Provided under NDA
$5.2MILLIONannual savings
$4.4MILLIONprojected additional
savings
Nearly
50%REDUCTIONin mainframe usageconsumption
1M+records/secondthroughput rate
Results
Challenge: To develop a multi-vendor loyalty program required gathering information from a wide variety of sources, bring it all into a unified platform, and give marketers access to the data for customer targeting.Solution: Zaloni built a data lake to house all customer data, as well as an interactive visualization platform that enabled marketers to profile customers on selection (based on demographics, geography, etc.) for analysis.
Zaloni Enables 360˚ Customer View For Effective Loyalty Program
GLOBAL CREDIT CARD ISSUER Industry: Financial ServicesCompany Description: Global credit card issuerTechnical Use Case: Data Lake 360˚ - Agile AnalyticsBusiness Use Case: Customer 360˚ - Behavioral Analytics and Customer Segmentation Big Data Technologies:Zaloni Professional Services, MapR, Elasticsearch
Deployment: On premises
Zaloni Confidential and Proprietary - Provided under NDA
At a Glance
1Mcustomer sign-ups 22M
users
collecting data from 1M+
hits/dayon the customer
Results
in 3 weeks system
©2016byTheEnterpriseStrategyGroup,Inc.
ESGFindingsontheValueofZaloniBedrock
KeyCustomerBenefitsSummary:• Dramaticallyreducedtimeto
implementBedrockversuscustom-developedtools
• Easeofoffloadingongoingenhancementsandoperationalmanagement
• Increasedanalystproductivitywithlessexpertiserequired
• Reducedtimetoinsightaddingtothebottomline
“Forevery300dataengineersouttheremaybeoneortwohasreallygoodHadoopexperience.Weestimated6-8months,and$2-3Mdevelopmenttorolloutjustthebasics,assumingwecouldfindthepeople.”
“Iwouldneedatleasttwoengineerstosupportandmaintainapplicationsovertime.Thisequatestoatleast$300KinstaffOPEXvs.ourlicensingcostwithBedrock.Nottomention,weareabletoleverageenhancementsthatserveZaloni’sentirecommunityofcustomers,notjustwhatourinternalpeoplerequest.”
©2016byTheEnterpriseStrategyGroup,Inc.
EstimatedEconomicValueofZaloniBedrock
Table1.Three-yearTCO,ZaloniversusthePMO
Category Zaloni PMOHardware $250,000 $469,156Software $900,000 $904,870Infrastructure $27,500 $28,600MaintenanceandSupport $135,000 $660,974ProfessionalServices $17,500 $15,000StaffPersonnel $378,000 $1,312,250Totalthree-yearcosts $1,708,000 $3,390,850
Source:EnterpriseStrategyGroup,2016.
©2016byTheEnterpriseStrategyGroup,Inc.
ITInvestmentJustification
14%
19%
19%
22%
27%
32%
35%
39%
Speedofpayback
Reductionincapitalexpenditures
Reducedtime-to-marketforourproductsorservices
Improvedregulatorycompliance
Reductioninoperationalexpenditures
Businessprocessimprovement
Returnoninvestment
Improvedsecurity/riskmanagement
WhichofthefollowingconsiderationsdoyoubelievewillbemostimportantinjustifyingITinvestmentstoyourorganization’sbusinessmanagementteamoverthenext12months?(Percentofrespondents,N=633,threeresponsesaccepted)
©2016byTheEnterpriseStrategyGroup,Inc.
ThankYou
EnterpriseStrategyGroup|Gettingtothebiggertruth.™
http://www.twitter.com/esg-global
http://www.facebook.com/ESGglobal
https://www.linkedin.com/groups?gid=1295607&trk=myg_ugrp_ovr
http://www.youtube.com/user/ESGglobal
FOLLOWESG
NikRouda,[email protected]