Top Banner
Copyright © 2016 Splunk Inc. Dr. Adam Oliner Director of Engineering, Data Science, Splunk Using the Splunk Machine Learning Toolkit to Create Your Own Custom Models Manish Sainani Principal Product Manager, Splunk
61

Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Mar 09, 2018

Download

Documents

lenguyet
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Page 1: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Copyright©2016Splunk Inc.

Dr.AdamOlinerDirectorofEngineering,DataScience,Splunk

UsingtheSplunk MachineLearningToolkittoCreateYourOwnCustomModels

ManishSainaniPrincipalProductManager,Splunk

Page 2: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Disclaimer

2

Duringthecourseofthispresentation,wemaymakeforwardlookingstatementsregardingfutureeventsortheexpectedperformanceofthecompany.Wecautionyouthatsuchstatementsreflectourcurrentexpectationsandestimatesbasedonfactorscurrentlyknowntousandthatactualeventsorresultscoulddiffermaterially.Forimportantfactorsthatmaycauseactualresultstodifferfromthose

containedinourforward-lookingstatements,pleasereviewourfilingswiththeSEC.Theforward-lookingstatementsmadeinthethispresentationarebeingmadeasofthetimeanddateofitslivepresentation.Ifreviewedafteritslivepresentation,thispresentationmaynotcontaincurrentoraccurateinformation.Wedonotassumeanyobligationtoupdateanyforwardlookingstatementswemaymake.Inaddition,anyinformationaboutourroadmapoutlinesourgeneralproductdirectionandissubjecttochangeatanytimewithoutnotice.Itisforinformationalpurposesonlyandshallnot,beincorporatedintoanycontractorothercommitment.Splunkundertakesnoobligationeithertodevelopthefeaturesor

functionalitydescribedortoincludeanysuchfeatureorfunctionalityinafuturerelease.

Page 3: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Whoarewe?

3

Dr.AdamOliner– DirectorofEngineering,DataScience&MachineLearning– Splunker for2years– Embarrassinglyovereducated

ManishSainani– PrincipalProductManager,MachineLearning– Splunker for2years– FirstMLhireatSplunk!

Page 4: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Whatarewedoinghere?

4

OverviewofMachineLearningTheAssistants:GuidedMachineLearning– Prepare– Fit– Validate– Deploy

Examples– DIYAnomalyDetector– CustomerApplications

Page 5: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

OverviewofMLatSplunk

CorePlatformSearch PackagedPremiumSolutions CustomML

PlatformforOperationalIntelligence

Page 6: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

SplunkMachineLearningToolkit

Assistants: Guidemodelbuilding,testing,&deployingforcommonobjectivesShowcases: InteractiveexamplesfortypicalIT,security,business,IoTusecases

Algorithms: 25+standardalgorithmsavailableprepackagedwiththetoolkitSPLMLCommands:Newcommandstofit,testandoperationalizemodelsPythonforScientificComputingLibrary:300+opensourcealgorithmsavailableforuse

Buildcustomanalyticsforanyusecase

ExtendsSplunkplatformfunctionsandprovidesaguidedmodelingenvironment

Page 7: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

What’sNewsinceour0.9BetaRelease(lastyear’s.conf)?

7

• Newnameandabbreviation;-)• Noeventlimits(removalof50Klimitonfittingmodels)

• Configurableresourcecapsviamlspl.conf

• Searchheadclusteringsupport• Distributed/streamingapply• Scheduledfit• Newalgorithms(nextslide)

– Featureengineeringandselection– Stochasticgradientdescent(e.g.)– ARIMA

• Multi-algorithmsupportacrossAssistants

• Scatterplotmatrixviz• Alerting• Tooltips• In-apptours• ClusterNumericEventsassistant• VideosvideosvideosforeachassistantacrossIT,Security,IoT andBusinessAnalytics

• ML-SPLCheatSheet

Page 8: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Algorithmssupported(v2.0,.conf2016)

Page 9: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

TheAssistants:GuidedMachineLearning

Page 10: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

MachineLearning

10

AprocessforgeneralizingfromexamplesExamples– A,B,…→ # (regression)– A,B,... → a (classification)– Xpast → Xfuture (forecasting)– likewithlike (clustering)– |Xpredicted – Xactual|>>0 (anomalydetection)

Page 11: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

MachineLearningProcess

11

CollectData

Explore/Visualize

Model

Evaluate

Clean/Transform

Publish/Deploy

Page 12: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

MachineLearningProcesswithSplunk

12

CollectData

Explore/Visualize

Model

Evaluate

Clean/Transform

Publish/Deploy

props.conf,transforms.conf,DatamodelsAdd-onsfromSplunkbase,etc.

Pivot,TableUI,SPLMLToolkit

Alerts,Dashboards,Reports

Page 13: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

DomainExpertise(IT,Security,…)

DataScienceExpertise

SplunkExpertise

CustomMachineLearning– SuccessFormula

Identifyusecases

Drivedecisions

Setbusiness/opspriorities

SPL

Dataprep

Statistics/mathbackground

Algorithmselection

Modelbuilding

SplunkMLToolkitfacilitatesandsimplifiesviaexamples&guidance

Operationalsuccess

Page 14: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

GuidedMLwiththeAssistants

14

Guidesyouthroughvariousanalytics– Prepare,fit,validate,anddeploy

AutomaticallygeneratesalltherelevantSPL

Page 15: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Assistants:Fit

15

Page 16: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Assistants:Validate

16

Page 17: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Assistants:Deploy

17

Page 18: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

TheAssistants

18

1. PredictNumericFields2. PredictCategoricalFields3. DetectNumericOutliers4. DetectCategoricalOutliers5. ForecastTimeSeries6. ClusterNumericEvents

Page 19: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

PredictNumericFields

19

Algorithms– LinearRegression

ê …includingLasso,Ridge,andElasticNet– KernelRidge– DecisionTreeRegressor– RandomForestRegressor– SGDRegressor

Validation– Fourvisualizationsofpredictionerror– R2 andRMSE

Page 20: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

PredictCategoricalFields

20

Algorithms– LogisticRegression– DecisionTreeClassifier– RandomForestClassifier– SGDClassifier– SVM– NaïveBayes

ê BernoulliNB andGuassianNB

Validation– Precision,recall,accuracy,F1– Confusionmatrix

Page 21: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

DetectNumericOutliers

21

Methods– Standarddeviation– Medianabsolutedeviation– Interquartilerange

Validation:

Page 22: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

DetectCategoricalOutliers

22

StatisticalmethodsValidation:

Page 23: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ForecastTimeSeries

23

Algorithms– State-spacemethodusingKalman filter– ARIMA

Validation

Page 24: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ClusterNumericEvents

24

Algorithms– KMeans– DBSCAN– Birch– SpectralClustering

Validation– ScatterplotMatrixviz

Page 25: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Prepare

Page 26: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

DataGatheringandPrep

26

Source:CrowdFlower

Page 27: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Splunk!

27

Leadingplatformforcollecting,cleaning,andtransformingdataInteractiveFieldExtractorDatamodelsHundredsofadd-onsfromSplunkbasetransforms.confprops.confetc.

Page 28: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

FeatureEngineeringTFIDF(term-frequencyxinversedocument-frequency)– Transformfree-formtextintonumericattributes

StandardScaler (i.e.normalization)FieldSelector (i.e.choosekbestfeaturesforregression/classification)PCAandKernelPCA

Page 29: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

PreprocessingintheAssistants

29

Page 30: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Fit

Page 31: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Fit:What’sNew

31

NoeventlimitsConfigurableresourcecaps(ml-spl.conf)SearchheadclusteringsupportScheduledfitNewalgorithms

Page 32: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Fit:What’sNew

32

Page 33: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Validate

Page 34: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Validate/Apply:What’sNew

34

ConfigurableresourcecapsSearchheadclusteringsupportDistributed/streamingapplyScatterplotmatrixviz

Page 35: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ScatterplotMatrixViz

35

Page 36: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Deploy

Page 37: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

DeployanywhereinSplunk!

37

ScheduledtrainingAlertingReportsanddashboardsAugmentedsearchresultsetc.

Page 38: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Deploy:What’sNew

38

DistributedApply– Applymodelstoindexeddata– Streaming

ScheduledtrainingAlerting

Page 39: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

What’sNew:ScheduledFit

39

Page 40: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

What’sNew:Alerting

40

Page 41: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Example:DIYAnomalyDetector

Page 42: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

Let’sBuildanAnomalyDetector!

42

We’llusetwoAssistants– PredictNumericFields– DetectNumericOutliers

Showautomatically-generatedintermediateSPL

Page 43: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

FitaPredictiveModel

43

Page 44: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

SetupScheduledTraining

44

Page 45: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

OpenResidualsinSearch

45

Page 46: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

OpenDetectNumericOutliersAssistant

46

Page 47: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

DetectOutliers(LargePredictionErrors)

47

Page 48: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ScheduleanAlert

48

Page 49: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ScheduleanAlert

49

Page 50: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ScheduleanAlert

50

Page 51: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ManageYourNewAnomalyDetector

51

Page 52: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

TheAssistantGeneratedtheSPLforYou

52

Page 53: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

TheAssistantGeneratedtheSPLforYou

53

Page 54: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

YouBuiltanAnomalyDetector!

54

YoubuiltapredictivemodelofACPowerWhenthepredictionerrorfromthismodelisanoutliercomparedtopasterrors,yougenerateanalertThispredictivemodelautomaticallyretrainsitselfonascheduleyoucontrolYoudidn’thavetotypeanySPL

Page 55: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

#winning

Page 56: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

MachineLearningCustomerSuccess

NetworkOptimizationDetect&PreventEquipmentFailure Security/FraudPrevention

PrioritizeWebsiteIssuesandPredictRootCause

PredictGamingOutagesFraudPrevention

MachineLearningConsultingServices AnalyticsAppbuiltonMLToolkit

Optimizingoperationsandbusinessresults

PreventCellTowerFailureOptimizeRepairOperations

Entertainment Company

15

Page 57: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

MachineLearningToolkitCustomerUseCases

57

Speedingwebsiteproblemresolutionbyautomaticallyrankingactionsforsupportengineers

Reducingcustomerservicedisruptionwithearlyidentificationofdifficult-to-detectnetworkincidents

Minimizingcelltowerdegradationanddowntimewithimprovedissuedetectionsensitivity

Improvinguptimeandloweringcostsbypredicting/preventingcelltowerfailuresandoptimizing repairtruckrolls

Predictingandavertingpotentialgamingoutageconditionswithfiner-graineddetection

EnsuringmobiledevicesecuritybydetectinganomaliesinIDauthentication

PreventingfraudbyIdentifyingmaliciousaccountsandsuspiciousactivitiesEntertainment Company

Page 58: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

DetectNetworkOutliersReduceddowntime+increasedserviceavailability=bettercustomersatisfaction

58

MLUseCase Monitornoiserisefor20,000+celltowerstoincreaseserviceanddeviceavailability,reduceMTTR

Technicaloverview • Acustomizedsolutiondeployedinproductionbasedonoutlierdetection.• Leveragepreviousmonthdataandvotingalgorithms

“TheabilitytomodelcomplexsystemsandalertondeviationsiswhereITandsecurityoperationsareheaded…SplunkMachineLearninghasgivenusaheadstart...”

Page 59: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

ReliablewebsiteupdatesProactivewebsitemonitoringleadstoreduceddowntime

59

“SplunkMLhelpsusrapidlyimproveend-userexperiencebyrankingissue severitywhichhelpsusdeterminerootcausesfasterthusreducingMTTRandimprovingSLA”

• Veryfrequentcodeandconfig updates(1000+daily)cancausesiteissues• Finderrorsinserverpools,thenprioritizeactionsandpredictrootcause

• CustomoutlierdetectionbuiltusingMLToolkitOutlierassistant• BuiltbySplunkArchitectwithnoDataSciencebackground

MLUseCase

Technicaloverview

Page 60: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

WhatNow?

60

GettheMachineLearningToolkitfromSplunkbaseGowatchMachineLearningVideosonSplunkYoutube Channelhttp://tiny.cc/splunkmlvideosGotoMachineLearningstalks:– AdvancedMachineLearninginSPLwiththeMachineLearningToolkitbyJacobLeverich– ExtendingSPLwithCustomSearchCommandsandtheSplunkSDKforPythonbyJacobLeverich

SeveralCustomersandPartnerTalks– Cisco,Scianta Analytics,AsianTelco,etc.EarlyAdopterAndCustomerAdvisoryProgram:[email protected]:[email protected]:[email protected]

http://tiny.cc/splunkmlapp

Page 61: Using the SplunkMachine Learning Toolkit to Create … Create Your Own Custom Models ... • Multi-algorithm support across ... – Extending SPL with Custom Search Commands and the

THANKYOU


Related Documents
How to Create a Custom SEO Dashboard

How to Create a Custom SEO Dashboard

Category: Marketing
How to Create a Custom Profile with the Advanced …...How to Create a Custom Profile with the Advanced Cable Tester v2 The Advanced Cable Tester v2 features an option to create custom

How to Create a Custom Profile with the Advanced …...How.....

Category: Documents
UCS Director Creating and Utilizing Custom Tasks and · •Cisco UCS Director –Creating and Utilizing Custom Tasks and Script Modules • Create a Script Module • Create a Custom

UCS Director Creating and Utilizing Custom Tasks and ·...

Category: Documents
Create a Custom Template.odt

Create a Custom Template.odt

Category: Documents
How to Create Custom Signatures in Network Malware Cleaner · 2008. 7. 18. · Adding File Exist Option To create a custom signature: To create a custom signature in EMCO Network

How to Create Custom Signatures in Network Malware...

Category: Documents
Create Custom Metrics for HANA Monitoring - SAP

Create Custom Metrics for HANA Monitoring - SAP

Category: Documents
How to Create a Custom Snapchat Geofilter

How to Create a Custom Snapchat Geofilter

Category: Social Media
Create Custom Function and Operation for Payroll

Create Custom Function and Operation for Payroll

Category: Documents
custom & commercial… · custom & commercial. Create your own custom fire pit, burner, or outdoor kitchen island built to order. You create the dream, we create the reality. Call

custom & commercial… · custom & commercial. Create your....

Category: Documents
How to Create Custom Shirt Tags

How to Create Custom Shirt Tags

Category: Small Business & Entrepreneurship
How to create a custom dashboard in SNMPc OnLine 2007 to create a custom...Title Microsoft Word - How to create a custom dashboard in SNMPc OnLine 2007.docx Author User Created Date

How to create a custom dashboard in SNMPc OnLine 2007 to...

Category: Documents
create a custom theme websphere portal 8.0.0.1

create a custom theme websphere portal 8.0.0.1

Category: Documents