Page 1
SharingisCaring:UnderstandingandMeasuringThreatIntelligenceSharing
Effectiveness(#ddti)
SharingisCaring:UnderstandingandMeasuringThreatIntelligenceSharing
Effectiveness(#ddti)AlexPinto
ChiefDataScientistMLSec Project/Niddel
@alexcpsec@MLSecProject @NiddelCorp
Page 2
• Previouslyon#ddti• ChallengesatTISharing• MeasuringTISharing• TheFutureofSharing
AgendaAgenda
Page 3
Thisisadata-driventalk!Thisisadata-driventalk!Pleasecheckyouranecdotesatthedoor
Page 5
Previouslyon#ddtiPreviouslyon#ddti• UsefulMethodsandMeasurementsforHandlingIndicators• AnalysisofThreatIntelligenceFeeds• Indirectly,amethodologyforanalyzingTIProviders
• Combine(https://github.com/mlsecproject/combine)• GathersTIdata(ip/host)fromInternetandlocalfiles
• TIQ-Test(https://github.com/mlsecproject/tiq-test)• RunsstatisticalsummariesandtestsonTIfeeds
Page 6
TIQ-TEST- TonsofThreat-yTestsTIQ-TEST- TonsofThreat-yTests
• NOVELTY – Howoftendothefeedsupdatethemselves?• AGING – Howlongdoesanindicatorsitonafeed?• POPULATION – Howdoesthispopulationdistributioncomparetomydata?
• OVERLAP– Howdotheindicatorscomparetotheonesyougot?
• UNIQUENESS – Howmanyindicatorsarefoundonlyononefeed?
Puttingthisthreatinteldatatowork
Page 7
OverlapTest- OutboundOverlapTest- Outbound
Page 9
Ihatequotingmyself,but…Ihatequotingmyself,but…
Page 10
KeyTakeaway#1KeyTakeaway#1
MORE!=BETTERThreatIntelligenceIndicatorFeeds
ThreatIntelligenceProgram
Page 11
ConstructiveFeedbackfromtheInternet:
“TISharingisTOTALLYgoingtosolvethis”
ConstructiveFeedbackfromtheInternet:
“TISharingisTOTALLYgoingtosolvethis”
Right,folks?Right?
Page 12
TISharingSolutionPlan:TISharingSolutionPlan:
1. ThebestThreatIntelligenceistheonethatyouanalyzefromyourownincidents(homegrown/organicintelligence)
2. Thereisstrengthinnumbers– verticalherdimmunity!
3. ????????
4. PROFIT!!(oratleastSECURITY!!)
Oratleastaroughstrawman
Page 13
IfCONSUMINGisforthe1%,whatisthepercentageoforganizationsabletoPRODUCE?
Issue1- BYOTIIssue1- BYOTI
Page 14
Issue2- HerdImmunityIssue2- HerdImmunity
Source:www.vaccines.gov
• Wemaybeabletodetectmore”virusstrains”togetherbutweare*terrible*atinoculation.
• Thethingswedetectthemostmutatetoofast(PyramidofPain)
• Whodidn’tgetimmunized,stillgetssick(FOMO-TI)
Page 15
Issue?- WhatarewesharingIssue?- Whatarewesharing• AUTOMATION-DRIVEN(PLATFORMS)• StraighttothepointIOCsharing
• ANALYST-DRIVEN(COMMUNITIES)• Strategicdata,bestpractices,unstructuredIOCs
• ”Analyst-driven”hasbeenaroundforever(innon-IC,atleastsinceFS-ISACwascreated)
• Thesamepeoplewhobash”justIOCsharing”:• BashSTIX/TAXIIfortryingtoencodecomplexity• TellseveryoneitisIMPOSSIBLEtohireanalysts
Page 16
TheCognitiveDissonancesofTISharingTheCognitiveDissonancesofTISharing
Everybody shouldshare! TheCIRCLEOFTRUST
Page 17
Doyoutrustthegroupenoughtoconsume?
TheTwoSidesoftheTrustCoinTheTwoSidesoftheTrustCoin
Doyoutrustthegroupenoughtoshare?
Page 18
Okay,I’llbiteOkay,I’llbite
Canwemeasureourcurrentsharingplatformscommunities?
Page 19
ThreatIntelligenceSharingThreatIntelligenceSharingWewouldliketothankthekindcontributionofdatafromthefinefolksatFacebookThreatExchange andThreatConnect
…andalsothesharingcommunitiesthatchosetoremainanonymous.Youknowwhoyouare,andwe❤ youtoo.
Page 20
SharingCommunitiesARESocialNetworksSharingCommunitiesARESocialNetworks
SocialNetworkSelfie SharingCommunitySelfie
Page 21
Let’slookattheindicatorsfirstLet’slookattheindicatorsfirst
UsingTIQ-TESTOverlapandUniquenesstests
Page 24
UNIQUENESSSLIDE
Lookslikewewouldgetsimilarqualityona”good”ThreatIntelligenceSharingPlatformaswewouldon
a”paidfeed"
Page 25
SuggestedMetricsforSharingSuggestedMetricsforSharing
• ACTIVITY – Howmanyindicators/postsarebeingshareddaybyday?
• DIVERSITY –Whatisthepercentageofthepopulationthatisactivelysharing?
• FEEDBACK – Areorgscollaboratingonimprovingtheknowledgeinthesharingenvironment?
• TRUST– Howmuchdataisshared”openly”inrelationto”privately”?
Lookingforhealthydynamics
Page 26
ActivityMetricActivityMetricIsthereanyactualsharinggoing
on?
Page 27
Lessdata/Delays Moredata/Timely
LargeGroupisroughly40xbiggerthanSmallGroup
Page 28
Organizationsarelesslikelytoshareiftheyperceivethey”lostcontrol”ofwhocanconsume.
Page 29
DiversityMetricDiversityMetricCheckyoursharingprivilege
Page 30
Roughly10%oftheorganizationssharedataintothecommunity
Page 31
Someorganizationsareclearlyinabetterpositionoperationallyandlegallytoshare.Andthatis
expectedduetoourpremises.
Page 32
FeedbackMetricFeedbackMetricButisthedataanygood?
Page 34
🙀 I’msurewecandobetterthanthis🙀
Page 35
FeedbackMetricFeedbackMetric• Almostnosupportonautomation-drivenplatforms• Someallowyoutoleave”comments”or”newdescriptors”fortheIOCs– evenbycountingthoseverylow%inrelationtonewshareddata
• Analyst-drivenenvironmentsallowforcollaborationone-mailsandforumpoststodescribeandrefinestrategiesandbestpractices.
Howcanwemakethiscollaborationworkonautomation-drivenplatforms?
Page 36
TrustMetricTrustMetricArewehelpingallthecommunity
orjustafeworgsatatime?
Page 38
76%.Again,soundsaboutright
Page 39
Overall”quality”ofdatagoesuptoo!
Page 40
TrustMetricTrustMetric• Theroughestimateseemstobethatmorethan80%of”sharing”(IOCs,messages,etc)happensin”privategroups”insidetheinfrastructureofthesharingplatform
• Allcommunitieshavethem:• PartoftheDNAoftheIC/clearedcommunity• Offsetsthetrustequation,butdefeatsthe”herdimmunity”argument• UsuallyMANDATORYoncollaborationwithLEA
Butthenthe”good”dataisnothelping”thecommunity”!Isthereanywaywecanreconcile?
Page 41
TheFutureofSharing🔮TheFutureofSharing🔮Attheveryleastmyhumble
opinion
Page 42
#squadgoals#squadgoalsIncreasetheTRUST
amongpeers
ReducetheTECHNICALBARRIERforsharinguseful
information
Page 43
TRUST:ReputationandAnonymityTRUST:ReputationandAnonymity
Page 44
AlienVault OTXclearlygotthememoAlienVault OTXclearlygotthememo
Page 45
TRUST:Anonymity+GoodCurationTRUST:Anonymity+GoodCuration
Somesharingcommunitiesacceptanonymoussubmissionsthattheythencurateanddisseminate
toallorganizations
Page 46
IOCs
Feedback
TelemetryLESSMATURE
MOREMATURE
With❤ andapologiesto@DavidJBiancoWith❤ andapologiesto@DavidJBianco
TECHNICALBARRIER:”PyramidofSharing”TECHNICALBARRIER:”PyramidofSharing”
Page 47
TakeawaysTakeaways• IntelligenceSharingisaveryanalyst-centricactivitythatwehavebeentaskedwithscalingoutwithautomation.Nowonderitseemssohard.
• Datacanbeasgoodasapaidfeed,butyouhavetobeintherightcirclesoftrust
• Doesnotsolveanalystshortageandmakingtheindicators/strategiesoperationalintoyourenvironment
Page 49
Thanks!Thanks!
• Q&A?• Feedback!
”Themeasureofintelligenceistheabilitytochange."- AlbertEinstein
AlexPinto@alexcpsec
@MLSecProject /@NiddelCorp