Sharing is Caring: Understanding and Measuring Threat ... · Sharing is Caring: Understanding and Measuring Threat Intelligence Sharing Effectiveness (#ddti) Alex Pinto Chief Data

SharingisCaring:UnderstandingandMeasuringThreatIntelligenceSharing

Effectiveness(#ddti)

SharingisCaring:UnderstandingandMeasuringThreatIntelligenceSharing

Effectiveness(#ddti)AlexPinto

ChiefDataScientistMLSec Project/Niddel

@alexcpsec@MLSecProject @NiddelCorp

• Previouslyon#ddti• ChallengesatTISharing• MeasuringTISharing• TheFutureofSharing

AgendaAgenda

Thisisadata-driventalk!Thisisadata-driventalk!Pleasecheckyouranecdotesatthedoor

Previouslyon#ddtiPreviouslyon#ddti• UsefulMethodsandMeasurementsforHandlingIndicators• AnalysisofThreatIntelligenceFeeds• Indirectly,amethodologyforanalyzingTIProviders

• Combine(https://github.com/mlsecproject/combine)• GathersTIdata(ip/host)fromInternetandlocalfiles

• TIQ-Test(https://github.com/mlsecproject/tiq-test)• RunsstatisticalsummariesandtestsonTIfeeds

TIQ-TEST- TonsofThreat-yTestsTIQ-TEST- TonsofThreat-yTests

• NOVELTY – Howoftendothefeedsupdatethemselves?• AGING – Howlongdoesanindicatorsitonafeed?• POPULATION – Howdoesthispopulationdistributioncomparetomydata?

• OVERLAP– Howdotheindicatorscomparetotheonesyougot?

• UNIQUENESS – Howmanyindicatorsarefoundonlyononefeed?

Puttingthisthreatinteldatatowork

OverlapTest- OutboundOverlapTest- Outbound

Ihatequotingmyself,but…Ihatequotingmyself,but…

KeyTakeaway#1KeyTakeaway#1

MORE!=BETTERThreatIntelligenceIndicatorFeeds

ThreatIntelligenceProgram

ConstructiveFeedbackfromtheInternet:

“TISharingisTOTALLYgoingtosolvethis”

ConstructiveFeedbackfromtheInternet:

“TISharingisTOTALLYgoingtosolvethis”

Right,folks?Right?

TISharingSolutionPlan:TISharingSolutionPlan:

1. ThebestThreatIntelligenceistheonethatyouanalyzefromyourownincidents(homegrown/organicintelligence)

2. Thereisstrengthinnumbers– verticalherdimmunity!

3. ????????

4. PROFIT!!(oratleastSECURITY!!)

Oratleastaroughstrawman

IfCONSUMINGisforthe1%,whatisthepercentageoforganizationsabletoPRODUCE?

Issue1- BYOTIIssue1- BYOTI

Issue2- HerdImmunityIssue2- HerdImmunity

Source:www.vaccines.gov

• Wemaybeabletodetectmore”virusstrains”togetherbutweare*terrible*atinoculation.

• Thethingswedetectthemostmutatetoofast(PyramidofPain)

• Whodidn’tgetimmunized,stillgetssick(FOMO-TI)

Issue?- WhatarewesharingIssue?- Whatarewesharing• AUTOMATION-DRIVEN(PLATFORMS)• StraighttothepointIOCsharing

• ANALYST-DRIVEN(COMMUNITIES)• Strategicdata,bestpractices,unstructuredIOCs

• ”Analyst-driven”hasbeenaroundforever(innon-IC,atleastsinceFS-ISACwascreated)

• Thesamepeoplewhobash”justIOCsharing”:• BashSTIX/TAXIIfortryingtoencodecomplexity• TellseveryoneitisIMPOSSIBLEtohireanalysts

TheCognitiveDissonancesofTISharingTheCognitiveDissonancesofTISharing

Everybody shouldshare! TheCIRCLEOFTRUST

Doyoutrustthegroupenoughtoconsume?

TheTwoSidesoftheTrustCoinTheTwoSidesoftheTrustCoin

Doyoutrustthegroupenoughtoshare?

Okay,I’llbiteOkay,I’llbite

Canwemeasureourcurrentsharingplatformscommunities?

ThreatIntelligenceSharingThreatIntelligenceSharingWewouldliketothankthekindcontributionofdatafromthefinefolksatFacebookThreatExchange andThreatConnect

…andalsothesharingcommunitiesthatchosetoremainanonymous.Youknowwhoyouare,andwe❤ youtoo.

SharingCommunitiesARESocialNetworksSharingCommunitiesARESocialNetworks

SocialNetworkSelfie SharingCommunitySelfie

Let’slookattheindicatorsfirstLet’slookattheindicatorsfirst

UsingTIQ-TESTOverlapandUniquenesstests

OVERLAPSLIDE

OVERLAPSLIDE

UNIQUENESSSLIDE

Lookslikewewouldgetsimilarqualityona”good”ThreatIntelligenceSharingPlatformaswewouldon

a”paidfeed"

SuggestedMetricsforSharingSuggestedMetricsforSharing

• ACTIVITY – Howmanyindicators/postsarebeingshareddaybyday?

• DIVERSITY –Whatisthepercentageofthepopulationthatisactivelysharing?

• FEEDBACK – Areorgscollaboratingonimprovingtheknowledgeinthesharingenvironment?

• TRUST– Howmuchdataisshared”openly”inrelationto”privately”?

Lookingforhealthydynamics

ActivityMetricActivityMetricIsthereanyactualsharinggoing

on?

Lessdata/Delays Moredata/Timely

LargeGroupisroughly40xbiggerthanSmallGroup

Organizationsarelesslikelytoshareiftheyperceivethey”lostcontrol”ofwhocanconsume.

DiversityMetricDiversityMetricCheckyoursharingprivilege

Roughly10%oftheorganizationssharedataintothecommunity

Someorganizationsareclearlyinabetterpositionoperationallyandlegallytoshare.Andthatis

expectedduetoourpremises.

FeedbackMetricFeedbackMetricButisthedataanygood?

🙀 I’msurewecandobetterthanthis🙀

FeedbackMetricFeedbackMetric• Almostnosupportonautomation-drivenplatforms• Someallowyoutoleave”comments”or”newdescriptors”fortheIOCs– evenbycountingthoseverylow%inrelationtonewshareddata

• Analyst-drivenenvironmentsallowforcollaborationone-mailsandforumpoststodescribeandrefinestrategiesandbestpractices.

Howcanwemakethiscollaborationworkonautomation-drivenplatforms?

TrustMetricTrustMetricArewehelpingallthecommunity

orjustafeworgsatatime?

76%.Again,soundsaboutright

Overall”quality”ofdatagoesuptoo!

TrustMetricTrustMetric• Theroughestimateseemstobethatmorethan80%of”sharing”(IOCs,messages,etc)happensin”privategroups”insidetheinfrastructureofthesharingplatform

• Allcommunitieshavethem:• PartoftheDNAoftheIC/clearedcommunity• Offsetsthetrustequation,butdefeatsthe”herdimmunity”argument• UsuallyMANDATORYoncollaborationwithLEA

Butthenthe”good”dataisnothelping”thecommunity”!Isthereanywaywecanreconcile?

TheFutureofSharing🔮TheFutureofSharing🔮Attheveryleastmyhumble

opinion

#squadgoals#squadgoalsIncreasetheTRUST

amongpeers

ReducetheTECHNICALBARRIERforsharinguseful

information

TRUST:ReputationandAnonymityTRUST:ReputationandAnonymity

AlienVault OTXclearlygotthememoAlienVault OTXclearlygotthememo

TRUST:Anonymity+GoodCurationTRUST:Anonymity+GoodCuration

Somesharingcommunitiesacceptanonymoussubmissionsthattheythencurateanddisseminate

toallorganizations

IOCs

Feedback

TelemetryLESSMATURE

MOREMATURE

With❤ andapologiesto@DavidJBiancoWith❤ andapologiesto@DavidJBianco

TECHNICALBARRIER:”PyramidofSharing”TECHNICALBARRIER:”PyramidofSharing”

TakeawaysTakeaways• IntelligenceSharingisaveryanalyst-centricactivitythatwehavebeentaskedwithscalingoutwithautomation.Nowonderitseemssohard.

• Datacanbeasgoodasapaidfeed,butyouhavetobeintherightcirclesoftrust

• Doesnotsolveanalystshortageandmakingtheindicators/strategiesoperationalintoyourenvironment

Thanks!Thanks!

• Q&A?• Feedback!

”Themeasureofintelligenceistheabilitytochange."- AlbertEinstein

AlexPinto@alexcpsec

@MLSecProject /@NiddelCorp