Analytical Results of a Cyber Threat Intelligence Survey

Analytical Results of a Cyber Threat Intelligence Survey

Ryan Trost

© 2017 ThreatQuotient

whoami()

© 2017 ThreatQuotient 2

• Ryan Trost, Co-Founder of ThreatQuotient• “…career SOC-dweller” - sysAdmin > security analyst > IR > SOC Mgr• SOC Ops Manager - General Dynamics & several USG• Author of “Practical Intrusion Analysis” © 2009• Developed a geospatial intrusion detection model• Security Conference lectures include

• DEFCON16, SANS, BlackHat 2014, ISACA ISRM, InfoSec World • Chairman, Technical Advisory Board – Cyber Security AAS Collegiate

program

DISCLAIMER


The views and opinions expressed in this presentation are those of the author and not of my Employer.

Early vendor comparison triggered my fascination…

© 2017 ThreatQuotient 4 Cite: Trost, Ryan: US Blackhat 2014

Survey Purpose


Commercial Intel Providers lean on various requirements before publishing datapoints – what dictates those requirements?

• DEADEND question as commercial providers won’t tell you

Flip the curiosity on its head by posing the question to the industry

• What IOC Types and supporting Attributes pose the most value/benefit?

Methodology• Identify the top ~20 IOC Types across intel providers

• Identify the top 35 TTPs [read: attributes] across intel providers

• Design a questionnaire long enough to have stability but short enough where swamped analysts will actually complete it…and speak to you again!


CIDR FQDN MD5Hash SHA-512Hash User-AgentEmailAddress FuzzyHash ServiceName RegistryKey X.509S/NEmailSubject IPAddress SHA-1Hash URL X.509Subject

Filename Mutex SHA-256Hash URLPath

ASN Role CompileTime Motivation TargetedIndustry CNCNameFileSize FirstSeen DomainType Intent TargetedGeography MalwareNamePacker LastSeen EmailAddressType Langauge MalwareFamily MalwareCategoryPort SourceofInformation IPAddressType AdversaryGroup Vector Geolocation

Protocol Confidence Status CVE AttackCategory CVSSAttackCountry

Origin Threat/RiskScore Severity Impact BotName

Rating Scale – IOC TYPE• Evaluate each IOC Type based on 3 characteristics

• Strength – can it stand alone?• Deployment Versatility – how many detection technologies can it be

deployed?• Burnability – how easy is it for the adversary to replenish/re-create?

• Scale 1-5 (5 = most valuable)• 19 IOC Types * 3 scores = 57 answers…a big ask of the participant

Calculate AVERAGES and results in a fascinating multi-tier prioritization


Rating Scale - TTP• TTP needed to be easier/faster – in fear the analyst wouldn’t finish

the survey!• Assess each TTP

1. No Value2. Poor Value3. Good Value4. Great Value

• A 4-option scale was strategic so participants could NOT be indifferent – and select the ‘middle’ option


Participant Breakdown


Security Analyst 258 Hunter 36

Incident Response 124 Malware 34

IntelligenceAnalyst 94 Other 19

Security Analyst 46% Hunter 6%

Incident Response 22% Malware 6%

IntelligenceAnalyst 17% Other 3%


IOC Type Results Analysis

10

Overall Results


IOC Type Results - Overall


IOC Type Result by Category


IOCTypeStrengthOrder IOCTypeStrength

SHA-512Hash 4.20X.509SerialNumber 4.09

MD5Hash 4.01SHA-256Hash 4.00

FQDN 3.74RegistryKey 3.71SHA-1Hash 3.57

X.509Subject 3.52Mutex 3.47

URL 3.36User-Agent 3.36URLPath 3.19

ServiceName 3.18IPAddress 3.04

EmailAddress 3.04FuzzyHash 2.93Filename 2.56

EmailSubject 2.54CIDR 2.25

DeploymentOrder DeploymentVersatility

IPAddress 4.29URL 3.91

FQDN 3.81MD5Hash 3.47

SHA-256Hash 3.38URLPath 3.37

SHA-512Hash 3.36EmailAddress 2.99

SHA-1Hash 2.97RegistryKey 2.88

Filename 2.82EmailSubject 2.81User-Agent 2.78

Mutex 2.65ServiceName 2.52

FuzzyHash 2.39CIDR 2.32

X.509SerialNumber 2.18X.509Subject 2.00

Burnability Order Burn-ability

X.509SerialNumber 4.02X.509Subject 3.45RegistryKey 3.29

SHA-512Hash 3.28SHA-256Hash 3.13

MD5Hash 3.07User-Agent 3.05SHA-1Hash 3.02

Mutex 3.00FQDN 2.83

ServiceName 2.68IPAddress 2.56URLPath 2.55

URL 2.52EmailAddress 2.52

FuzzyHash 2.30CIDR 2.29

EmailSubject 2.27Filename 2.15


Attribute Results Analysis

14

List of TTPs/Attributes


Attributes Results



Security Analyst Results Breakdown

17

Security Analyst Results


SecAnalyst – Results & Observations


Observations:- Interestingseveralhost-basedhashIOCsrankedso

high- Maybede-sensitizedbynumberoffalse

positivesfromIP/FQDN/URL/etc.?- Deltascore[2.59]betweenthehighestandlowest

averageamongstthevariousIOCtypesisthehighestspreadacrossthevariousroles

- A.27differencebetween#1[4.04]and#2[3.77]isahugegapcomparatively

- InterestingX.509Subjectwassohigh(#3);thehighestpositionanotherrolehaditwas#10

- Deployment– IPAddressyieldedthehighestscoreinthesurveyw/4.89

SecAnalyst – IOC Type Breakdown


IOCTypeStrength

X.509SerialNumber 4.82SHA-512Hash 4.65SHA-256Hash 4.56

MD5Hash 4.50SHA-1Hash 4.15

X.509Subject 4.11User-Agent 3.93RegistryKey 3.54

FQDN 3.51URL 3.51

ServiceName 3.41URLPath 3.28

Mutex 3.05IPAddress 2.73

EmailSubject 2.65EmailAddress 2.52

Filename 2.39FuzzyHash 2.12

CIDR 1.56

DeploymentVersatility

IPAddress 4.89SHA-256Hash 3.95SHA-512Hash 3.92SHA-1Hash 3.86

FQDN 3.84URL 3.78

RegistryKey 3.61URLPath 3.48

Mutex 3.16Filename 3.12

EmailSubject 3.02User-Agent 2.89

X.509Subject 2.75EmailAddress 2.63

MD5Hash 2.56X.509SerialNumber 2.48

ServiceName 2.21FuzzyHash 1.92

CIDR 1.47

Burn-ability

X.509SerialNumber 4.82X.509Subject 4.38User-Agent 3.24RegistryKey 3.21SHA-1Hash 2.78

SHA-512Hash 2.75SHA-256Hash 2.70ServiceName 2.58

FQDN 2.53MD5Hash 2.50

Mutex 2.34FuzzyHash 2.23IPAddress 2.18

URL 2.18URLPath 2.16Filename 2.10

EmailAddress 1.92EmailSubject 1.57

CIDR 1.32

SecAnalyst – IOC-centric Breakdown


Observations withinthisattributecategory:

- Rolewassuperior(65%)forGreatValue

- SourceofInformation(79%)forGoodValue

- Domain/EmailAddress/IPTypealsodemonstratedconsistentconsensusamongstSecAnalysts

- CompileTimereceivedthemostpushback(50%)forNoValue

SecurityAnalyst NoValue PoorValue GoodValue GreatValueASN 38% 39% 21% 2%

FileSize 17% 31% 48% 4% Packer 9% 36% 52% 3% Port 14% 33% 50% 3%

Protocol 25% 47% 22% 7% AttackCountryOrigin 28% 10% 55% 7%

Role 0% 5% 30% 65% FirstSeen 6% 8% 55% 31% LastSeen 5% 6% 58% 31%

SourceofInformation 5% 3% 79% 14% Confidence 12% 36% 28% 24%

Threat/RiskScore 10% 34% 34% 22% CompileTime 50% 27% 18% 5% DomainType 0% 5% 65% 30%

EmailAddressType 5% 9% 67% 20% IPAddressType 0% 4% 67% 29%

Status 9% 3% 59% 28% Severity 9% 29% 47% 15%

SecAnalyst – Adversary-centric Breakdown


SecurityAnalyst NoValue PoorValue GoodValue GreatValueMotivation 10% 28% 42% 19%

Intent 10% 33% 44% 12% Langauge 7% 25% 52% 16%

AdversaryGroup 9% 5% 67% 19%


- OverallaprettyboringsplitacrossAdversary-centricattributes

SecAnalyst – Attack-centric Breakdown


SecurityAnalyst NoValue PoorValue GoodValue GreatValueCVE 10% 27% 46% 17%

Impact 6% 21% 58% 16% TargetedIndustry 5% 11% 37% 48%

TargetedGeography 36% 30% 17% 16% MalwareFamily 7% 5% 59% 30%

Vector 1% 5% 86% 9% AttackCategory 0% 5% 55% 40%

BotName 3% 6% 53% 38% CNCName 2% 8% 50% 40%

MalwareName 6% 3% 61% 29% MalwareCategory 0% 3% 58% 39%

Geolocation 19% 59% 14% 8% CVSS 46% 26% 20% 8%


- Vector (86%)dominatedtheresultswithaGoodValue

- TargetedGeographyandCVSSreceivedthemostpushback(36%)and(46%)respectivelyforNoValue

SecAnalyst – Attribute Analysis


SecurityAnalyst NoValue PoorValue GoodValue GreatValue

AttributeBreakdownObservation:- re:GreatValuescoresSecAnalysts leantowardsAttack-centricTTPsvs.IOC- or

Adversary-centric- re:Allothercategoriesareprettyevenlysplitacrossthesurveyparticipants

TotalAverage 12% 19% 48% 21%

IOC-CentricAverage 13% 21% 47% 19% Adversary-Centric Average 9% 23% 51% 17% Attack-Centric Average 11% 16% 47% 26%

…compare assessments within a category

TotalAverageObservation– SecurityAnalystpredominantlyleantowards“GoodValue”


Lessons Learned

25

Lessons LearnedParticipate breakdown by Role resulted in interesting data; however, should have asked

• # of years of experience!• Average size of team across work experience• Previous career path (i.e., 10 years as a security analyst and now

spearhead incident response, etc.)Get more friends who aren’t Security Analysts!


Questions?ryan . trost @ threatq . com


Analytical Results of a Cyber Threat Intelligence Survey

Documents