IT Data Visualization Raffael Marty, GCIA, CISSP Chief Security Strategist @ Splunk> SUMIT, Michigan - October ‘08
Aug 20, 2015
IT Data Visualization
Raffael Marty GCIA CISSPChief Security Strategist Splunkgt
SUMIT Michigan - October lsquo08
bull Chief Security Strategist Splunkgt
bull Looked at logsIT data for over 10 years
- IBM Research
- Conference boards committees
bull Presenting around the world on SecViz
bull Passion for Visualization
- httpsecvizorg
- httpafterglowsourceforgenet
Raffael Marty
Applied Security VisualizationPaperback 552 pages
Publisher Addison Wesley (August 2008)ISBN 0321510100
Agendabull IT Data Visualization
- Security Visualization Dichotomy
- Research Dichotomy
bull IT Data Management
- A shifted crime landscape
bull Perimeter Threat
bull Insider Threat
bull Security Visualization Community
3
Visualization is a more effective way of IT data management and
analysis
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
bull Chief Security Strategist Splunkgt
bull Looked at logsIT data for over 10 years
- IBM Research
- Conference boards committees
bull Presenting around the world on SecViz
bull Passion for Visualization
- httpsecvizorg
- httpafterglowsourceforgenet
Raffael Marty
Applied Security VisualizationPaperback 552 pages
Publisher Addison Wesley (August 2008)ISBN 0321510100
Agendabull IT Data Visualization
- Security Visualization Dichotomy
- Research Dichotomy
bull IT Data Management
- A shifted crime landscape
bull Perimeter Threat
bull Insider Threat
bull Security Visualization Community
3
Visualization is a more effective way of IT data management and
analysis
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Agendabull IT Data Visualization
- Security Visualization Dichotomy
- Research Dichotomy
bull IT Data Management
- A shifted crime landscape
bull Perimeter Threat
bull Insider Threat
bull Security Visualization Community
3
Visualization is a more effective way of IT data management and
analysis
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Visualization Questionsbull Who analyzes logs
bull Who uses visualization for log analysis
bull Who has used DAVIX
bull Have you heard of SecVizorg
bull What tools are you using for log analysis
4
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
IT Data Visualization
Applied Security Visualization Chapter 3
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
What is Visualization
6
A picture is worth a thousand log records
Generate a picture from IT data
Inspire
Pose a New Question
Explore and Discover
Support Decisions
Communicate Information
Increase Efficiency
Answer a Question
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Information Visualization Process
7
Capture Process Visualize
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The 1st Dichotomy
bull security databull networking protocolsbull routing protocols (the Internet)bull security impactbull security policybull jargonbull use-casesbull are the end-users
8
bull types of databull perceptionbull opticsbull color theorybull depth cue theorybull interaction theory bull types of graphsbull human computer interaction
two domainsSecurity amp Visualization
Security Visualization
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Failure - New Graphs
9
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Right Thing - Reuse Graphs
10
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Failure - The Wrong Graph
11
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Right Thing - Adequate Graphs
12
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Failure - The Wrong Integration
13
bull Using proprietary data formatbull Provide parsers for various data formats
bull does not scalebull is probably buggy incomplete
bull Use wrong data access paradigm bull complex configuration
eg needs an SSH connection
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
bull Keep It Simple Stupidbull Use CSV inputbull Use files as inputbull Offload to other tools
bull parsersbull data conversions
The Right Thing - KISS
14
Using node sizessizesource=1sizetarget=200maxNodeSize=02
usrsharemanman5launchdplist5ltxml version=10 encoding=UTF-8gtltDOCTYPE plist PUBLIC -Apple ComputerDTD PLIST 10EN httpwwwapplecomDTDsPropertyList-10dtdgtltplist version=10gtltdictgt ltkeygt_nameltkeygt ltdictgt ltkeygt_isColumnltkeygt ltstringgtYESltstringgt ltkeygt_isOutlineColumnltkeygt ltstringgtYESltstringgt ltkeygt_orderltkeygt ltstringgt0ltstringgt ltdictgt ltkeygtbsd_nameltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt62ltstringgt ltdictgt ltkeygtdetachable_driveltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt59ltstringgt ltdictgt ltkeygtdevice_manufacturerltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt41ltstringgt ltdictgt ltkeygtdevice_modelltkeygt ltdictgt ltkeygt_orderltkeygt ltstringgt42ltstringgt ltdictgt ltkeygtdevice_revisionltkeygt
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Failure - Unnecessary Ink
15
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Right Thing - Apply Good Visualization Practices
16
bull Dont use graphics to decorate a few numbersbull Reduce data ink ratiobull Visualization principles
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The 2nd Dichotomy
17
bull donrsquot understand the real impactbull get the 70 solutionbull donrsquot think bigbull no timemoney for real researchbull canrsquot scalebull work based off of a few
customerrsquos input
bull donrsquot know whatrsquos been done in industrybull donrsquot understand the use-casesbull donrsquot understand the environments
data domainbull work on simulated databull construct their own problems bull use overly complicated impractical
solutionsbull use graphs visualization where it is not
needed
Some comments are based on paper reviews from RAID 200708 VizSec 200708
Industry Academia
two worldsIndustry amp Academia
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The Way Forward
18
bull Building a secviz disciplinebull Bridging the gapbull Learning the ldquootherrdquo disciplinebull More academia industry collaboration
Security Visualization
SecViz
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
My Focus Areas
19
bull Use-case oriented visualizationbull IT data managementbull Perimeter Threatbull Governance Risk Compliance (GRC)bull Insider Threat bull IT data visualizationbull SecVizOrgbull DAVIX
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
IT Data Management
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
A Shifted Crime Landscapebull Crimes are moving up the stack
bull Insider crime
bull Large-scale spread of many small attacks
bull Are you prepared
bull Are you monitoring enough
21
Application Layer
Transport Layer
Network Layer
Link Layer
Physical Layer
Questions are not known in advance Have the data when you need it
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
The IT Search Company
Configurations
Change Events
Traps amp Alerts
Scripts amp Code
Logs
What Is IT Datavarlogmessagsoptlog
etcsyslogconfetchosts
1361212533122iso org dod internet mgmt mib-2 host hrDevice hrProcessorTable hrProcessorEntry hrProcessorLoad
psnetstat
File system changesWindows Registry
multi-line files
entire files
multi-line structures
multi-line table format
hooks into the OS
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Perimeter Threat
Applied Security Visualization Chapter 6
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Sparklines
24
bull Data-intense design-simple word-sized graphics
bull Examples- stock price over a day- access to port 80 over the last week
Edward Tufte (2006) Beautiful Evidence Graphics Press
Average Standard Deviation
bull Java Script Implementationhttpomnipotentnetjquerysparkline
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Sparklines
25
Port Source IP Destination IP
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Insider Threat
Applied Security Visualization Chapter 8
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Three Types of Insider Threats
27
Fraud InformationLeak
Sabotage
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Example - Insider Threat Visualization
bull More and other data sources than for the traditional security use-cases
bull Insiders often have legitimate access to machines and data You need to log more than the exceptions
bull Insider crimes are often executed on the application layer You need transaction data and chatty application logs
28
bull The questions are not known in advance bull Visualization provokes questions and
helps find answersbull Dynamic nature of fraud bull Problem for static algorithmsbull Bandits quickly adapt to fixed threshold-
based detection systemsbull Looking for any unusual patterns
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
User Activity
High ratio of failed logins
29
Color indicates failed logins
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
30
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Security VisualizationCommunity
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
SecViz - Security VisualizationThis is a place to share discuss challenge and learn about
security visualization
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Data Analysis and Visualization Linuxdavixsecvizorg
D
V
X
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
ToolsCapture
- Network tools
Argus
Snort
Wireshark
- Logging
syslog-ng
- Fetching data
wget
ftp
scp
Processing
- Shell tools
awk grep sed
- Graphic preprocessing
Afterglow
LGL
- Date enrichment
geoiplookup
whoisgwhois
Visualization
- Network Traffic
EtherApe
InetVis
tnv
- Generic
Afterglow
Treemap
Mondrian
R Project
Non-concluding list of tools
Thank You
raffy splunk com
Thank You
raffy splunk com