Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness Fabian Fischer and Daniel A. Keim Symposium on Visualization for Cyber Security (VizSec 2014) 10th November 2014, Paris, France
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz
NStreamAware: Real-Time Visual Analytics forData Streams to Enhance Situational AwarenessFabian Fischer and Daniel A. Keim
Symposium on Visualization for Cyber Security (VizSec 2014)10th November 2014, Paris, France
2Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Motivation: Heterogeneous Data Streams
• Network Alerts (e.g., OSSEC)
• Syslog Messages
• NetFlow Data
Analyzing Data Streams =Crucial for security in your network!
Monitoring & Exploration
Crucial for situational awareness (SA)!
REAL-TIME
VISUAL
ANALYTICS
DATA CHALLENGE
How to make streamanalysis scalable?
4Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
NStreamAware: Infrastructure
Web
Ap
plic
atio
n(N
Vis
Aw
are)
REST REST Service
(VACS-REST)
Dat
a St
ream
s(f
rom
vari
ou
sso
urc
es)
MongoDB ElasticSearch
Scalable
SPARK Service(VACS-Spark)
Distributed Streaming Analytics
Apache Spark™ is a fast and general engine for large-scale data processing which can run on a distributed computer cluster.
5Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Integrated Perspectives
• Real-Time Data Stream Monitoring
• Real-Time Sliding Slices (NVisAware)
• Visual Feature Selection
• Summarized Sliding Slices
• Event Timeline & Insights
• Search & Exploration
6Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Real-Time Data Stream Monitoring
Demo
SITUATIONAL AWARENESS CHALLENGE
How to reduce thecognitive load?
10Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
NVisAware: Analytics
Web
Ap
plic
atio
n(N
Vis
Aw
are)
REST REST Service
(VACS-REST)
Dat
a St
ream
s(f
rom
vari
ou
sso
urc
es)
MongoDB ElasticSearch
SPARK Service(VACS-Spark)
Distributed Streaming Analytics
Visual Analytics Approach:Calculate and visualize sliding slices.
(based on sliding windows)
• Calculate Sliding Slice Summaryfor each sliding window.
• Push slicet to web application.
11Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
slicet
Real-Time Sliding Slice• Interactive Widgets
– Treemaps
– Counters
– Node-link diagrams
• Interactions– Star/Annotate slice
– Remove slice
– Retrieve data
• Color Encoding– Background for similarity
– Importance of alerts
Demo
EXPLORATION CHALLENGE
How to exploremany sliding slices?
18Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Visual Feature Selection Visual Analytics Approach:Aggregate / Summarize
according interest function(visually steered by the expert)
19Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Example: Using Visual Analytics for Interactive Summarization
… …
Demo
22Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Application to Real-Time Social Media Analysis (VAST Challenge 2014 MC3)
• Real-Time Monitoring Task:Discover major events in the streamto support an ongoing police operation.
• Available Data Stream:Real-time feeds of microblogsand emergency calls.
• Successful participation:“Award for Outstanding ComprehensiveMini-Challenge 3 Submission”
23Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Further Challenges and Future Work
• Challenge: Parameter adjustmentfor sliding slices and clustering.
• Automated merging of sliding slicesbased on the interest function.
• Performance Evaluation for a large networkusing security operational data stream.
• Responsiveness issueswhen increasing the number of complex interactive visualizations.
• Data retention and rotationfor the visualization interface.
24Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Contributions
• NStreamAware – Building a web-based visualanalytics system using scalable technologies.
• NVisAware – Sliding Slices Visualizationwith embedded visualization widgets.
• NVisAware – Summarized Sliding Slicessteered using interactive visualizations.
EXPLORATION CHALLENGE
How to explore manysliding slices?
SA CHALLENGE
How to reduce thecognitive load?
DATA CHALLENGE
How to make stream analysis scalable?
25Fabian Fischer | NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness
Thank you very much for your attention!
Questions?
For more informationabout this work please contact
Fabian FischerTel. +49 7531 88-2780
http://ff.cx/
@f2cx
Related Documents
![NStreamAware: Real-Time Visual Analytics for Data Streams to Enhance Situational Awareness · 2019. 10. 21. · of Event Visualizer [8], is to provide real-time visualizations for](https://static.cupdf.com/doc/110x72/609f55b3ecbd2346046c148d/nstreamaware-real-time-visual-analytics-for-data-streams-to-enhance-situational.jpg)
