Architectural Solutions for Next Generation Software Systems Presenter: Faheem Ullah & Nguyen K. Tran PhD Students Supervisor: M. Ali Babar CREST – The Centre for Research on Engineering Software Technologies The University of Adelaide, Australia [email protected], [email protected]
27
Embed
Architectural Solutions for Next Generation Software Systemschristoph/seschool/... · Faheem Ullah & Nguyen Tran Architectural Solutions for Next Generation Software Systems CREST
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Architectural Solutions for
Next Generation Software Systems
Presenter: Faheem Ullah & Nguyen K. TranPhD Students
Supervisor: M. Ali Babar
CREST – The Centre for Research on Engineering Software Technologies The University of Adelaide, Australia
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Outline
• Architecting for Big Data Cybersecurity Analytics– How software architecture enables the achievement of the quality of
service delivered by Big Data Cybersecurity Analytics Systems
• An Architectural Solution for Internet of Things Search Engines– How software architecture enables Search Engine Systems for the Future
Internet and research on these systems.
3
ARCHITECTING FOR BIG DATA CYBERSECURITY ANALYTICS
How software architecture enables the achievement of the quality of service delivered by Big Data Cybersecurity Analytics Systems
4
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Outline
▪ Introduction
▪ Architectural Tactics for Big Data Cybersecurity Analytics: A Systematic Literature Review
▪ Towards Evidence-Based Understanding of Architectural Tactics for Cybersecurity Analytics
▪ Architecture-Driven Self-Adaptation for Cybersecurity Analytics
5
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Big Data Cybersecurity Analytics
Intrusion Detection System (IDS)
Security Information and
Event Management (SIEM)
Big Data Analytics for Cyber Security
6
Big Data Working Group, “Big Data Analytics for Security Intelligence”, Sept 2013
A research domain that leverage big data technologies for analysing security events data to protect organizational networks, computers, and data from cyber attacks
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Architectural Tactics for Big Data Cybersecurity Analytics: An SLR
7
Research Questions
RQ1: Which are the most important quality attributes for security analytic systems?
RQ2: What are the architectural tactics for addressing quality concerns in security analytic systems?
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Quality Attributes
8
0 20 40 60 80
Performance
Accuracy
Scalability
Reliability
Usability
Interoperability
Adaptability
Modifiability
Generality
Privacy assurance
Security
Stealthiness
Number of papers
Qu
alit
y at
trib
ute
Performance
Accuracy
Scalability
Reliability
Usability
Interoperability
Adaptivity
Modifiability
Privacy
Realtime response required to attacks
Size and speed of security event data hinders real-time response
Catastrophic consequences of letting attack go undetected
Only detect attacks and not shield legitimate access
Challenging to estimate the speed and size of security data
Long period attacks such as Advanced Persistent Threats
High speed security data input can crash the resources
Reliable data collection for ensuring attack detection
Unfriendly system can lead to delay in response to attack
Large number of alerts generated by the system
Collaborate with other security systems i.e., security orchestration
Data collection from a variety of sources
Adapt to comply with the required Quality of Service
Comply with the privacy laws while analysing the data
Qu
alit
y A
ttri
bu
tes
Avoid processing content of a packet
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Architectural Tactics
9
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Future Research Areas
10
Under addressed Quality Attributes
Tactics Evaluation
Quality Trade-offs among tactics
Dependencies among tactics
Modelling the tactics
▪ Several quality attributes such as interoperability, adaptivity, modifiability, generality, and stealthines requires further investigation from the architectural perspective
▪ The codified tactics should be evaluated both qualitatively and quantitatively to investigate their impact on various quality attributes
▪ The quality trade-offs among the tactics should be established to help a software architect select the required set of tactics
▪ Considering that the tactics cannot be applied in isolation, it is important to explore the possible dependencies and collaborations among the codified set of tactics
▪ To facilitate the software architect, the codified tactics need to be modelling using a standard modelling language such as UML
Futu
re R
esea
rch
Are
as
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Validation of Tactics
11
Data
Sources
Applications
Data
Collection
Removal of
DuplicatesFeature
SelectionFeature
Extraction
Model
Training
Data
Collection
Feature
Selection
Feature
Extraction
Attack
Detection
Model Alerts
Alert
Ranking
Visualization
Network
Databases
UserTraining phase
Testing phase
Removal of
Duplicates Tactic
Feature Selection and
Extraction Tactic
Alert Ranking
Tactic
collects security
event data removes duplicated
records
selects specific
features
extracts the
selected features
trains the
model
collects security event data for
testing the trained model
selects specific
features from the data
extracts the
selected features
tests the
model
generates the
alertsranks the
generated alerts
visualizes
the alerts
user responds
to alerts
Legend
Tactic
Phase
Model
Component
Data
Sources
1 2 3 4 5
6 7 8
910 11
12
13
Motivation
• Establishing quality trade-offs among the tactics• Developing evidence-based design space• Quantification of the contribution of tactics to
the intended quality attributes
Research Questions
What is the impact of the
RQ1: Removal of Duplicates TacticRQ2: Feature Selection and Extraction TacticRQ3: Alert Ranking Tactic
on the accuracy and response time of a security analytics system
An illustration tactics applied in the system
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Empirical Findings
12
Without the Tactic With the Tactic
Tactic Quality
Attribute
Matric Mean Median Mean Median Wilcoxon’s P
Removal of
Duplicates
Accuracy DR 91.532 91.533 92.625 92.627 -2.611
FPR 29.632 29.621 7.372 7.373 -2.611
Response
Time
TrT 2090.2 2087 1101.6 1100 -3.521
PrT 19.6 19.0 19.5 19.6 0.522
Feature
Selection and
Extraction
Accuracy DR 91.403 91.404 78.54 78.54 0.000137
FPR 6.706 6.706 13.863 13.861 0.000086
Response
Time
TrT 632 625 538.2 540 0.000173
PrT 21 21 20.6 20.8 0.000135
Alert Ranking Response
Time
TrT 632 625 630.2 625 0.5655
PrT 21 21 41.6 42 0.000055
Removal of Duplicates Tactic improves DR by 1.11%, reduces FPR by 22.26%, and improves TrT by 89.74%
Feature Selection and Extraction Tactic reduces DR by 12.86%, increases FPR by 7.15%, and improves TrT and
PrT by 17.43% and 1.93%.
Alert Ranking Tactic improves usability, which leads to enhanced accuracy, but increases PrT by 98.11%.DR – Detection Rate
FPR – False Positive RateTrT – Training TimePrT – Prediction Time
RQ1
RQ2
RQ3
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Architecture-Driven Self-Adaptation for Security Analytics
13
Operating Environment
Visualization
Removal of Duplicates
Data CutOff
Static Feature Selection
Dynamic Feature Selection
ML Algorithm Selection
Hadoop Booster
MapReduce Job Adjustment
MinMaxNormalization
ML Algorithm Application
Signature-based Detection
Alert Correlation
False Positive Reduction
Alert Ranking
Result Polling
Adaptation Analysis
Adaptation Model
Adaptation Application
Data Source
s
Security Analytics
Self-Adaptation
Network
Database
Application
Dashboard
Report
Email Notification
Visualization
Motivation
• Accuracy and response time are the most significant quality attributes
• Increasing accuracy reduces response time and vice versa
Architectural Solutions for Next Generation Software SystemsFaheem Ullah & Nguyen Tran CREST
Architecture-Driven Self-Adaptation for Security Analytics