Cortex Program Status Brief Christopher Geib July 12th 2005
Jan 03, 2016
Cortex Program Status Brief
Christopher GeibJuly 12th 2005
2 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Self Regeneration is not enough
• Systems must be designed that are capable of response and regeneration after deliberate attack and catastrophic failures.
• However, response and regeneration must be sensitive to…- the mission that is being executed,
What tools and services are critical for the current mission goals? What tools and services are not critical for the current mission goals?
- and the lessons learned from the previous failure. What features of the protocol were exploited in the attack? Have features of the domain changed?
• Are there new kinds of connections that should be blocked?• Are some kinds of attacks more common than we thought?
Mission Aware Planning and Learning
3 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Program Technical Objectives
• Prove the viability of automatically synthesizing a meta-level controller for model-based, system response and regeneration.
• Such regeneration control systems must have two critical capabilities:- planning that is sensitive to the system mission,
How much of the systems resources should be committed or held in reserve?
Planning conditional responses to known threats. Trading off commitments for mission critical objectives.
- and learning to prevent similar failures in the future. Patch the services that were exploited Modify the mission model to capture changes to the world.
Meta Level Planning and Learning
4 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Existing Practice
• Automated system rebooting will bring elements of the system back on line after an attack, but these are limited:- Simple scripts that are not sensitive to the mission model.
- Can’t make trade offs between competing needs.
- Single system reboot.
• Learning of exploits and diagnosing root causes of the failure is handled offline by system experts.- Conclusions are not captured at the mission model level.
- Often performed somewhere else.
- Slow and requires significant expertise.
Done By Hand If At All
5 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Talk Outline
• What is the mission? - Mission model
- Conops for Cortex within this Model
• What is our technical approach?- Current system design and use cases
- Thrust areas Learning Planning
• What progress have we made?• How are we performing against our metrics?• Conclusions
What Is The Mission?
7 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Mission Model Motivation
• Critical Problem Features- Real world applicability and importance.
- Military relevance clear.
- Challenging complexity exceeds simple solutions.
- Multiple mission phases with differing objectives requiring differing responses to attack.
- Easy to find cyber attack methods within the literature.
• Daily mission planning cycle - Based on DARPA Cyber Panel Grand Challenge Problem
- Defense of an operations critical MySQL database for mission planning. Direct access Apache/php mediated access
8 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Daily Mission Cycle
1: High Level Planning by Commander
2: Detailed Planning by
Naval Operations
3: Scheduling & Task Orders by
Naval Operations
4: Task OrdersSent to Ships
5: ScheduledMaintenance
Ongoing: BattleDamage Assessment
from other ships
17:00-19:45 20:00-22:10
08:00-11:20
12:00-12:05
15:00-16:00
9 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Database and Web Server Host
Mission Domain System Architecture
Apache WebServer
PHPinterpreter
MySQLPlanning dB
StaticContent(.html)
DynamicContent(.php)
SQL
https
http
MySQL_BRP_Plan
radiolink
(Blueridge LAN)
(Planning LAN)
FW
10 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Cortex Role in this Domain
• Protect the MySQL database:- Guarantee availability- Guarantee data integrity
• Making use of:- Redundant “taste tester” architecture- CIRCA planning for response and resource use- Learning based on active experimentation
• Cortex control capabilities:- Control query replication.- Manage tasters (which is lead-taster, building new ones)- Control access to DB (block known exploits)- Invoke learning module
What Is Our Technical Approach?
12 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
New Technology
• What is truly new?- Automated synthesis of meta-level controllers based on
decision theoretic tradeoffs among mission requirements for response and regeneration.
- Active experimentation based learning to prevent zero-day exploits and extend our mission model.
• How much will it improve?- New capability in some cases
Mission-based cost benefit trade offs and dynamic reallocation of resources after system failure.
- In others automating a process currently done by hand. Automated learning/generalization of exploits can’t increase the
runtime.
13 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
CIRCA Planning: Generalized Semi-Markov Decision Processes
• Most existing decision-theoretic planning systems are based on the Markov Decision Processes and have difficulty handling multiple, asynchronous events.
• GSMDP provides the most natural framework for this purpose.
• CIRCA (Cooperative Intelligent Real-Time Control Architecture) is the first GSMDP planner.
Uses the decision-theoretic principle of maximizing expected utility (e.g. to trade off performance against safety).
Uses rich stochastic models of world, transitions, actions, and time to construct best defense plans.
Does not hand-build, but automatically synthesizes plans and can thus adapt to defend against combinations/mutations of existing exploits based on the mission model.
14 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Exploratory Experiments With New Heuristics
1870
1880
1890
1900
1910
1920
1930
1940
1950
100 1000 10000 100000 1e+06 1e+07
Exp
ect
ed
Utilit
y
Time (ms)
OriginalNew
Optimal plan found inonly 1% time.
15 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Active Learning Approach
• We know an attack succeeded, but we don’t know why.- Need an educated guess as to culprits, then validate
• We want to explore the possibilities.- Model normal mission traffic according to several axes of
variability Score each attack according to these axes Experiment for most suspicious values
Build Model of Normal Traffic
Use model to identify suspicious elements
ExperimentHistorical(Normal)Queries
Attack Query
Learner
Blocking Rules
Model of
Normal
16 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Axes of Variability (Definition)
• The different ways an attack (e.g. to MySQL) might be formulated.- Provided a priori by a domain expert
• Query content- Word order (e.g. some permutations cause MySQL to crash)
- Binary machine instructions
- Unusual payload (e.g. unix commands, registry keys, database administrative commands)
• Query length (single/multiple terms)• Resource consumption patterns• Probing (e.g. password guessing)• Session-wide (multiple queries)
17 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Axes of Variability (detectors)
• We currently detect: - Text Queries:
String length: overall query length SQL parser: term length (table, col, row), number of terms (table, row,
col) Word order: if there are no other suspicious elements in the attack
- Packet Content (Using hex values of the packet): Command type Joint probability: each command has different expected byte patterns
• Plausible to design new or better detectors:- String content (e.g. look for hex)- Unusual payload (parser, keyword filter, content-filtering)- Session-wide (frequent patterns analysis)- Word order (patterns analysis)- More Joint probability distributions (e.g. strings may generally
be long, but passwords are short)
18 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Modeling Mission Traffic
• Model normal traffic- Domain expert creates measures for each axis of variability (or combinations
thereof)- Currently stored as histograms of the computed values- e.g.
• Compare attack to the model- Compute a “suspicion score”: how unusual is this attack for this axis of
variability- Score = (value – μ) / σ- e.g. add “32” to above histogram;
score is 12.6
• Experiment (Active Learning)- Sort the suspicion scores.- Experiment in the “most” suspicious axis first.
Test hypothesis of culprit Discover the boundary conditions
19 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
TastersTastersTasters
Replicator
Delete tasters
Create tasters
Switch Tasters
Replicate queries
Heartbeat Status
RTSReplicate
Switch Tasters
Rebuild Tasters
Send to Learning
.
AMP
CSM
Once per phase
Proxy (Dexter)Block known bad queries
Taste test
Log results
Master DBQuery
LearnerRead Training Data
Experiment
Generate Rules
Normal Query
Cortex Demo Architecture and Use Cases
New AbilitySince January Demo
20 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
TastersTastersTasters
Replicator
Delete tasters
Create tasters
Switch Tasters
Replicate queries
Heartbeat Status
RTSReplicate
Switch Tasters
Rebuild Tasters
Send to Learning
.
AMP
CSM
Proxy (Dexter)Block known bad queries
Taste test
Log results
Master DBQuery
LearnerRead Training Data
Experiment
Generate Rules
Attack gets throughAttack is blocked
Cortex Demo Architecture and Use Cases
21 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Risks and Mitigations
• Search space for the controller is large.- Mitigation: Our work is focused on heuristics that are
solving these problems without covering the whole space.
- Mitigation: Plans are built offline before system commission.
- Mitigation: Possible to build plans that provide safe reduced functionality states to move to while regenerating.
• Identification of a covering set of axes of variability and experimentation methods.- Mitigation: Fixed protocols have limited degrees of freedom
making it easier to enumerate.
- Mitigation: Axis only has to be identified once .
• “Binary poisons”- Mitigation: Don’t eat the second half of the poison.
- Mitigation: Use program diversity tools to push some code level violations that would be binary poisons into our space.
What Progress Have We Made?
23 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Current Demo
• Complete system protecting MySQL with normal background traffic for our mission model.
• All three end-to-end use cases.- Integration of CIRCA planning for single mission phase.- Learning via active experimentation.
• Two different attacks in Mission Model Phase 1- Both kill MySQL 3.23.49 server but are very different.
Password buffer-overflow (BID 8590)• Exploits the lack of bounds-checking on the password field for a user.
Binary attack against COM_TABLE_DUMP command (BID 6368)• Exploits a casting vulnerability of signed integer values. • Sends a negative value as one of the string length parameters to the
command, and corrupts internal MySQL memory.
24 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Major Accomplishments Since Last Meeting
• Infrastructure- Mission model defined.- 2 Specific attacks identified and implemented.- Background traffic generators built.- Improved & new GUIs developed.- Extending to Apache/PHP server (in progress).- Identifying Apache exploits (in progress).
• Learning - Design and Implementation of learning architecture and active-
experimentation approach. Identification of axes of variability. Traffic modeling.
- Successful testing against 2 very different attacks.
• Planning- Successful exploratory experiments on new planning heuristics.
How Are We Performing Against Our Metrics?
26 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Program Level Metrics
• Correctly diagnosing root cause 10% of attacks.- System correctly diagnoses all of the attacks covered by an
identified axis of variability.
- Question becomes one of coverage of the axes of variability relative to the set of attacks.
• Correctly responding to 5% of diagnosed attacks- CORTEX plans controllers that respond to all (100%) of the
possible attacks enumerated within the mission model.
- Learning blocking rules for all (100%) of the diagnosed attacks.
27 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Project Metrics
• Time to synthesize optimal controller (measured)- Experiments have cut the time to find the optimal plan for
this problem by >90%. Continuing experiments in other problems to identify efficacy of the method.
• Accuracy of the rules learned (measured)- Anecdotal analysis suggests no false positives; performing
measurements against traffic model.
• Overhead added to normal processing (measured)- Anecdotal evidence suggests this is not too bad, but
awaiting final system configuration to measure.
• Improved throughput from increased mission specific system availability. (measured)
28 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
Expected Major Achievements
• Prove the viability of automatically synthesizing a meta-level controller for model-based system response and regeneration.- Limited impact on current users
- Over all improved system throughput from increased system mission sensitive availability.
- Demonstrate system scalability in two mission phases
• Demonstrate system planning for at least two different mission phases with significantly different requirements and responses.
• Demonstrate the viability of learning zero-day exploits within well understood protocols for at least two different protocols.
Significant Research Results
29 HONEYWELL - CONFIDENTIAL SiteVisitOverview.ppt
SCHEDULE
CORTEX – Mission-Aware Closed-Loop Cyber Assessment and Response
• System Reference Model drives intrusion assessment, diagnosis, and response.
• Automatically search for response policies that optimize tradeoff of security against mission ops.
• “Taste-tester” server redundancy supports robustness and learning from new attacks.
• High confidence intrusion assessment and diagnosis.
• Pre-planned automatic responses to contain and recover from faults and attacks.
• Automatic tradeoffs of security vs. service level & accessibility.
• Learns to recognize and defeat novel attacks.
Computing services
Active Security ControllerExecutive
Controller Synthesis ModuleNetworks, Computers
Attacks, intrusions
IMPACT
NEW IDEASSecurity Tradeoff Planner
Scyllarus Intrusion
Assessment
Demos:Thin slice
demo
Jan 05 Jul 05
Learning Demo
Dec 05
Mission Aware Learning and
Response Demo
May 05
PI Meeting Demo
End
Come See the Demo