Army Department of Defense DASD - DT&E / TRMC CTEIP Air Force Cyber TASE Navy DISA UNCLASSIFIED//FOUO UNCLASSIFIED//FOUO Cyber Test Analysis and Simulation Environment (TASE) August 20, 2015 UNCLASSIFIED//FOR OFFICIAL USE ONLY Michael Winslow Rich Wride Donn Puckle Cyber TASE PM Cyber TASE DPM Cyber TASE Army PM SPAWARSYSCENPAC 96 TW/46 TS USAEPG/ATEC [email protected][email protected][email protected]619-553-0341 850-882-0765 520-538-4830
54
Embed
Cyber Test Analysis and Simulation Environment (TASE)itea.org/images/pdf/conferences/2015_Symposium/Presentations/Wride Track 1.pdf · Cyber TASE provides integrated instrumentation
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
UNCLASSIFIED//FOUO
Cyber Test Analysis and Simulation Environment (TASE)
• Revision of DoDI 5000.02: Issued 6 Jan 2015 – New/better guidance for both developmental and operational testing of IT
• Revision of DoD 8500.01, Cybersecurity: 14 Mar
2014
– Expanded scope and specificity
• DoDI 8510.01 – Risk Management Framework (RMF)
for DoD IT: 14 Mar 2014 – Provides policy, clarity and guidance on the RMF and compliance
• Four Phased Cybersecurity DT&E Process: In Work– Incorporated into Defense Acquisition Guidebook Chapter 9
• OSD DOT&E- Procedures for Operational Test and
Evaluation of Cybersecurity in Acquisition Programs:
01 Aug 2014– Formalizes OT&E Phases
• Cybersecurity Implementation Guidebook for PMs– Address Cybersecurity T&E across the acquisition lifecycle
• Cybersecurity T&E Guidebook planned– To provide detailed Cybersecurity T&E guidance for DT/OT Community
Current DT&E Cybersecurity Guidance
Phase 1: Understand
Cybersecurity Requirements
Phase 2: Characterize Cyber Attack
Surface
Phase 3: Cooperative Vulnerability Identification
Phase 4: Adversarial
Cybersecurity DT&E
Understand Cybersecurity
requirements and develop
an approach for
cybersecurity T&E
Characterize the attack
surface; in the integrated
environment, determine
possible threat vectors.
Analyze and evaluate
potential vulnerabilities to
determine measures to
improve resilience.
Cybersecurity DT&E event in a
realistic mission environment,
with use of cyber range,
CNDSP, representative users
and Cybersecurity threat
representation.
MS B
TechnologyMaturation &
CDD Risk Reduction
SRR SFR PDR CDR
MS CATO
MS A
Engineering &ManufacturingDevelopment
TRR EventSVR DT&E
ASR
MaterielSolution AOA
DRAFT
AnalysisMDD CDD CPD
IATT
DT&E
Req Decision
Pre-EMD
DT&EAssess-
ment
Cyber TASE will aid in performing both the functional and Cybersecurity DT testing in EMD phase.
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Cybersecurity OT Requirements
Phase 5: Cooperative Vulnerability and
Penetration Assessment
Phase 6: Adversarial Assessment
This phase assesses the ability of a unit equipped with a system to support its missions while withstanding validated and representative cyber threat activity.
Provide a comprehensive characterization of the cybersecurity status of a system in a fully operational context, and to substitute for reconnaissance activities in support of adversarial testing when necessary
O&SProduction andDeploymentCPD
Assess- OTRR IOT&Ement
Full Rate Production
Decision ReviewMS C
ATO
Cyber TASE will greatly aid in the analysis required for both Phases of theOSD-DoT&E Cybersecurity Test Memo levied upon acquisition programs.
Engineering &ManufacturingDevelopment
Memorandum from
Dr. J. Michael Gilmore (DOT&E)
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Cybersecurity OT Requirements
Phase 5: Cooperative Vulnerability and
Penetration Assessment
Phase 6: Adversarial Assessment
This phase assesses the ability of a unit equipped with a system to support its missions while withstanding validated and representative cyber threat activity.
Provide a comprehensive characterization of the cybersecurity status of a system in a fully operational context, and to substitute for reconnaissance activities in support of adversarial testing when necessary
O&SProduction andDeploymentCPD
Assess- OTRR IOT&Ement
Full Rate Production
Decision ReviewMS C
ATO
Cyber TASE will greatly aid in the analysis required for both Phases of theOSD-DoT&E Cybersecurity Test Memo levied upon acquisition programs.
Engineering &ManufacturingDevelopment
Memorandum from
Dr. J. Michael Gilmore (DOT&E)
“All oversight systems capable of sending or receiving digital information are required to conduct cybersecurity testing. This includes uploading or downloading data by physical means such as Universal Serial Bus (USB) connections or removable data devices.”
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Integrated Test Concept
Phase 1: Understand
Cybersecurity Requirements
Phase 2: Characterize Cyber Attack
Surface
Phase 3: Cooperative Vulnerability Identification
Phase 4: DT&E Cyber Aggressor
Team
Phase 5: Cooperative Vulnerability and
Penetration Assessment
Phase 6: Adversarial Assessment Cyber
Red Team
DT
OT
Cyber TASE
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Cyber Testing Capability Gaps
• Two gaps addressed by Cyber TASE• Lack of effective IT and Data Link testing
– Runs the Centralized T&E Improvement Program (CTEIP) – Cyber TASE
– Runs the T&E S&T Program - CTT
– Runs the Joint Mission Environment Test Capability (JMETC) Program - JMN
Secretary of Defense(SECDEF)
Under Secretary of Defense – Acquisition, Technology,
Logistics (USD-AT&L)
Assistant Secretary of Defense – Research and
Engineering(ASD-R&E)
Assistant Secretary of Defense – Developmental
Test and Evaluation(ASD-DT&E)
Test Resource Management Center
(TRMC)
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Relationship within TRMC
21
TRMC
Service
Improvement &
Modernization/
Programs
Acquisition Programs /
Advanced Concept
Technology
Demonstrations
T&E Multi-Service /
Agency Capabilities
Risk mitigation needsTechnology shortfalls
Risk mitigation solutions
Advanced development
Requirements
Capabilities
DoD Corporate
Distributed Test
Capability
TRMCJointInvestmentPrograms
6.3 6.4 6.5
DoD Corporate
Cyber
George
Rumford
Chris Paust Chip
Ferguson
Derrick Hinton
DASD-DT&E
DT&E
Dr. C. David
Brown
Dr. C. David
Brown
TRL 3 - 6 TRL 7 - 9 TRL 9
Cyber TASE
CTTJMN
NCR
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
AT&L, DT&E / TRMC Organization
Chris
Paust
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
DOT&E, ECRE Active Requirements
• Purpose of ECRE ACTIVE is to:• Create a rapidly reconfigurable LVC simulated cyberspace environment / solution
to address various SUTs.
• Use ECRE ACTIVE to assess DOD C4I system compliance with Department of Defense Instruction (DODI) 8500.01, Cybersecurity, dated 14 March 2014.
• Use ECRE ACTIVE to assess DOD C4I system compliance with the National Institute of Standards and Technology (NIST) Special Publication (SP) 800-53, Security and Privacy Controls for Federal Information Systems and Organizations, dated 30 April 2013.
• Ascertain the suitability, survivability, effectiveness and interoperability of C4I systems under operationally realistic adversarial cyber threats and heavy network traffic loads.
• Characterization of an SUTs attack surface by the execution of fuzzing conditions (e.g., providing invalid, unexpected, or random data to the inputs of a SUT)
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
DOT&E, ECRE Active Solutions
• Enable Cyber Opposing Force (OPFOR) to inject enumerated operationally realistic adversarial cyberspace threats against SUTs
• Monitor and measure network cyberspace activity
• Measure C4I system effectiveness (detection, protection, interoperability with other C4I systems, and restoration processes) against cyber attacks in a burdened network
• Assess ECRE ACTIVE tools and C4I compliance with DOD cybersecurity policies per DODI 8500.01 or NIST SP 800-53
• Collect system and mission data and maintain a central repository
• Interpret collected data against assessment criteria to generate cyberspace vulnerability and C4I effectiveness and interoperability assessment reports.
• Develop and refine ECRE ACTIVE processes and measurement methodology.
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
ECRE ACTIVE Operational View
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Planning
26
Planning Environment Execution Analysis
Cyber TASE Unique Capabilities1. TASE will address basic Cyber Security Metrics development 2. Planning process used for data collection & analysis plan will be used to develop
• Develop integrated data collection across NW, host, & SUT applications• Automated analysis of threat progress & effect on SUT• Configurable user friendly tool set – consistent user interface & simple installation • Based on distributed test concept
2. Visualization and Situation Awareness Tools
Network environment
Instrumentation / Detection
Attack methods Grey Network Data AnalysisUser
environment / interface
Distributed connections
CTT project Provided by JMETC
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Execution
28
Planning Environment Execution Analysis
Cyber TASE Unique Capabilities1. Track threats through NW to application level2. Assess effect of threat on CIA* for SUT
Non-threat operation
AttackDetection /
Threat Identification
Blue Team Assessment -
Human Factors
Near-Real-Time Analyze
Data
Reset systems to known
configuration
Daily Reports Generated /
Feedback
Input from CTT project
* Confidentiality, Integrity, & Availability
Provide path
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Analysis
29
Planning Environment Execution Analysis
Cyber TASE Unique Capabilities1. Correlate data – locally & distributed2. Automate analysis3. Visualize and provide situation awareness4. Provide report inputs for risks & deficiencies
Document Attacks used
Analyze DataThreat
Identification
System response to
threat
Operator response to
threat
Determine Risks
Reports Generated /
Feedback
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
T&E Reliance Process
Proposal Submission
Concept Refinement / Briefing Phase
Technical Subject Matter
Experts (TSMEs)
Test Resource Activity Group
(TRAG)
Board of Directors
(Executive Secretariat
Staff) (BoD(ESS))
Board of Directors
(Executive Secretariat) (BoD(ES))
Project Initiation
Submit Proposal Online
Combine Proposal with Similar Joint
Efforts
TSMEs Review Proposals
Form Joint Proposal Team
400+
10
4
2Receive T&E
Executive Approval
Assign Service Lead
Brief TRMC / CTEIP Program
Enter Pre-Phase 0 (Risk
Reduction)
Proposals
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Functional ArchitectureInstrumentation
31
Instrumentation
Network Data Collection
Collect IP Data
Collect TDL
Host Data Collection
Collect Systems Data
Collect Operator Data
Ground Truth Data Collection
Collect Red Team Ground Truth
Collect Operator Ground Truth
Correlation and Data Analysis
Conduct near-Real-Time Analysis
Conduct Post-Test Analysis
Near-Real-Time Visualization
Visualize System Data
Visualize Operator Data
Post-Test Visualization
Visualize System Data
Visualize Operator Data
Cyber TASE
• Provides integrated instrumentation for collecting, analyzing, and visualizing the test data across multiple layers/sources to understand the mission impacts in a Cyber contested environment.
• Provides constructive simulation to scale L-V-C environment so we can represent a full scaled operational environment and the impact of Cyber threats on conducting mission operations.
Army Department of Defense
DASD-DT&E / TRMC
CTEIP
Air Force
Cyber TASENavy
DISA
UNCLASSIFIED//FOUO
Functional ArchitectureConstructive Simulation
32
Constructive Simulation
(STEALTHNET)
Host Models
Vulnerabilities of Host
Host Topology Pallet
Network Models
Vulnerabilities of Network
Infrastructure
Network Topology Pallet
Threat ModelsInterface to Live/Virtual
Instrument Simulation
Visualize Data
Cyber TASE
• Provides integrated instrumentation for collecting, analyzing, and visualizing the test data across multiple layers/sources to understand the mission impacts in a Cyber contested environment.
• Provides constructive simulation to scale L-V-C environment so we can represent a full scaled operational environment and the impact of Cyber threats on conducting mission operations.