DIRECTION OF CYBER TEST AND EVALUATION IN THE US AIR …...traditional avionics test expertise, computer network penetration expertise, and advanced avionics penetration testers Developing

DIRECTION OF CYBER TEST AND EVALUATION IN THE US AIR FORCEITEA 6TH CYBERSECURITY WORKSHOP7 – 8 MARCH 2018FT WALTON BEACH, FL

Joseph Nichols, PhDTechnical Advisor for Flight Test and EvaluationAir Force Test CenterEdwards AFB [email protected]

DISTRIBUTION STATEMENT A. Approved for public release; distribution unlimited.412TW-PA-18080

FUNDAMENTAL GOAL OF CYBER T&E

CYBER SURVIVABILITYIN A CYBER-CONTESTED ENVIRONMENT (ACTIVE THREAT)

Prevent attack Mitigate attack Identify, Respond, & Restore from attack

2

CYBER T&E – AIRCRAFT WEAPON SYSTEMS 3

USAF TEST INFRASTRUCTURE 4

Crystal City, VABeale AFB

Greenville, TX

Melbourne, FL

Kirtland AFB

Peterson AFB

Gunter Annex

Holloman AFB

Hanscom AFB

White Sands Missile Range

Lackland AFB

Denver CO

Nellis AFB

Waco, TX

Langley AFB

Boeing Field

Wright-Patterson AFB

96 TWEglin AFB

Charleston AFB

NAS Patuxent River

Tinker AFB

Tucson, AZAF Plant 04

Warner-Robins AFBHurlburt Field

JB Lewis-McChord

Wright-Patterson AFB

Hill AFB

Creech AFB

412 TW

Edwards AFB

Moffett Field

White Oak, MD

AEDCArnold AFB

HQ AFTCHQ AFOTEC

VISION AND GOALS

Vision Cyber resiliency testing will be modeled, measured,

executed, and evaluated just like other threats to the mission.

Goals Develop a robust & integrated cyber test force Develop a cyber infrastructure to test traditional IT

and embedded weapons systems for potential cyber vulnerabilities

6

MANPOWER REQUIREMENTS

Cyber T&E expertise for aircraft and weapons requires a merge of traditional avionics test expertise, computer network penetration expertise, and advanced avionics penetration testers

Developing DoD cyber training courses Hiring penetration testers and embedded system penetration SMEs Standing up new test organization dedicated to all aspects of cyber

test and evaluation – networks, aircraft, weapons

7

96TH CYBERSPACE GROUP9

96TH CYBERSPACE TEST GROUP

Air OperationsC2

96 Cyberspace Test Group

45 Test Squadron 46 Test Squadron47 Cyberspace Test Squadron

(Lackland)Support Division

96 Cyberspace Test Group, Det 1

(Hanscom)

Space & IntelC2

Mission Planning Systems

Business Enterprise Systems

(Gunter Annex)

C4I Systems

Sensors & Defensive Systems

Tactical & Strategic Datalinks

JSTARS(Patrick OL)

IT & Avionics Cybersecurity

Avionics Cybersecurity (Edwards OL)

Offensive & Defensive Cyber

(Lackland)

Programming Engineers

Budget & Finance

Logistics, Security and IT

GSUEglin

10

CONGRESSIONAL INTEREST – NDAA FY16, SECTION 1647/1649

National Defense Authorization Act, Fiscal Year 2016, Section 1647 (DoD Major Weapon Systems) NDAA FY16, Sec 1647. Evaluation of cyber vulnerabilities of major weapon systems

of the Department of Defense

EVALUATION REQUIRED – Complete evaluation not later than December 31, 2019

National Defense Authorization Act, Fiscal Year 2017, Section 1649 (Adds F-35) NDAA FY17, Sec. 1649. Evaluation of cyber vulnerabilities of F–35 aircraft and

support systems

EVALUATION – Complete evaluation not later than 120 days after the date of the enactment of this Act, under NDAA FY16, Sec 1647

USAF 50 systems – AFTC and AFOTEC (evaluations in progress)

Report through CROWS to USD(AT&L) to SECDEF to Congress

11

12

AFRL – Assessment methodology and support developing mitigation efforts

AFTC – Leverage weapon system expertise

AFOTEC – Existing scheduled OTA efforts

24th AF – Cyber Protection Team support

Intel – Threat input to risk analysis

Red Teams – Performing CVPAs

AO Teams – Leverage existing ATO docs and augment certification process

PEOs, Program Offices, and users – Support of CSRAs

NDAA 1647 Partnerships

13

7 Lines of Action (LOAs)LOA 1: Perform Cyber Mission Thread AnalysisLOA 2: “Bake-In” Cyber ResiliencyLOA 3: Recruit, Hire & Train Cyber WorkforceLOA 4: Improve Weapon System Agility & AdaptabilityLOA 5: Develop Common Security EnvironmentLOA 6: Assess & Protect Fielded FleetLOA 7: Cyber Intel Support

Cyber Squadron InitiativesCyber Resiliency Office for Weapon Systems (CROWS)Test & Evaluation (infrastructure & capability growth)Industrial Control Systems/SCADA cyber protection measures

People, Processes, & Products

13AF Cyber Campaign Plan:Weapon System Focus

COOPERATIVE RESEARCH AND DEVELOPMENT AGREEMENT (CRADA)

The CRADAs are an enabler for any organization within the DoD to interact with a contractor outside of a program of record regarding aircraft cyber. Contractors with a valid clearance/need to know, should have

equal opportunity to collaborate on contractor/government aircraft cyber efforts.

With the appropriate clearances, contractors deserve the right to know vulnerabilities applicable to their avionics systems.

Contractors have subject matter experts with a wealth of knowledge of the avionics systems that are critical to helping fix cyber vulnerability issues.

14

INTEGRATING CYBER T&E INTO TRADITIONAL ACQUISITION AND T&E PROCESSES

15

AVIONICS CYBER RANGE (ACR) 16

AVIONICS CYBER TEST INFRASTRUCTURE

• DoD test facility capable of conducting cyber testing compatible with the unique features of aircraft avionics and airborne munitions

• Center of Excellence for avionics cyber T&E and developer of cyber test techniques and test tools, including automated test tools

• Connected with the NCR and other aircraft and weapons cyber test facilities

• Construction to begin in 2020• Test operations to begin in 2022

18

19