Revealing the Characteristics of Cyber Analysts’ Reasoning Processes: A Trace Analysis Approach Annual Review ARO MURI on Computer-aided Human-centric Cyber SA October 29, 2013 Pennsylvania State University John Yen Chen Zhong Peng Liu Army Research Laboratory Robert Erbacher Steve Hutchinson Renee Etoty Hasan Cam William Glodek
45
Embed
Pennsylvania State University John Yen Chen Zhong Peng Liu
Revealing the Characteristics of Cyber Analysts’ Reasoning Processes: A Trace Analysis Approach Annual Review ARO MURI on Computer-aided Human-centric Cyber SA October 29, 2013. Pennsylvania State University John Yen Chen Zhong Peng Liu. Army Research Laboratory Robert Erbacher - PowerPoint PPT Presentation
Welcome message from author
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Revealing the Characteristics of Cyber Analysts’ Reasoning Processes:
A Trace Analysis Approach
Annual ReviewARO MURI on Computer-aided Human-centric Cyber SA
October 29, 2013
Pennsylvania State UniversityJohn Yen
Chen ZhongPeng Liu
Army Research LaboratoryRobert ErbacherSteve Hutchinson
Renee EtotyHasan Cam
William Glodek
Objectives:• Understand the analytical reasoning process of
cyber analysts• Capture the analytical reasoning trace of cyber
analyst through non-invasive tool• Develop a model of analytical reasoning process
that can capture rich trace and enable automated trace analysis
• Conduct experiments involving cyber analysts
Scientific/Technical Approach• Developed Observation-Hypothesis-ActionHypothesis
(OHA) model of analytical reasoning process• Developed and implemented Analytical Reasoning Support
Tool for Cyber Analysis (ARSCA)• Designed experiments that capture realistic challenges in
cyber SA using VAST 2012.• Collaborated with an ARL study about visualization of
cyber SA led by Dr. Erbacher.• Conducted multiple pilot studies (at Penn State and Army
Research Lab) to polish ARSCA
Accomplishments• Conducted experiments, in collaboration with Army Research
Lab, involving subjects from Penn State and ARL.• Initial case study about trace analysis provided new insights
about the reasoning process of analysts• Initial correlation analysis suggest relationship between
characteristics of traces and performance/expertise
Opportunities• Improve performance of analysts through OHA-based training• Investigate the difference strategies between experts and novice• Investigate using aggregated analyst experiences to support
analytical reasoning process.
Computer-Aided Human Centric CyberSituation Awareness
J. Yen, C. Zhong, P. Liu, R. Erbacher, S. Hutchinson, R. Etoty, H. Cam, W. Glodek
System Analysts
Computer network
SoftwareSensors, probes• Hyper Sentry• Cruiser
Mu
lti-
Sen
sory
Hu
man
C
om
pu
ter
Inte
ract
ion
• Enterprise Model• Activity Logs • IDS reports
• Vulnerabilities
Cognitive Models & Decision Aids• Instance Based Learning Models
• Simulation• Measures of SA & Shared SA
• • •
Da
ta C
on
dit
ion
ing
As
so
cia
tio
n &
Co
rre
lati
on
Automated Reasoning Tools• R-CAST• Plan-based
narratives• Graphical
models• Uncertainty
analysis
Information Aggregation
& Fusion• Transaction Graph methods
•Damage assessment
Computer network
• •
•
Real World
Test-bed
3
4
Year 4 Accomplishments at a GlancePublications: 1. Zhong, C., Kirubakaran, D.S., Yen, J., Liu, P.,
Hutchinson, S., & Cam, H., “How to Use Experience in Cyber Analysis: An Analytical Reasoning Support System”, in Proceedings of IEEE Conference on Intelligence and Security Informatics (ISI), 2013.
2. Chen, P.C., Liu, P., Yen, J., & Mullen, T., “Experience-based cyber situation recognition using relaxable logic patterns”, in IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), pp. 243-250, 2012.
3. Chen Zhong, VAST 2013 Workshop Presenter4. Working papers for CogSIMA 2014
Tools: • ARSCA
Technology transfer:
• J. Yen as summer faculty fellow at ARL • Deep collaborations with ARL researchers: • Brought the ARSCA toolkit to
Adelphi site • 12 ARL security analysts
participated• Weekly teleconferences• Joint work on a series of
papers •Invention Disclosure to PSU
Awards: • Best Paper Award, CogSIMA 2012.• Chen Zhong: Grace Hopper Celebration of Women in
• Continue to conduct, in collaboration with ARL researchers, Analytical Reasoning Experiment (VAST 2012)
• Analyze the traces of analytical reasoning• Is the first thought important for an analyst’s performance?• How will the key observation influence the analytical reasoning process?• What are the differences between strategies used by experts and novice?
• Design and conduct, in collaboration with ARL researchers, a collaborative analytical reasoning experiment